Details

Reviewers

Group Reviewers

Commits

rDSTOCf734c4e4ded3: api.server: Remove unneeded stats.increment call
rDSTOC3c2dfc00232b: swh.storage.api.server: Add metrics to storage endpoints
rDSTOCc47f92a00344: storage: Add a timed decoration to count calls and time it
rDSTOC3a7f8a9b1393: Add @timed decorator around all api.server endpoints
R65:3c2dfc00232b: swh.storage.api.server: Add metrics to storage endpoints
R65:f734c4e4ded3: api.server: Remove unneeded stats.increment call
R65:c47f92a00344: storage: Add a timed decoration to count calls and time it
R65:3a7f8a9b1393: Add @timed decorator around all api.server endpoints
rDSTOf734c4e4ded3: api.server: Remove unneeded stats.increment call
rDSTO3a7f8a9b1393: Add @timed decorator around all api.server endpoints
rDSTO3c2dfc00232b: swh.storage.api.server: Add metrics to storage endpoints
rDSTOc47f92a00344: storage: Add a timed decoration to count calls and time it

Summary

Add @timed decorator around all storage.api.server endpoints.

Test Plan

from docker-dev:

Start storage server:

docker-compose down --volumes; SWH_SERVICE=swh-storage; docker-compose build $SWH_SERVICE; docker-compose up $SWH_SERVICE

Then in another shell:

docker-compose exec swh-storage bash -c 'ngrep -d lo udp and port 8125

Run some ingestion and check we see counters like the following:

U 127.0.0.1:39597 -> 127.0.0.1:8125
  swh_storage_request_duration_seconds:0.003577995812520385|ms|#endpoint:get_storage
#
U 127.0.0.1:52223 -> 127.0.0.1:8125
  swh_storage_request_count:1|c|#endpoint:snapshot_add
#
U 127.0.0.1:35545 -> 127.0.0.1:8125
  swh_storage_request_duration_seconds:0.005665002390742302|ms|#endpoint:get_storage
#
U 127.0.0.1:39042 -> 127.0.0.1:8125
  swh_storage_request_count:1|c|#endpoint:fetch_history_end
#
U 127.0.0.1:55143 -> 127.0.0.1:8125
  swh_storage_request_duration_seconds:0.0034950062399730086|ms|#endpoint:get_storage
#
U 127.0.0.1:56208 -> 127.0.0.1:8125
  swh_storage_request_count:1|c|#endpoint:origin_visit_update
#
U 127.0.0.1:58816 -> 127.0.0.1:8125
  swh_storage_request_duration_seconds:0.003449007635936141|ms|#endpoint:get_storage
(swh2)

Also tox is fine for non regression.

Diff Detail

Repository

rDSTO Storage manager

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Mar 27 2019, 8:15 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tox/227/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tox/227/console

Harbormaster failed remote builds in B4902: Diff 4147!Mar 27 2019, 8:16 PM

Barf, wrong machine...

Add missing commit

(because diff'ed from the wrong machine)

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tox/228/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tox/228/console

Harbormaster failed remote builds in B4903: Diff 4148!Mar 27 2019, 8:21 PM

swh.storage.api.server: Add metrics to storage endpoints

(for real this time)

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tox/229/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tox/229/console

Harbormaster failed remote builds in B4904: Diff 4149!Mar 27 2019, 8:23 PM

Fix pep8 violation

ardumont edited the test plan for this revision. (Show Details)Mar 27 2019, 8:27 PM

ardumont added inline comments.

swh/storage/api/server.py
26	@douardda that's one of the reason why it did not work (as per our discussion)
49	I had to invert the order of decorator because the app.route decorator, as far as i understood it, does not use the functool.wraps so we lose information about the decorated function.