Page MenuHomeSoftware Heritage

Move metrics handling from backends to RPC server
ClosedPublic

Authored by vlorentz on Mar 2 2022, 12:18 PM.

Details

Summary

Motivation: replaces code duplication in the backends with a single one,
to be consistent with the objstorage (which has many more backends)

This also fixes the issue of metrics from 'extid_add' to be missing
when using the postgresql storage.

Test Plan

Not tested

Diff Detail

Repository
rDSTO Storage manager
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 27228
Build 42596: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 42595: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D7276 (id=26351)

Rebasing onto 215162b2c1...

Current branch diff-target is up to date.
Changes applied before test
commit 572308d7ca186fbf58b75a11ec75136fb212ea71
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 2 12:17:07 2022 +0100

    Move metrics handling from backends to RPC server
    
    Motivation: replaces code duplication in the backends with a single one,
    to be consistent with the objstorage (which has many more backends)
    
    This also fixes the issue of metrics from 'extid_add' to be missing
    when using the postgresql storage.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1562/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1562/console

Harbormaster returned this revision to the author for changes because remote builds failed.Mar 2 2022, 12:19 PM
Harbormaster failed remote builds in B27226: Diff 26351!

Build is green

Patch application report for D7276 (id=26352)

Rebasing onto 215162b2c1...

Current branch diff-target is up to date.
Changes applied before test
commit 0cda6faaaab8fcdffe508ec8321f455614b032db
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 2 12:17:07 2022 +0100

    Move metrics handling from backends to RPC server
    
    Motivation: replaces code duplication in the backends with a single one,
    to be consistent with the objstorage (which has many more backends)
    
    This also fixes the issue of metrics from 'extid_add' to be missing
    when using the postgresql storage.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1563/ for more details.

Build is green

Patch application report for D7276 (id=26353)

Rebasing onto 215162b2c1...

Current branch diff-target is up to date.
Changes applied before test
commit 284a4ab3066956dc403c4ada700e8421990c0528
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 2 12:17:07 2022 +0100

    Move metrics handling from backends to RPC server
    
    Motivation: replaces code duplication in the backends with a single one,
    to be consistent with the objstorage (which has many more backends)
    
    This also fixes the issue of metrics from 'extid_add' to be missing
    when using the postgresql storage.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1564/ for more details.

ardumont added inline comments.
swh/storage/api/server.py
34

does that mean, all methods are timed ^ ?

40

Are those post_<method> are called after the main <method> is called?

ardumont added inline comments.
swh/storage/cassandra/storage.py
980

why does that change?

This revision is now accepted and ready to land.Mar 7 2022, 6:10 PM
swh/storage/cassandra/storage.py
980

because @timed decorators preventing mypy from seeing the type of self.content_missing_per_sha1_git didn't match.