Page MenuHomeSoftware Heritage

Move metrics handling from backends to RPC server
ClosedPublic

Authored by vlorentz on Mar 2 2022, 12:18 PM.

Details

Summary

Motivation: replaces code duplication in the backends with a single one,
to be consistent with the objstorage (which has many more backends)

This also fixes the issue of metrics from 'extid_add' to be missing
when using the postgresql storage.

Test Plan

Not tested

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build has FAILED

Patch application report for D7276 (id=26351)

Rebasing onto 215162b2c1...

Current branch diff-target is up to date.
Changes applied before test
commit 572308d7ca186fbf58b75a11ec75136fb212ea71
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 2 12:17:07 2022 +0100

    Move metrics handling from backends to RPC server
    
    Motivation: replaces code duplication in the backends with a single one,
    to be consistent with the objstorage (which has many more backends)
    
    This also fixes the issue of metrics from 'extid_add' to be missing
    when using the postgresql storage.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1562/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1562/console

Harbormaster returned this revision to the author for changes because remote builds failed.Mar 2 2022, 12:19 PM
Harbormaster failed remote builds in B27226: Diff 26351!

Build is green

Patch application report for D7276 (id=26352)

Rebasing onto 215162b2c1...

Current branch diff-target is up to date.
Changes applied before test
commit 0cda6faaaab8fcdffe508ec8321f455614b032db
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 2 12:17:07 2022 +0100

    Move metrics handling from backends to RPC server
    
    Motivation: replaces code duplication in the backends with a single one,
    to be consistent with the objstorage (which has many more backends)
    
    This also fixes the issue of metrics from 'extid_add' to be missing
    when using the postgresql storage.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1563/ for more details.

Build is green

Patch application report for D7276 (id=26353)

Rebasing onto 215162b2c1...

Current branch diff-target is up to date.
Changes applied before test
commit 284a4ab3066956dc403c4ada700e8421990c0528
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 2 12:17:07 2022 +0100

    Move metrics handling from backends to RPC server
    
    Motivation: replaces code duplication in the backends with a single one,
    to be consistent with the objstorage (which has many more backends)
    
    This also fixes the issue of metrics from 'extid_add' to be missing
    when using the postgresql storage.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1564/ for more details.

ardumont added inline comments.
swh/storage/api/server.py
33

does that mean, all methods are timed ^ ?

39

Are those post_<method> are called after the main <method> is called?

ardumont added inline comments.
swh/storage/cassandra/storage.py
980

why does that change?

This revision is now accepted and ready to land.Mar 7 2022, 6:10 PM
swh/storage/cassandra/storage.py
980

because @timed decorators preventing mypy from seeing the type of self.content_missing_per_sha1_git didn't match.