Page MenuHomeSoftware Heritage

The checker now have the possibility to have multiple backup servers
ClosedPublic

Authored by qcampos on May 31 2016, 5:11 PM.

Diff Detail

Repository
rDSTO Storage manager
Branch
T304
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 146
Build 215: Software Heritage Python tests
Build 214: arc lint + arc unit

Event Timeline

qcampos retitled this revision from to The checker now have the possibility to have multiple backup servers.
qcampos updated this object.
qcampos edited the test plan for this revision. (Show Details)

Those changes are fully compatible with the previous version (a single remote storage considered as backup) but have a better opening to backup server's graph modifications.

Nice!

A small nitpick on the main changed function's docstring.
And one question about the backup order.

Please adapt it and it's good to go ^^

swh/storage/checker/checker.py
91–100

Maybe adding some more description about what's going on here.
That is from what i understood.

Try and restore some contents by asking nicely to backup storages (in the same order).
If the first one does not know the contents, the second one will be asked about it, and so on and so forth.
If some content is not repaired, we simply log the error.

102

Do we want to ask the backup always in the same order?

147

Maybe, randomize this to avoid asking always the same backup storage.

It's relevant only if for example, this is a cron which starts this regularly (or something similar).
Otherwise, the randomization can be done more deeply in the code (cf. previous mark).

ardumont added a reviewer: ardumont.
This revision now requires changes to proceed.May 31 2016, 9:30 PM
qcampos edited edge metadata.

Add the checker to the debian packaging paths

swh/storage/checker/checker.py
147

Due to the very large amount of content to verify, the checker will very likely run continuously. Given that, I suppose it's a better idea to have it as a daemon iterating again and again over the content.

qcampos edited edge metadata.

Add tag for the v0.0.40

ardumont edited edge metadata.
This revision is now accepted and ready to land.Jun 1 2016, 12:29 PM
qcampos edited edge metadata.
  • The checker now have the possibility to have multiple backup servers
  • Improve content integrity checker's test coverage
  • Add the checker to the debian packaging paths
This revision now requires review to proceed.Jun 1 2016, 1:20 PM
ardumont edited edge metadata.
This revision is now accepted and ready to land.Jun 1 2016, 1:23 PM
This revision was automatically updated to reflect the committed changes.