Page MenuHomeSoftware Heritage

Add checking and reporting of references to missing objects
ClosedPublic

Authored by vlorentz on Aug 17 2022, 3:39 PM.

Diff Detail

Repository
rDSCRUB Datastore Scrubber
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 30841
Build 48224: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 48223: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8248 (id=29786)

Rebasing onto 3d999c5129...

Current branch diff-target is up to date.
Changes applied before test
commit 08f92de0fc261bf78980c974368963e4d314dfde
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Aug 17 15:38:16 2022 +0200

    Add checking and reporting of references to missing objects

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/50/ for more details.

anlambert added a subscriber: anlambert.

Looks good to me.

swh/scrubber/db.py
328

"Yields all objects missing from datastores." seems a better doctring here.

353

"Yields all referenced objects missing from datastores." seems a better doctring here.

swh/scrubber/storage_checker.py
109

missing dot at end of sentence.

186–190

Should this queries be executed in batch in case the number of objects to check is really large ? We do not have any timeouts on those though.

This revision is now accepted and ready to land.Aug 17 2022, 4:31 PM
vlorentz added inline comments.
swh/scrubber/db.py
328

copy-pasted too fast ;)

swh/scrubber/storage_checker.py
186–190

It's indirectly batched by the length of objects, so I'll keep it this way until it becomes an issue

vlorentz marked an inline comment as done.

fix docstring typos

Build is green

Patch application report for D8248 (id=29789)

Rebasing onto 3d999c5129...

Current branch diff-target is up to date.
Changes applied before test
commit 708b2b2ca73897c725cb257fdbfc6c430979aa4c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Aug 17 15:38:16 2022 +0200

    Add checking and reporting of references to missing objects

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/51/ for more details.