Page MenuHomeSoftware Heritage

db: Add table checked_ranges
ClosedPublic

Authored by vlorentz on Oct 4 2022, 2:17 PM.

Details

Summary

It will be used by the storage_checker to 'remember' what ranges
it already checked recently across runs (and crashes) for T4527, and to
monitor progress.

Diff Detail

Repository
rDSCRUB Datastore Scrubber
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D8608 (id=31093)

Rebasing onto 630001ce7b...

First, rewinding head to replay your work on top of it...
Applying: db: Add table checked_ranges
Changes applied before test
commit 7ca602c798b7a0f86e77a2354d13f0ef29c90847
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/63/ for more details.

ardumont added a subscriber: ardumont.

lgtm, a couple of typos to fix.

swh/scrubber/db.py
112
swh/scrubber/sql/30-schema.sql
35

more like?

swh/scrubber/sql/upgrades/4.sql
17

ditto

This revision is now accepted and ready to land.Oct 6 2022, 5:57 PM
vlorentz marked 2 inline comments as done.

fix typo in comment

Build has FAILED

Patch application report for D8608 (id=31197)

Rebasing onto 630001ce7b...

First, rewinding head to replay your work on top of it...
Applying: db: Add table checked_ranges
Changes applied before test
commit 74946c4fd5797ca6b5deb11e24a049b534694d91
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

Link to build: https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/66/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/66/console

swh/scrubber/db.py
112

I disagree. The original meaning is:

Enregistre que l'intervalle donné a été vérifié pour la dernière fois à la date donnée

But your change is:

Enregistre que l'intervalle donné est celui qui a été vérifié pour la dernière fois à la date donnée

ie. I mean that the table is a range -> date map, but your change makes it look like it's a date -> range map.

Build is green

Patch application report for D8608 (id=31197)

Rebasing onto 630001ce7b...

First, rewinding head to replay your work on top of it...
Applying: db: Add table checked_ranges
Changes applied before test
commit 35d414feb9c451a3c4abf62d3caf26c159a297cb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/68/ for more details.

Build is green

Patch application report for D8608 (id=31208)

Rebasing onto 630001ce7b...

Current branch diff-target is up to date.
Changes applied before test
commit fef8a513dfead265788a24e5f580fbe9c746bd32
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/69/ for more details.

This revision was automatically updated to reflect the committed changes.