Page MenuHomeSoftware Heritage

db: Add table checked_ranges
ClosedPublic

Authored by vlorentz on Oct 4 2022, 2:17 PM.

Details

Summary

It will be used by the storage_checker to 'remember' what ranges
it already checked recently across runs (and crashes) for T4527, and to
monitor progress.

Event Timeline

Build is green

Patch application report for D8608 (id=31093)

Rebasing onto 630001ce7b...

First, rewinding head to replay your work on top of it...
Applying: db: Add table checked_ranges
Changes applied before test
commit 7ca602c798b7a0f86e77a2354d13f0ef29c90847
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/63/ for more details.

ardumont added a subscriber: ardumont.

lgtm, a couple of typos to fix.

swh/scrubber/db.py
112
swh/scrubber/sql/30-schema.sql
35

more like?

swh/scrubber/sql/upgrades/4.sql
17

ditto

This revision is now accepted and ready to land.Oct 6 2022, 5:57 PM
vlorentz marked 2 inline comments as done.

fix typo in comment

Build has FAILED

Patch application report for D8608 (id=31197)

Rebasing onto 630001ce7b...

First, rewinding head to replay your work on top of it...
Applying: db: Add table checked_ranges
Changes applied before test
commit 74946c4fd5797ca6b5deb11e24a049b534694d91
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

Link to build: https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/66/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/66/console

swh/scrubber/db.py
112

I disagree. The original meaning is:

Enregistre que l'intervalle donné a été vérifié pour la dernière fois à la date donnée

But your change is:

Enregistre que l'intervalle donné est celui qui a été vérifié pour la dernière fois à la date donnée

ie. I mean that the table is a range -> date map, but your change makes it look like it's a date -> range map.

Build is green

Patch application report for D8608 (id=31197)

Rebasing onto 630001ce7b...

First, rewinding head to replay your work on top of it...
Applying: db: Add table checked_ranges
Changes applied before test
commit 35d414feb9c451a3c4abf62d3caf26c159a297cb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/68/ for more details.

Build is green

Patch application report for D8608 (id=31208)

Rebasing onto 630001ce7b...

Current branch diff-target is up to date.
Changes applied before test
commit fef8a513dfead265788a24e5f580fbe9c746bd32
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/69/ for more details.

This revision was automatically updated to reflect the committed changes.