Page MenuHomeSoftware Heritage

db: Add table checked_ranges
ClosedPublic

Authored by vlorentz on Oct 4 2022, 2:17 PM.

Details

Summary

It will be used by the storage_checker to 'remember' what ranges
it already checked recently across runs (and crashes) for T4527, and to
monitor progress.

Diff Detail

Repository
rDSCRUB Datastore Scrubber
Branch
checkpoint
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 32063
Build 50202: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 50201: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8608 (id=31093)

Rebasing onto 630001ce7b...

First, rewinding head to replay your work on top of it...
Applying: db: Add table checked_ranges
Changes applied before test
commit 7ca602c798b7a0f86e77a2354d13f0ef29c90847
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/63/ for more details.

ardumont added a subscriber: ardumont.

lgtm, a couple of typos to fix.

swh/scrubber/db.py
113
swh/scrubber/sql/30-schema.sql
36

more like?

swh/scrubber/sql/upgrades/4.sql
17

ditto

This revision is now accepted and ready to land.Oct 6 2022, 5:57 PM
vlorentz marked 2 inline comments as done.

fix typo in comment

Build has FAILED

Patch application report for D8608 (id=31197)

Rebasing onto 630001ce7b...

First, rewinding head to replay your work on top of it...
Applying: db: Add table checked_ranges
Changes applied before test
commit 74946c4fd5797ca6b5deb11e24a049b534694d91
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

Link to build: https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/66/
See console output for more information: https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/66/console

swh/scrubber/db.py
113

I disagree. The original meaning is:

Enregistre que l'intervalle donné a été vérifié pour la dernière fois à la date donnée

But your change is:

Enregistre que l'intervalle donné est celui qui a été vérifié pour la dernière fois à la date donnée

ie. I mean that the table is a range -> date map, but your change makes it look like it's a date -> range map.

Build is green

Patch application report for D8608 (id=31197)

Rebasing onto 630001ce7b...

First, rewinding head to replay your work on top of it...
Applying: db: Add table checked_ranges
Changes applied before test
commit 35d414feb9c451a3c4abf62d3caf26c159a297cb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/68/ for more details.

Build is green

Patch application report for D8608 (id=31208)

Rebasing onto 630001ce7b...

Current branch diff-target is up to date.
Changes applied before test
commit fef8a513dfead265788a24e5f580fbe9c746bd32
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Oct 4 14:17:12 2022 +0200

    db: Add table checked_ranges
    
    It will be used by the storage_checker to 'remember' what ranges
    it already checked recently across runs (and crashes), and to
    monitor progress.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/69/ for more details.

This revision was automatically updated to reflect the committed changes.