Page MenuHomeSoftware Heritage
Differential D8257

Add missing DB version 3
ClosedPublic
Actions

Authored by vlorentz on Aug 18 2022, 4:15 PM.

Details

Reviewers
olasd
Group Reviewers
Reviewers
Commits
rDSCRUB36d16bcdda42: Add missing DB version 3

Diff Detail

Repository
rDSCRUB Datastore Scrubber
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 30868
Build 48269: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 48268: arc lint + arc unit

Event Timeline

vlorentz created this revision.Aug 18 2022, 4:15 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptAug 18 2022, 4:15 PM
swh-public-ci added a comment.Aug 18 2022, 4:18 PM

Build is green

Patch application report for D8257 (id=29816)

Could not rebase; Attempt merge onto c6f711ecbf...

Updating c6f711e..286c4c5
Fast-forward
 swh/scrubber/db.py              |  2 +-
 swh/scrubber/sql/upgrades/3.sql | 43 +++++++++++++++++++++++++++
 swh/scrubber/storage_checker.py | 66 +++++++++++++++++++++++++++++------------
 3 files changed, 91 insertions(+), 20 deletions(-)
 create mode 100644 swh/scrubber/sql/upgrades/3.sql
Changes applied before test
commit 286c4c56a0a46de37db803522c16fe6376032508
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 16:15:47 2022 +0200

    Add missing DB version 3

commit 4e34e1d1bf1f3e05ff72220cfd93db5a0cad9770
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:19:43 2022 +0200

    Add metrics on references to missing objects

commit f1a61b1f501a5c3e342972c66688b08400dacea6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:12:37 2022 +0200

    Split batch_duration_seconds metric in two
    
    It will probably be useful to know what part of the check
    takes the most time.

commit 0cc86b0e45f015429406112b16e8b2e5196512ba
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:07:19 2022 +0200

    Deduplicate statsd calls
    
    This will allow adding more tags easily in future commits

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/56/ for more details.

Harbormaster completed remote builds in B30868: Diff 29816.Aug 18 2022, 4:18 PM
vlorentz requested review of this revision.Aug 18 2022, 4:18 PM
vlorentz updated this revision to Diff 29817.Aug 18 2022, 4:21 PM

Add missing dbversion rows

vlorentz updated this revision to Diff 29818.Aug 18 2022, 4:23 PM

remove them, they aren't actually needed anymore

swh-public-ci added a comment.Aug 18 2022, 4:24 PM

Build is green

Patch application report for D8257 (id=29817)

Could not rebase; Attempt merge onto c6f711ecbf...

Updating c6f711e..ac91a92
Fast-forward
 swh/scrubber/db.py              |  2 +-
 swh/scrubber/sql/upgrades/2.sql |  3 ++
 swh/scrubber/sql/upgrades/3.sql | 46 ++++++++++++++++++++++++++++
 swh/scrubber/storage_checker.py | 66 +++++++++++++++++++++++++++++------------
 4 files changed, 97 insertions(+), 20 deletions(-)
 create mode 100644 swh/scrubber/sql/upgrades/3.sql
Changes applied before test
commit ac91a929bea7d9b0a3b6564c5f06979c6cf74852
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 16:15:47 2022 +0200

    Add missing DB version 3

commit 4e34e1d1bf1f3e05ff72220cfd93db5a0cad9770
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:19:43 2022 +0200

    Add metrics on references to missing objects

commit f1a61b1f501a5c3e342972c66688b08400dacea6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:12:37 2022 +0200

    Split batch_duration_seconds metric in two
    
    It will probably be useful to know what part of the check
    takes the most time.

commit 0cc86b0e45f015429406112b16e8b2e5196512ba
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:07:19 2022 +0200

    Deduplicate statsd calls
    
    This will allow adding more tags easily in future commits

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/57/ for more details.

Harbormaster completed remote builds in B30869: Diff 29817.Aug 18 2022, 4:24 PM
swh-public-ci added a comment.Aug 18 2022, 4:26 PM

Build is green

Patch application report for D8257 (id=29818)

Could not rebase; Attempt merge onto c6f711ecbf...

Updating c6f711e..b76e148
Fast-forward
 swh/scrubber/db.py              |  2 +-
 swh/scrubber/sql/upgrades/3.sql | 43 +++++++++++++++++++++++++++
 swh/scrubber/storage_checker.py | 66 +++++++++++++++++++++++++++++------------
 3 files changed, 91 insertions(+), 20 deletions(-)
 create mode 100644 swh/scrubber/sql/upgrades/3.sql
Changes applied before test
commit b76e148ddf2ed709511eda39a006645199092584
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 16:15:47 2022 +0200

    Add missing DB version 3

commit 4e34e1d1bf1f3e05ff72220cfd93db5a0cad9770
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:19:43 2022 +0200

    Add metrics on references to missing objects

commit f1a61b1f501a5c3e342972c66688b08400dacea6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:12:37 2022 +0200

    Split batch_duration_seconds metric in two
    
    It will probably be useful to know what part of the check
    takes the most time.

commit 0cc86b0e45f015429406112b16e8b2e5196512ba
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:07:19 2022 +0200

    Deduplicate statsd calls
    
    This will allow adding more tags easily in future commits

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/58/ for more details.

Harbormaster completed remote builds in B30870: Diff 29818.Aug 18 2022, 4:26 PM
olasd accepted this revision.Aug 18 2022, 4:45 PM
olasd added a subscriber: olasd.

Thanks for checking the swh.core code, and sorry for the handwringing!

This revision is now accepted and ready to land.Aug 18 2022, 4:45 PM
vlorentz updated this revision to Diff 29823.Aug 18 2022, 4:53 PM

rebase

swh-public-ci added a comment.Aug 18 2022, 4:56 PM

Build is green

Patch application report for D8257 (id=29823)

Could not rebase; Attempt merge onto c6f711ecbf...

Updating c6f711e..36d16bc
Fast-forward
 swh/scrubber/db.py              |  2 +-
 swh/scrubber/sql/upgrades/3.sql | 43 ++++++++++++++++++++++++++
 swh/scrubber/storage_checker.py | 68 ++++++++++++++++++++++++++++++-----------
 3 files changed, 94 insertions(+), 19 deletions(-)
 create mode 100644 swh/scrubber/sql/upgrades/3.sql
Changes applied before test
commit 36d16bcdda42e991c3085ec5a92d1a854f6dddd1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 16:15:47 2022 +0200

    Add missing DB version 3

commit 53830b24f26ccd1faaca296c0e3096e34fd25fd8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:19:43 2022 +0200

    Add metrics on references to missing objects

commit aaae867e8d297d8f1aee26e4326209d69c3b392f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:12:37 2022 +0200

    Split batch_duration_seconds metric in two
    
    It will probably be useful to know what part of the check
    takes the most time.

commit 0cc86b0e45f015429406112b16e8b2e5196512ba
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Aug 18 15:07:19 2022 +0200

    Deduplicate statsd calls
    
    This will allow adding more tags easily in future commits

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/61/ for more details.

Harbormaster completed remote builds in B30875: Diff 29823.Aug 18 2022, 4:56 PM
Closed by commit rDSCRUB36d16bcdda42: Add missing DB version 3 (authored by vlorentz). · Explain WhyAug 18 2022, 4:59 PM
This revision was automatically updated to reflect the committed changes.
vlorentz added a commit: rDSCRUB36d16bcdda42: Add missing DB version 3.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
scrubber/
2 lines95%
Loading...
sql/
upgrades/
43 lines--

Diff 29816

View Options

swh/scrubber/db.py

Loading...
View Options

swh/scrubber/sql/upgrades/3.sql

Loading...
About · Developer information · Legal