Page MenuHomeSoftware Heritage

deposit.migrations: Migrate to latest storage api change
ClosedPublic

Authored by ardumont on Jul 29 2020, 7:35 PM.

Details

Summary

It's code that no longer runs [1]. Just maintaining it along for the sake of
correctness.

[1] As all deposit migration script, it's a one time use production wise.

Related to T645

Test Plan

tox
(failing, needs storage release > 0.11.6)

Diff Detail

Repository
rDDEP Push deposit
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 14078
Build 21618: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 21617: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D3645 (id=12836)

Rebasing onto 402e4b83bb...

Current branch diff-target is up to date.
Changes applied before test
commit 693e8605e4edbbed44546386150fe87769115297
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jul 29 19:32:46 2020 +0200

    deposit.migrations: Migrate to latest storage api change
    
    It's code that no longer runs. Just maintaining it along for the sake of
    correctness.
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/47/
See console output for more information: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/47/console

This feels like a function that should be in swh.storage.algos

yeah...

Opening it to snapshot.algo, that will give me incentive to add tests on it ;)
Here i'm not too big on adding those on no longer used code.

Adapt according to swh.storage.snapshot.algos new algo function

Adapt according to swh.storage.snapshot.algos new algo function

Build has FAILED

Patch application report for D3645 (id=12842)

Rebasing onto 402e4b83bb...

Current branch diff-target is up to date.
Changes applied before test
commit 0661eb608470b59726d276f6bf2a8c420f69b5ee
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jul 29 19:32:46 2020 +0200

    deposit.migrations: Migrate to latest storage api change
    
    It's code that no longer runs. Just maintaining it along for the sake of
    correctness.
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/48/
See console output for more information: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/48/console

Build is green

Patch application report for D3645 (id=12841)

Could not rebase; Attempt merge onto 7667f7eef9...

Updating 7667f7ee..e026d33d
Fast-forward
 swh/storage/algos/snapshot.py            |  56 ++++++++-
 swh/storage/cassandra/cql.py             |  74 +++++++++++
 swh/storage/cassandra/storage.py         |  34 +++++
 swh/storage/db.py                        |  53 ++++----
 swh/storage/in_memory.py                 |  40 ++++++
 swh/storage/interface.py                 |  28 +++++
 swh/storage/storage.py                   |  47 +++++++
 swh/storage/tests/algos/test_snapshot.py |  64 +++++++++-
 swh/storage/tests/test_storage.py        | 205 +++++++++++++++++++++++++++++++
 9 files changed, 570 insertions(+), 31 deletions(-)
Changes applied before test
commit e026d33d08a488bc73f11f67ac1b8343cec5cc86
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 30 09:58:40 2020 +0200

    algos.snapshot: Open snapshot_get_from_revision
    
    This leverages the latest change in origin_visit_get and
    origin_visit_status_get to iterate over visit and visit status to detect a
    snapshot targetting a revision.
    
    This algo got used recently in the deposit. It may serve again.
    
    Related to T645

commit de11ef60ea74474c16df45149a97e9cfd8244ff0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jul 29 16:33:31 2020 +0200

    storage*: add origin_visit_status_get(...) -> PagedResult[OriginVisitStatus]
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/615/ for more details.

Build has FAILED

Patch application report for D3645 (id=12844)

Rebasing onto 402e4b83bb...

Current branch diff-target is up to date.
Changes applied before test
commit e189378e1353d629d3d79a4e480379863622d68c
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jul 29 19:32:46 2020 +0200

    deposit.migrations: Migrate to latest storage api change
    
    It's code that no longer runs. Just maintaining it along for the sake of
    correctness.
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/49/
See console output for more information: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/49/console

This revision is now accepted and ready to land.Jul 30 2020, 3:31 PM

Build has FAILED

Patch application report for D3645 (id=12844)

Rebasing onto 402e4b83bb...

Current branch diff-target is up to date.
Changes applied before test
commit e189378e1353d629d3d79a4e480379863622d68c
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jul 29 19:32:46 2020 +0200

    deposit.migrations: Migrate to latest storage api change
    
    It's code that no longer runs. Just maintaining it along for the sake of
    correctness.
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/50/
See console output for more information: https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/50/console

Build is green

Patch application report for D3645 (id=12844)

Rebasing onto 402e4b83bb...

Current branch diff-target is up to date.
Changes applied before test
commit e189378e1353d629d3d79a4e480379863622d68c
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jul 29 19:32:46 2020 +0200

    deposit.migrations: Migrate to latest storage api change
    
    It's code that no longer runs. Just maintaining it along for the sake of
    correctness.
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/51/ for more details.