Page MenuHomeSoftware Heritage

Update swhid according to latest specification
ClosedPublic

Authored by ardumont on May 12 2020, 3:36 PM.

Details

Summary

This updates the deposit_swh_id_context id following the latest specification, for example, from:

{
 'deposit_swh_id_context': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
 ...
 'deposit_id': '11',
 'deposit_status': 'done',
 'deposit_swh_id': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9',
 'deposit_swh_anchor_id': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb',
 'deposit_swh_anchor_id_context': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
 'deposit_status_detail': 'The deposit has been successfully \
                           loaded into the Software Heritage archive'
}

To:

{

 'deposit_swh_id_context': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9;origin=https://forge.softwareheritage.org/source/jesuisgpl/;visit=swh:1:snp:68c0d26104d47e278dd6be07ed61fafb561d0d20;anchor=swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;path=/',
...
 'deposit_id': '11',
 'deposit_status': 'done',
 'deposit_swh_id': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9',
 'deposit_swh_anchor_id': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb',
 'deposit_swh_anchor_id_context': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
 'deposit_status_detail': 'The deposit has been successfully \
                           loaded into the Software Heritage archive'
}

Note:

  • The deposit_swh_anchor_id and deposit_swh_anchor_id_context fields are no longer needed but are kept for now.
  • A slight change in that api, this now requires the client (deposit loader in that context) to send all the required information now. The loader already did but it was not required by that api.
  • This diff does not deal with the migration of the existing deposits to the new swhid format. Nor with dropping the "unnecessary" columns. That can be done later (and in other diffs).

Related to T2398

Test Plan

tox

Diff Detail

Repository
rDDEP Push deposit
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 12340
Build 18710: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 18709: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D3141 (id=11159)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit 18b553a3fb7a0d80b7a81fff678bb4a65dfac146
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update swhid according to latest specification
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/19/ for more details.

Build is green

Patch application report for D3141 (id=11160)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit 2c31aae61ef300dfb7afb39f21b8cb2235430ac9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update swhid according to latest specification
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/20/ for more details.

swh/deposit/api/private/deposit_update_status.py
107–109

We keep those 2 for now.
That allows to uncouple the migration of the previous deposit to the new format.

  • Update docs
  • Improve slightly the commit message

Build is green

Patch application report for D3141 (id=11161)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit ece9402e8a8428ed8a45d67b2606bba8097a2462
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/21/ for more details.

swh/deposit/api/private/deposit_update_status.py
107–109

(The code moved so this comment is unreadable now...)

I meant, we keep the deposit.swh_anchor_id and
deposit.swh_anchor_id_context for now. That way, we can deal with that
migration in multiple steps (first migrate the old deposit swh_id values to the
new one, then drop the columns).

ardumont edited the summary of this revision. (Show Details)
anlambert added a subscriber: anlambert.

Looks good to me.

swh/deposit/api/private/deposit_update_status.py
59

you can remove the + here

67–70

s/new ids/SWHIDs/

This revision is now accepted and ready to land.May 13 2020, 11:57 AM
ardumont edited the summary of this revision. (Show Details)

Adapt according to review

Thanks

Build is green

Patch application report for D3141 (id=11180)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit 51ce63737eb3a03b5c5cce2413b6dff108e06f07
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/22/ for more details.

Build is green

Patch application report for D3141 (id=11196)

Rebasing onto aaf05610dc...

Current branch diff-target is up to date.
Changes applied before test
commit 6604038ec7b11faf58e5600a7087ce9b575a6578
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/25/ for more details.

Build is green

Patch application report for D3141 (id=11198)

Rebasing onto 9113b6107e...

Current branch diff-target is up to date.
Changes applied before test
commit fb97bf9832bed13516d62848934cd0eea6a29f81
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/27/ for more details.