Page MenuHomeSoftware Heritage

Update swhid according to latest specification
ClosedPublic

Authored by ardumont on Tue, May 12, 3:36 PM.

Details

Summary

This updates the deposit_swh_id_context id following the latest specification, for example, from:

{
 'deposit_swh_id_context': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
 ...
 'deposit_id': '11',
 'deposit_status': 'done',
 'deposit_swh_id': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9',
 'deposit_swh_anchor_id': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb',
 'deposit_swh_anchor_id_context': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
 'deposit_status_detail': 'The deposit has been successfully \
                           loaded into the Software Heritage archive'
}

To:

{

 'deposit_swh_id_context': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9;origin=https://forge.softwareheritage.org/source/jesuisgpl/;visit=swh:1:snp:68c0d26104d47e278dd6be07ed61fafb561d0d20;anchor=swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;path=/',
...
 'deposit_id': '11',
 'deposit_status': 'done',
 'deposit_swh_id': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9',
 'deposit_swh_anchor_id': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb',
 'deposit_swh_anchor_id_context': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
 'deposit_status_detail': 'The deposit has been successfully \
                           loaded into the Software Heritage archive'
}

Note:

  • The deposit_swh_anchor_id and deposit_swh_anchor_id_context fields are no longer needed but are kept for now.
  • A slight change in that api, this now requires the client (deposit loader in that context) to send all the required information now. The loader already did but it was not required by that api.
  • This diff does not deal with the migration of the existing deposits to the new swhid format. Nor with dropping the "unnecessary" columns. That can be done later (and in other diffs).

Related to T2398

Test Plan

tox

Diff Detail

Repository
rDDEP Push deposit
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Tue, May 12, 3:36 PM
ardumont updated this revision to Diff 11160.Tue, May 12, 3:39 PM

Update requirements

Build is green

Patch application report for D3141 (id=11159)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit 18b553a3fb7a0d80b7a81fff678bb4a65dfac146
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update swhid according to latest specification
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/19/ for more details.

Build is green

Patch application report for D3141 (id=11160)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit 2c31aae61ef300dfb7afb39f21b8cb2235430ac9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update swhid according to latest specification
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/20/ for more details.

ardumont added inline comments.Tue, May 12, 3:43 PM
swh/deposit/api/private/deposit_update_status.py
105–107

We keep those 2 for now.
That allows to uncouple the migration of the previous deposit to the new format.

ardumont updated this revision to Diff 11161.Tue, May 12, 3:51 PM
  • Update docs
  • Improve slightly the commit message

Build is green

Patch application report for D3141 (id=11161)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit ece9402e8a8428ed8a45d67b2606bba8097a2462
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/21/ for more details.

ardumont edited the summary of this revision. (Show Details)Tue, May 12, 4:08 PM
ardumont edited the summary of this revision. (Show Details)Wed, May 13, 10:40 AM
ardumont added inline comments.Wed, May 13, 10:43 AM
swh/deposit/api/private/deposit_update_status.py
105–107

(The code moved so this comment is unreadable now...)

I meant, we keep the deposit.swh_anchor_id and
deposit.swh_anchor_id_context for now. That way, we can deal with that
migration in multiple steps (first migrate the old deposit swh_id values to the
new one, then drop the columns).

ardumont edited the summary of this revision. (Show Details)Wed, May 13, 11:04 AM
ardumont edited the summary of this revision. (Show Details)
ardumont edited the summary of this revision. (Show Details)Wed, May 13, 11:07 AM
ardumont edited the summary of this revision. (Show Details)Wed, May 13, 11:09 AM
anlambert accepted this revision.Wed, May 13, 11:57 AM
anlambert added a subscriber: anlambert.

Looks good to me.

swh/deposit/api/private/deposit_update_status.py
60

you can remove the + here

67

s/new ids/SWHIDs/

This revision is now accepted and ready to land.Wed, May 13, 11:57 AM
ardumont updated this revision to Diff 11180.Wed, May 13, 12:24 PM
ardumont edited the summary of this revision. (Show Details)

Adapt according to review

Thanks

Build is green

Patch application report for D3141 (id=11180)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit 51ce63737eb3a03b5c5cce2413b6dff108e06f07
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/22/ for more details.

ardumont updated this revision to Diff 11196.Thu, May 14, 12:40 PM

Rebase on latest master

Build is green

Patch application report for D3141 (id=11196)

Rebasing onto aaf05610dc...

Current branch diff-target is up to date.
Changes applied before test
commit 6604038ec7b11faf58e5600a7087ce9b575a6578
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/25/ for more details.

ardumont updated this revision to Diff 11198.Thu, May 14, 2:37 PM

Rebase on latest master

Build is green

Patch application report for D3141 (id=11198)

Rebasing onto 9113b6107e...

Current branch diff-target is up to date.
Changes applied before test
commit fb97bf9832bed13516d62848934cd0eea6a29f81
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/27/ for more details.