Page MenuHomeSoftware Heritage
Differential D3141

Update swhid according to latest specification
ClosedPublic
Actions

Authored by ardumont on May 12 2020, 3:36 PM.

Details

Reviewers
moranegg
anlambert
Group Reviewers
Reviewers
Maniphest Tasks
T2398: Simplify/update the SWHIDs returned from a SWORD deposit
Commits
rDDEPfb97bf9832be: Update deposit swhid to respect the latest specification update
Summary

This updates the deposit_swh_id_context id following the latest specification, for example, from:

{
 'deposit_swh_id_context': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
 ...
 'deposit_id': '11',
 'deposit_status': 'done',
 'deposit_swh_id': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9',
 'deposit_swh_anchor_id': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb',
 'deposit_swh_anchor_id_context': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
 'deposit_status_detail': 'The deposit has been successfully \
                           loaded into the Software Heritage archive'
}

To:

{

 'deposit_swh_id_context': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9;origin=https://forge.softwareheritage.org/source/jesuisgpl/;visit=swh:1:snp:68c0d26104d47e278dd6be07ed61fafb561d0d20;anchor=swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;path=/',
...
 'deposit_id': '11',
 'deposit_status': 'done',
 'deposit_swh_id': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9',
 'deposit_swh_anchor_id': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb',
 'deposit_swh_anchor_id_context': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
 'deposit_status_detail': 'The deposit has been successfully \
                           loaded into the Software Heritage archive'
}

Note:

  • The deposit_swh_anchor_id and deposit_swh_anchor_id_context fields are no longer needed but are kept for now.
  • A slight change in that api, this now requires the client (deposit loader in that context) to send all the required information now. The loader already did but it was not required by that api.
  • This diff does not deal with the migration of the existing deposits to the new swhid format. Nor with dropping the "unnecessary" columns. That can be done later (and in other diffs).

Related to T2398

Test Plan

tox

Diff Detail

Repository
rDDEP Push deposit
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 12340
Build 18710: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 18709: arc lint + arc unit

Event Timeline

ardumont created this revision.May 12 2020, 3:36 PM
ardumont updated this revision to Diff 11160.May 12 2020, 3:39 PM

Update requirements

swh-public-ci added a comment.May 12 2020, 3:39 PM

Build is green

Patch application report for D3141 (id=11159)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit 18b553a3fb7a0d80b7a81fff678bb4a65dfac146
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update swhid according to latest specification
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/19/ for more details.

Harbormaster completed remote builds in B12340: Diff 11159.May 12 2020, 3:39 PM
swh-public-ci added a comment.May 12 2020, 3:42 PM

Build is green

Patch application report for D3141 (id=11160)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit 2c31aae61ef300dfb7afb39f21b8cb2235430ac9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update swhid according to latest specification
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/20/ for more details.

Harbormaster completed remote builds in B12341: Diff 11160.May 12 2020, 3:42 PM
ardumont added inline comments.May 12 2020, 3:43 PM
swh/deposit/api/private/deposit_update_status.py
107–109

We keep those 2 for now.
That allows to uncouple the migration of the previous deposit to the new format.

ardumont updated this revision to Diff 11161.May 12 2020, 3:51 PM
  • Update docs
  • Improve slightly the commit message
swh-public-ci added a comment.May 12 2020, 3:54 PM

Build is green

Patch application report for D3141 (id=11161)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit ece9402e8a8428ed8a45d67b2606bba8097a2462
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/21/ for more details.

Harbormaster completed remote builds in B12342: Diff 11161.May 12 2020, 3:54 PM
ardumont mentioned this in D3142: deposit: Adapt loader to use the latest deposit update api.May 12 2020, 3:54 PM
ardumont edited the summary of this revision. (Show Details)May 12 2020, 4:08 PM
ardumont edited the summary of this revision. (Show Details)May 13 2020, 10:40 AM
ardumont added inline comments.May 13 2020, 10:43 AM
swh/deposit/api/private/deposit_update_status.py
107–109

(The code moved so this comment is unreadable now...)

I meant, we keep the deposit.swh_anchor_id and
deposit.swh_anchor_id_context for now. That way, we can deal with that
migration in multiple steps (first migrate the old deposit swh_id values to the
new one, then drop the columns).

ardumont mentioned this in T2398: Simplify/update the SWHIDs returned from a SWORD deposit.May 13 2020, 10:47 AM
ardumont edited the summary of this revision. (Show Details)May 13 2020, 11:04 AM
ardumont edited the summary of this revision. (Show Details)
ardumont added projects: SWORD deposit, Data Model.
ardumont edited the summary of this revision. (Show Details)May 13 2020, 11:07 AM
ardumont edited the summary of this revision. (Show Details)May 13 2020, 11:09 AM
anlambert accepted this revision.May 13 2020, 11:57 AM
anlambert added a subscriber: anlambert.

Looks good to me.

swh/deposit/api/private/deposit_update_status.py
59

you can remove the + here

67–70

s/new ids/SWHIDs/

This revision is now accepted and ready to land.May 13 2020, 11:57 AM
ardumont updated this revision to Diff 11180.May 13 2020, 12:24 PM
ardumont edited the summary of this revision. (Show Details)

Adapt according to review

Thanks

swh-public-ci added a comment.May 13 2020, 12:27 PM

Build is green

Patch application report for D3141 (id=11180)

Rebasing onto 85e1ff3eea...

Current branch diff-target is up to date.
Changes applied before test
commit 51ce63737eb3a03b5c5cce2413b6dff108e06f07
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/22/ for more details.

Harbormaster completed remote builds in B12362: Diff 11180.May 13 2020, 12:27 PM
ardumont updated this revision to Diff 11196.May 14 2020, 12:40 PM

Rebase on latest master

swh-public-ci added a comment.May 14 2020, 12:44 PM

Build is green

Patch application report for D3141 (id=11196)

Rebasing onto aaf05610dc...

Current branch diff-target is up to date.
Changes applied before test
commit 6604038ec7b11faf58e5600a7087ce9b575a6578
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/25/ for more details.

Harbormaster completed remote builds in B12382: Diff 11196.May 14 2020, 12:44 PM
ardumont updated this revision to Diff 11198.May 14 2020, 2:37 PM

Rebase on latest master

swh-public-ci added a comment.May 14 2020, 2:40 PM

Build is green

Patch application report for D3141 (id=11198)

Rebasing onto 9113b6107e...

Current branch diff-target is up to date.
Changes applied before test
commit fb97bf9832bed13516d62848934cd0eea6a29f81
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 12 15:33:00 2020 +0200

    Update deposit swhid to respect the latest specification update
    
    This also ensures the loader sends all the required information now (it did
    already but it was a bit laxed on checks for that part).
    
    Related to T2398

See https://jenkins.softwareheritage.org/job/DDEP/job/tests-on-diff/27/ for more details.

Harbormaster completed remote builds in B12385: Diff 11198.May 14 2020, 2:40 PM
ardumont mentioned this in D3153: Migrate deposit SWHIDs (data) to the new specification.May 15 2020, 11:46 AM
Closed by commit rDDEPfb97bf9832be: Update deposit swhid to respect the latest specification update (authored by ardumont). · Explain WhyMay 15 2020, 12:40 PM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDDEPfb97bf9832be: Update deposit swhid to respect the latest specification update.
ardumont mentioned this in rDDEPa631dabb6fdd: Migrate deposit SWHIDs (data) to the new specification.
ardumont mentioned this in rDLDBASE8bd38f9bb468: deposit: Adapt loader to use the latest deposit update api.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
deposit/
api/
private/
65 lines100%
Loading...
tests/
api/
146 lines100%
Loading...

Diff 11159

View Options

swh/deposit/api/private/deposit_update_status.py

Loading...
View Options

swh/deposit/tests/api/test_deposit_private_update_status.py

Loading...
About · Developer information · Legal