Page MenuHomeSoftware Heritage

package loaders: write extrinsic metadata to directories instead of revisions.
ClosedPublic

Authored by vlorentz on Oct 23 2020, 4:53 PM.

Details

Summary

They are more useful on directories, as directory ids are more intrinsic than
synthetic revision ids.
And this adds the revision swhid in the context, so the revision relationship
is still available when it's useful (eg. because the same directory can be
referenced from multiple revisions).

Note that package loaders still write the original_artifacts dict to the revision.

Part of the solution for T2668

Event Timeline

Build is green

Patch application report for D4346 (id=15378)

Rebasing onto c30e485120...

Current branch diff-target is up to date.
Changes applied before test
commit 28bd28ca57a23d123dcc0931843e9a1d83c4d53d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Oct 23 16:52:23 2020 +0200

    package loaders: write extrinsic metadata to directories instead of revisions.
    
    They are more useful on directories, as directory ids are more intrinsic than
    synthetic revision ids.
    And this adds the revision swhid in the context, so the revision relationship
    is still available when it's useful (eg. because the same directory can be
    referenced from multiple revisions).

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/330/ for more details.

Couple of questions inline.

swh/loader/package/deposit/tests/test_deposit.py
215

Shouldn't this be .DIRECTORY?

What did i misunderstand?

218

dir_meta?

swh/loader/package/deposit/tests/test_deposit.py
218–223

dir_meta in each assertion when it is changed on line 220

403–404

is this variable supposed to stay rev_metadata?

406–407

rev_metadata => dir_metadata

Build is green

Patch application report for D4346 (id=15484)

Rebasing onto e2726ef650...

First, rewinding head to replay your work on top of it...
Applying: package loaders: write extrinsic metadata to directories instead of revisions.
Changes applied before test
commit 48e60d7d3b8c1baa78cb4252d9005b672237aa57
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Oct 23 16:52:23 2020 +0200

    package loaders: write extrinsic metadata to directories instead of revisions.
    
    They are more useful on directories, as directory ids are more intrinsic than
    synthetic revision ids.
    And this adds the revision swhid in the context, so the revision relationship
    is still available when it's useful (eg. because the same directory can be
    referenced from multiple revisions).

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/333/ for more details.

swh/loader/package/deposit/tests/test_deposit.py
215

why did the test passed with the wong .REVISION?

Build is green

Patch application report for D4346 (id=15486)

Rebasing onto e2726ef650...

Current branch diff-target is up to date.
Changes applied before test
commit cf58604ccaa469a7d31c30c476a3b9187493e9ce
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Oct 23 16:52:23 2020 +0200

    package loaders: write extrinsic metadata to directories instead of revisions.
    
    They are more useful on directories, as directory ids are more intrinsic than
    synthetic revision ids.
    And this adds the revision swhid in the context, so the revision relationship
    is still available when it's useful (eg. because the same directory can be
    referenced from multiple revisions).

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/335/ for more details.

vlorentz added inline comments.
swh/loader/package/deposit/tests/test_deposit.py
215

because the storage only checks if it's MetadataTargetType.ORIGIN or not. We should probably add a check to make sure it matches the type in the SWHID

This revision is now accepted and ready to land.Oct 29 2020, 12:44 PM