Page MenuHomeSoftware Heritage

debian: Make resolve_revision_from use the sha256 of the .dsc
ClosedPublic

Authored by vlorentz on Mar 19 2021, 2:21 PM.

Details

Summary

Instead of the sha256 + name + ... of all the files of a package.

This will be needed to transition to ExtID, as we can't reasonably write
this large set in the ExtID storage; and the sha256 of the .dsc is good
enough, as the .dsc contains hashes and names of other files.

Diff Detail

Repository
rDLDBASE Generic VCS/Package Loader
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D5285 (id=18932)

Could not rebase; Attempt merge onto 132522e42e...

Merge made by the 'recursive' strategy.
 swh/loader/package/debian/loader.py              | 44 ++++++++++---------
 swh/loader/package/debian/tests/test_debian.py   | 55 ++++++++++++++++++++++++
 swh/loader/package/nixguix/tests/test_nixguix.py | 33 ++++++++------
 3 files changed, 100 insertions(+), 32 deletions(-)
Changes applied before test
commit e29c3d05b974020dcfdcec2da04e3b47ba75e7e5
Merge: 132522e 9a4991b
Author: Jenkins user <jenkins@localhost>
Date:   Fri Mar 19 13:21:22 2021 +0000

    Merge branch 'diff-target' into HEAD

commit 9a4991b391e6b825f9dee3008a410650d434ca6d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 10:02:21 2021 +0100

    debian: Make resolve_revision_from use the sha256 of the .dsc
    
    Instead of the sha256 + name + ... of all the files of a package.
    
    This will be needed to transition to ExtID, as we can't reasonably write
    this large set in the ExtID storage; and the sha256 of the .dsc is good
    enough, as the .dsc contains hashes and names of other files.

commit 455e05837dc2fb59b4adcf01df27c6f7d64781dd
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Mar 16 14:44:33 2021 +0100

    test_load_nixguix_one_common_artifact_from_other_loader: Test all log lines, not just the last one
    
    It's stricter and more readable.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/403/ for more details.

ardumont added inline comments.
swh/loader/package/debian/tests/test_debian.py
553

It's not immediately clear what's the difference between this test and the previous one.
Can you please clarify intents with a docstring for that test?

swh/loader/package/debian/tests/test_debian.py
501

why corrupt by the way?

and that should be corrupted, shouldn't it?

lgtm

required changes for the sake of clarifying intents in test and one more question.

This revision now requires changes to proceed.Mar 20 2021, 10:11 AM
swh/loader/package/debian/tests/test_debian.py
501

Because there should only be one .dsc per package. And "corrupt" and "corrupted" are synonyms

553

One is for the known artifacts, and one is for the new artifact

This revision is now accepted and ready to land.Mar 22 2021, 9:17 AM

Build has FAILED

Patch application report for D5285 (id=18996)

Could not rebase; Attempt merge onto 132522e42e...

Updating 132522e..4960aa3
Fast-forward
 swh/loader/package/debian/loader.py              | 44 ++++++++++---------
 swh/loader/package/debian/tests/test_debian.py   | 55 ++++++++++++++++++++++++
 swh/loader/package/nixguix/tests/test_nixguix.py | 33 ++++++++------
 3 files changed, 100 insertions(+), 32 deletions(-)
Changes applied before test
commit 4960aa38b10b29d249b28b92e78e0308f482e551
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 10:02:21 2021 +0100

    debian: Make resolve_revision_from use the sha256 of the .dsc
    
    Instead of the sha256 + name + ... of all the files of a package.
    
    This will be needed to transition to ExtID, as we can't reasonably write
    this large set in the ExtID storage; and the sha256 of the .dsc is good
    enough, as the .dsc contains hashes and names of other files.

commit cf8fc9e9bd19b4f303b33ddeb73dc285d835bfcc
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Mar 16 14:44:33 2021 +0100

    test_load_nixguix_one_common_artifact_from_other_loader: Test all log lines, not just the last one
    
    It's stricter and more readable.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/411/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/411/console

Build is green

Patch application report for D5285 (id=19023)

Rebasing onto 9a339e981a...

Current branch diff-target is up to date.
Changes applied before test
commit 268f83e943a439958840b4d8cdcff5c9eb16429e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 10:02:21 2021 +0100

    debian: Make resolve_revision_from use the sha256 of the .dsc
    
    Instead of the sha256 + name + ... of all the files of a package.
    
    This will be needed to transition to ExtID, as we can't reasonably write
    this large set in the ExtID storage; and the sha256 of the .dsc is good
    enough, as the .dsc contains hashes and names of other files.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/423/ for more details.