Page MenuHomeSoftware Heritage
Differential D5288

nixguix: Split 'integrity' extraction out of resolve_revision_from
ClosedPublic
Actions

Authored by vlorentz on Mar 19 2021, 2:21 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T3141: Make loaders pass ExtId objects between functions instead of "known_artifact"
Commits
rDLDBASEfdb41abd76b2: nixguix: Split 'integrity' extraction out of resolve_revision_from
Summary

We will need it independently in a future commit

Diff Detail

Event Timeline

vlorentz created this revision.Mar 19 2021, 2:21 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptMar 19 2021, 2:21 PM
swh-public-ci added a comment.Mar 19 2021, 2:28 PM

Build is green

Patch application report for D5288 (id=18935)

Could not rebase; Attempt merge onto 132522e42e...

Merge made by the 'recursive' strategy.
 swh/loader/package/debian/loader.py              | 53 +++++++++++++----------
 swh/loader/package/debian/tests/test_debian.py   | 55 ++++++++++++++++++++++++
 swh/loader/package/nixguix/loader.py             | 44 ++++++++++---------
 swh/loader/package/nixguix/tests/test_nixguix.py | 33 ++++++++------
 swh/loader/package/npm/loader.py                 | 26 ++++++-----
 swh/loader/package/pypi/loader.py                | 31 +++++++------
 6 files changed, 164 insertions(+), 78 deletions(-)
Changes applied before test
commit a513117d29a05f38da01ac14bff937f6c9dad0ce
Merge: 132522e 3357e64
Author: Jenkins user <jenkins@localhost>
Date:   Fri Mar 19 13:27:00 2021 +0000

    Merge branch 'diff-target' into HEAD

commit 3357e642e5676aa1ace50ecffe2a777f51b251f7
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 18 17:47:44 2021 +0100

    nixguix: Split 'integrity' extraction out of resolve_revision_from
    
    We will need it independently in a future commit

commit 9dc175c2aef95b0e5a2a83a71ec63e2c1f075c7b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 09:32:37 2021 +0100

    npm, pypi: Split original_artifact parsing out of artifact_to_revision_id
    
    We will need it independently in a future commit

commit 93c9aa8e014c19b02887e6c05a671ad7430235dd
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 18 16:18:40 2021 +0100

    debian: Split original_artifact parsing out of resolve_revision_from
    
    We will need it independently in a future commit

commit 9a4991b391e6b825f9dee3008a410650d434ca6d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 10:02:21 2021 +0100

    debian: Make resolve_revision_from use the sha256 of the .dsc
    
    Instead of the sha256 + name + ... of all the files of a package.
    
    This will be needed to transition to ExtID, as we can't reasonably write
    this large set in the ExtID storage; and the sha256 of the .dsc is good
    enough, as the .dsc contains hashes and names of other files.

commit 455e05837dc2fb59b4adcf01df27c6f7d64781dd
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Mar 16 14:44:33 2021 +0100

    test_load_nixguix_one_common_artifact_from_other_loader: Test all log lines, not just the last one
    
    It's stricter and more readable.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/406/ for more details.

Harbormaster completed remote builds in B20015: Diff 18935.Mar 19 2021, 2:28 PM
vlorentz requested review of this revision.Mar 19 2021, 2:28 PM
vlorentz added a task: T3141: Make loaders pass ExtId objects between functions instead of "known_artifact".Mar 19 2021, 2:48 PM
ardumont accepted this revision.Mar 20 2021, 10:20 AM
This revision is now accepted and ready to land.Mar 20 2021, 10:20 AM
vlorentz updated this revision to Diff 18999.Mar 23 2021, 10:20 AM

rebase

swh-public-ci added a comment.Mar 23 2021, 10:26 AM

Build has FAILED

Patch application report for D5288 (id=18999)

Could not rebase; Attempt merge onto 132522e42e...

Updating 132522e..fd70fe0
Fast-forward
 swh/loader/package/debian/loader.py              | 53 +++++++++++++----------
 swh/loader/package/debian/tests/test_debian.py   | 55 ++++++++++++++++++++++++
 swh/loader/package/nixguix/loader.py             | 44 ++++++++++---------
 swh/loader/package/nixguix/tests/test_nixguix.py | 33 ++++++++------
 swh/loader/package/npm/loader.py                 | 26 ++++++-----
 swh/loader/package/pypi/loader.py                | 37 ++++++++++------
 swh/loader/package/pypi/tests/test_pypi.py       | 11 +++++
 7 files changed, 181 insertions(+), 78 deletions(-)
Changes applied before test
commit fd70fe0e475a4f5d339d2a69ce3eb99aad009d76
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 18 17:47:44 2021 +0100

    nixguix: Split 'integrity' extraction out of resolve_revision_from
    
    We will need it independently in a future commit

commit 6da306159137c309ea3aa987589415c4b8f81de3
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 09:32:37 2021 +0100

    npm, pypi: Split original_artifact parsing out of artifact_to_revision_id
    
    We will need it independently in a future commit

commit 3190406239d231d62b8583bb1aeb74def1016a08
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 18 16:18:40 2021 +0100

    debian: Split original_artifact parsing out of resolve_revision_from
    
    We will need it independently in a future commit

commit 4960aa38b10b29d249b28b92e78e0308f482e551
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 10:02:21 2021 +0100

    debian: Make resolve_revision_from use the sha256 of the .dsc
    
    Instead of the sha256 + name + ... of all the files of a package.
    
    This will be needed to transition to ExtID, as we can't reasonably write
    this large set in the ExtID storage; and the sha256 of the .dsc is good
    enough, as the .dsc contains hashes and names of other files.

commit cf8fc9e9bd19b4f303b33ddeb73dc285d835bfcc
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Mar 16 14:44:33 2021 +0100

    test_load_nixguix_one_common_artifact_from_other_loader: Test all log lines, not just the last one
    
    It's stricter and more readable.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/414/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/414/console

Harbormaster failed remote builds in B20076: Diff 18999!Mar 23 2021, 10:26 AM
vlorentz updated this revision to Diff 19026.Mar 23 2021, 12:27 PM

rebase

swh-public-ci added a comment.Mar 23 2021, 12:32 PM

Build has FAILED

Patch application report for D5288 (id=19026)

Could not rebase; Attempt merge onto 9a339e981a...

Updating 9a339e9..d565e5b
Fast-forward
 swh/loader/package/debian/loader.py            | 53 ++++++++++++++-----------
 swh/loader/package/debian/tests/test_debian.py | 55 ++++++++++++++++++++++++++
 swh/loader/package/nixguix/loader.py           | 44 +++++++++++----------
 swh/loader/package/npm/loader.py               | 26 +++++++-----
 swh/loader/package/pypi/loader.py              | 37 +++++++++++------
 swh/loader/package/pypi/tests/test_pypi.py     | 11 ++++++
 6 files changed, 160 insertions(+), 66 deletions(-)
Changes applied before test
commit d565e5b307dbcf5763171fcce828d4305dccf5de
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 18 17:47:44 2021 +0100

    nixguix: Split 'integrity' extraction out of resolve_revision_from
    
    We will need it independently in a future commit

commit 749dd287432d116be5f4a69335ef35c244923ca6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 09:32:37 2021 +0100

    npm, pypi: Split original_artifact parsing out of artifact_to_revision_id
    
    We will need it independently in a future commit

commit e7ae6364c06a4d1e1863e8c1e948cd3a82ca4286
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 18 16:18:40 2021 +0100

    debian: Split original_artifact parsing out of resolve_revision_from
    
    We will need it independently in a future commit

commit 268f83e943a439958840b4d8cdcff5c9eb16429e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 10:02:21 2021 +0100

    debian: Make resolve_revision_from use the sha256 of the .dsc
    
    Instead of the sha256 + name + ... of all the files of a package.
    
    This will be needed to transition to ExtID, as we can't reasonably write
    this large set in the ExtID storage; and the sha256 of the .dsc is good
    enough, as the .dsc contains hashes and names of other files.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/426/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/426/console

Harbormaster failed remote builds in B20102: Diff 19026!Mar 23 2021, 12:33 PM
vlorentz updated this revision to Diff 19030.Mar 23 2021, 1:19 PM

rebase

swh-public-ci added a comment.Mar 23 2021, 1:23 PM

Build has FAILED

Patch application report for D5288 (id=19030)

Could not rebase; Attempt merge onto 9a339e981a...

Updating 9a339e9..589d975
Fast-forward
 swh/loader/package/debian/loader.py            | 53 ++++++++++++++-----------
 swh/loader/package/debian/tests/test_debian.py | 55 ++++++++++++++++++++++++++
 swh/loader/package/nixguix/loader.py           | 44 +++++++++++----------
 swh/loader/package/npm/loader.py               | 26 +++++++-----
 swh/loader/package/pypi/loader.py              | 37 +++++++++++------
 swh/loader/package/pypi/tests/test_pypi.py     | 11 ++++++
 6 files changed, 160 insertions(+), 66 deletions(-)
Changes applied before test
commit 589d97562923155d4e51de6367e8985bbfc92e5a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 18 17:47:44 2021 +0100

    nixguix: Split 'integrity' extraction out of resolve_revision_from
    
    We will need it independently in a future commit

commit dc197af60c60f061df3ac35d7e7f94b0a0262140
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 09:32:37 2021 +0100

    npm, pypi: Split original_artifact parsing out of artifact_to_revision_id
    
    We will need it independently in a future commit

commit e7ae6364c06a4d1e1863e8c1e948cd3a82ca4286
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 18 16:18:40 2021 +0100

    debian: Split original_artifact parsing out of resolve_revision_from
    
    We will need it independently in a future commit

commit 268f83e943a439958840b4d8cdcff5c9eb16429e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 10:02:21 2021 +0100

    debian: Make resolve_revision_from use the sha256 of the .dsc
    
    Instead of the sha256 + name + ... of all the files of a package.
    
    This will be needed to transition to ExtID, as we can't reasonably write
    this large set in the ExtID storage; and the sha256 of the .dsc is good
    enough, as the .dsc contains hashes and names of other files.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/430/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/430/console

Harbormaster failed remote builds in B20106: Diff 19030!Mar 23 2021, 1:23 PM
vlorentz updated this revision to Diff 19034.Mar 23 2021, 1:47 PM

rebase

swh-public-ci added a comment.Mar 23 2021, 1:51 PM

Build is green

Patch application report for D5288 (id=19034)

Could not rebase; Attempt merge onto 9a339e981a...

Updating 9a339e9..fdb41ab
Fast-forward
 swh/loader/package/debian/loader.py            | 53 ++++++++++++++-----------
 swh/loader/package/debian/tests/test_debian.py | 55 ++++++++++++++++++++++++++
 swh/loader/package/nixguix/loader.py           | 44 +++++++++++----------
 swh/loader/package/npm/loader.py               | 26 +++++++-----
 swh/loader/package/pypi/loader.py              | 37 +++++++++++------
 swh/loader/package/pypi/tests/test_pypi.py     | 14 +++++++
 6 files changed, 163 insertions(+), 66 deletions(-)
Changes applied before test
commit fdb41abd76b2d683c2fd44fcef0d267ee2788a25
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 18 17:47:44 2021 +0100

    nixguix: Split 'integrity' extraction out of resolve_revision_from
    
    We will need it independently in a future commit

commit f827d04cb538038cc38aceb7ee55c5465c444f52
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 09:32:37 2021 +0100

    npm, pypi: Split original_artifact parsing out of artifact_to_revision_id
    
    We will need it independently in a future commit

commit e7ae6364c06a4d1e1863e8c1e948cd3a82ca4286
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 18 16:18:40 2021 +0100

    debian: Split original_artifact parsing out of resolve_revision_from
    
    We will need it independently in a future commit

commit 268f83e943a439958840b4d8cdcff5c9eb16429e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 17 10:02:21 2021 +0100

    debian: Make resolve_revision_from use the sha256 of the .dsc
    
    Instead of the sha256 + name + ... of all the files of a package.
    
    This will be needed to transition to ExtID, as we can't reasonably write
    this large set in the ExtID storage; and the sha256 of the .dsc is good
    enough, as the .dsc contains hashes and names of other files.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/434/ for more details.

Harbormaster completed remote builds in B20110: Diff 19034.Mar 23 2021, 1:51 PM
Closed by commit rDLDBASEfdb41abd76b2: nixguix: Split 'integrity' extraction out of resolve_revision_from (authored by vlorentz). · Explain WhyMar 23 2021, 4:33 PM
This revision was automatically updated to reflect the committed changes.
vlorentz added a commit: rDLDBASEfdb41abd76b2: nixguix: Split 'integrity' extraction out of resolve_revision_from.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
loader/
package/
nixguix/
44 lines99%
Loading...

Diff 19034

View Options

swh/loader/package/nixguix/loader.py

Loading...
About · Developer information · Legal