Differential D8620

nixguix: Improve git origins detection
ClosedPublic
Actions

Authored by ardumont on Oct 4 2022, 11:32 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Maniphest Tasks

T3781: Replace the Nixguix loader with a lister

Commits

rDLSff80a91f0af8: nixguix: Improve git origins detection

Summary

Without this, some git repositories are detected as file (due to upstream
misqualification too). This does some extra effort to detect those to avoid sending
noise to loaders.

This also refactors some common code to build vcs artifacts to avoid duplication.

Related to T3781
Depends on D8619

Diff Detail

Repository

rDLS Listers

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Oct 4 2022, 11:32 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptOct 4 2022, 11:32 PM

Build is green

Patch application report for D8620 (id=31123)

Could not rebase; Attempt merge onto 944d4b5b60...

Updating 944d4b5..08be7a9
Fast-forward
 swh/lister/nixguix/lister.py                       | 67 ++++++++++++++++------
 .../nixguix/tests/data/nixpkgs-swh_sources.json    |  7 +++
 swh/lister/nixguix/tests/test_lister.py            | 22 ++++++-
 3 files changed, 75 insertions(+), 21 deletions(-)

Changes applied before test

commit 08be7a91c51fa391bc4880123520103e532cebe5
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 4 23:25:20 2022 +0200

    nixguix: Improve git origins detection
    
    Without this, some git repositories are detected as file (due to upstream
    misqualification too). This does some extra effort to detect those to avoid sending
    noise to loaders.
    
    This also refactors some common code to build vcs artifacts to avoid duplication.
    
    Related to T3781

commit c4c6df39e142648e066887f0ce6289974c99417d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 4 20:42:20 2022 +0200

    nixguix: Improve tarball detection
    
    Without this, some tarballs hidden within query parameters are not detected. This does
    some extra effort to detect those to avoid sending noise to loaders.
    
    Related to T3781

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/752/ for more details.

Harbormaster completed remote builds in B32092: Diff 31123.Oct 4 2022, 11:37 PM

ardumont requested review of this revision.Oct 4 2022, 11:37 PM

Rebase

ardumont mentioned this in T3781: Replace the Nixguix loader with a lister.Oct 5 2022, 10:11 AM

Build is green

Patch application report for D8620 (id=31127)

Could not rebase; Attempt merge onto 944d4b5b60...

Updating 944d4b5..ff80a91
Fast-forward
 swh/lister/nixguix/lister.py                       | 85 ++++++++++++++++------
 .../nixguix/tests/data/nixpkgs-swh_sources.json    |  7 ++
 swh/lister/nixguix/tests/test_lister.py            | 22 +++++-
 3 files changed, 89 insertions(+), 25 deletions(-)

Changes applied before test

commit ff80a91f0af81a067ea2c455b7fd2750ff81761b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 4 23:25:20 2022 +0200

    nixguix: Improve git origins detection
    
    Without this, some git repositories are detected as file (due to upstream
    misqualification too). This does some extra effort to detect those to avoid sending
    noise to loaders.
    
    This also refactors some common code to build vcs artifacts to avoid duplication.
    
    Related to T3781

commit 2fbd66778f3286c77149796339426aeccd11c1d0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 4 20:42:20 2022 +0200

    nixguix: Improve tarball detection
    
    Without this, some tarballs hidden within query parameters are not detected. This does
    some extra effort to detect those to avoid sending noise to loaders.
    
    Related to T3781

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/754/ for more details.

Harbormaster completed remote builds in B32096: Diff 31127.Oct 5 2022, 10:16 AM

vlorentz accepted this revision.Oct 5 2022, 10:31 AM

This revision is now accepted and ready to land.Oct 5 2022, 10:31 AM

Closed by commit rDLSff80a91f0af8: nixguix: Improve git origins detection (authored by ardumont). · Explain WhyOct 5 2022, 11:11 AM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDLSff80a91f0af8: nixguix: Improve git origins detection.