Page MenuHomeSoftware Heritage

nixguix: Improve git origins detection
ClosedPublic

Authored by ardumont on Oct 4 2022, 11:32 PM.

Details

Summary

Without this, some git repositories are detected as file (due to upstream
misqualification too). This does some extra effort to detect those to avoid sending
noise to loaders.

This also refactors some common code to build vcs artifacts to avoid duplication.

Related to T3781
Depends on D8619

Diff Detail

Repository
rDLS Listers
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 32096
Build 50255: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 50254: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8620 (id=31123)

Could not rebase; Attempt merge onto 944d4b5b60...

Updating 944d4b5..08be7a9
Fast-forward
 swh/lister/nixguix/lister.py                       | 67 ++++++++++++++++------
 .../nixguix/tests/data/nixpkgs-swh_sources.json    |  7 +++
 swh/lister/nixguix/tests/test_lister.py            | 22 ++++++-
 3 files changed, 75 insertions(+), 21 deletions(-)
Changes applied before test
commit 08be7a91c51fa391bc4880123520103e532cebe5
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 4 23:25:20 2022 +0200

    nixguix: Improve git origins detection
    
    Without this, some git repositories are detected as file (due to upstream
    misqualification too). This does some extra effort to detect those to avoid sending
    noise to loaders.
    
    This also refactors some common code to build vcs artifacts to avoid duplication.
    
    Related to T3781

commit c4c6df39e142648e066887f0ce6289974c99417d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 4 20:42:20 2022 +0200

    nixguix: Improve tarball detection
    
    Without this, some tarballs hidden within query parameters are not detected. This does
    some extra effort to detect those to avoid sending noise to loaders.
    
    Related to T3781

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/752/ for more details.

Build is green

Patch application report for D8620 (id=31127)

Could not rebase; Attempt merge onto 944d4b5b60...

Updating 944d4b5..ff80a91
Fast-forward
 swh/lister/nixguix/lister.py                       | 85 ++++++++++++++++------
 .../nixguix/tests/data/nixpkgs-swh_sources.json    |  7 ++
 swh/lister/nixguix/tests/test_lister.py            | 22 +++++-
 3 files changed, 89 insertions(+), 25 deletions(-)
Changes applied before test
commit ff80a91f0af81a067ea2c455b7fd2750ff81761b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 4 23:25:20 2022 +0200

    nixguix: Improve git origins detection
    
    Without this, some git repositories are detected as file (due to upstream
    misqualification too). This does some extra effort to detect those to avoid sending
    noise to loaders.
    
    This also refactors some common code to build vcs artifacts to avoid duplication.
    
    Related to T3781

commit 2fbd66778f3286c77149796339426aeccd11c1d0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Oct 4 20:42:20 2022 +0200

    nixguix: Improve tarball detection
    
    Without this, some tarballs hidden within query parameters are not detected. This does
    some extra effort to detect those to avoid sending noise to loaders.
    
    Related to T3781

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/754/ for more details.

This revision is now accepted and ready to land.Oct 5 2022, 10:31 AM
This revision was automatically updated to reflect the committed changes.