Paths

Table of Contentst

Differential D6271

tarball: Add support for uncompressing crate files
ClosedPublic
Actions

Authored by anlambert on Sep 15 2021, 3:55 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Commits

rDCORE0ef57a4dd346: tarball: Add support for uncompressing crate files

Summary

Rust crate files are standard tar.gz archives so add support
to uncompress them through shutil.register_unpack_format.

For the record, those kind of source files archive are processed
by the nixguix loader, see that sentry issue report for instance.

Test Plan

Small crate file has been added in tests data and existing test has been adapted.

Diff Detail

Repository

rDCORE Foundations and core functionalities

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Sep 15 2021, 3:55 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptSep 15 2021, 3:55 PM

Build is green

Patch application report for D6271 (id=22702)

Rebasing onto 632673f6bd...

Current branch diff-target is up to date.

Changes applied before test

commit 590e03756dea42bc00dd25af3c66431bb1d4e5da
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Sep 15 14:20:11 2021 +0200

    tarball: Add support for uncompressing crate files
    
    Rust crate files are standard tar.gz archives so add support
    to uncompress them through shutil.register_unpack_format.
    
    For the record, those kind of source files archive are processed
    by the nixguix loader.

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/244/ for more details.

Harbormaster completed remote builds in B23635: Diff 22702.Sep 15 2021, 3:56 PM

anlambert requested review of this revision.Sep 15 2021, 3:56 PM

vlorentz accepted this revision.Sep 15 2021, 4:14 PM

vlorentz added a subscriber: vlorentz.

vlorentz added inline comments.

swh/core/tests/test_tarball.py
172

This revision is now accepted and ready to land.Sep 15 2021, 4:14 PM

anlambert added inline comments.Sep 15 2021, 4:22 PM

swh/core/tests/test_tarball.py

172

str.endswith(suffix[, start[, end]])

    Return True if the string ends with the specified suffix, otherwise return False. suffix can also be a tuple of suffixes to look for. With optional start, test beginning at that position. With optional end, stop comparing at that position.

Awesome, thanks!

Simply use of str.endswith

Build is green

Patch application report for D6271 (id=22709)

Rebasing onto 632673f6bd...

Current branch diff-target is up to date.

Changes applied before test

commit 0ef57a4dd346b6c32c1080d38a30922f765c5618
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Sep 15 14:20:11 2021 +0200

    tarball: Add support for uncompressing crate files
    
    Rust crate files are standard tar.gz archives so add support
    to uncompress them through shutil.register_unpack_format.
    
    For the record, those kind of source files archive are processed
    by the nixguix loader.

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/245/ for more details.

Harbormaster completed remote builds in B23642: Diff 22709.Sep 15 2021, 4:25 PM

Closed by commit rDCORE0ef57a4dd346: tarball: Add support for uncompressing crate files (authored by anlambert). · Explain WhySep 15 2021, 4:32 PM

This revision was automatically updated to reflect the committed changes.

anlambert added a commit: rDCORE0ef57a4dd346: tarball: Add support for uncompressing crate files.

oh i missed that, great ;)

In D6271#163819, @ardumont wrote:

oh i missed that, great ;)

I pushed rDCOREe268896d7652c7d9ccb83b702b151eb1ef88bd97 after that change which should fix the loading of tgz archives with extension unknown to shutil.unpack_archive.