tarball: Add support to unpack ZIP archive with implode compression
ClosedPublic
Actions

Authored by anlambert on Jun 10 2021, 1:38 PM.

Details

Reviewers

vlorentz
ardumont

Group Reviewers

Reviewers

Maniphest Tasks

T3369: swh.core.tarballl.uncompress: Some zip files fail to be uncompressed

Commits

rDCOREcbf07d953e4a: tarball: Add support to unpack ZIP archive with implode compression

Summary

Python zipfile module does not support the unpacking of ZIP archive
with legacy compression type 6 (implode).

Fallback using the unzip command in that case as it supports the
unpacking of such ZIP file.

Closes T3369

Diff Detail

Repository

rDCORE Foundations and core functionalities

Branch

tarball-imploded-zip

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 21910
Build 34074: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 34073: arc lint + arc unit

Event Timeline

anlambert created this revision.Jun 10 2021, 1:38 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJun 10 2021, 1:38 PM

anlambert edited the summary of this revision. (Show Details)Jun 10 2021, 1:39 PM

Build is green

Patch application report for D5850 (id=20914)

Rebasing onto 5aaaa522d5...

Current branch diff-target is up to date.

Changes applied before test

commit f249b40700f2e332f74853f91a66ad25c11a3742
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Wed Jun 9 14:17:59 2021 +0200

    tarball: Add support to unpack ZIP archive with implode compression
    
    Python zipfile module does not support the unpacking of ZIP archive
    with legacy compression type 6 (implode).
    
    Fallback using the unzip command in that case as it supports the
    unpacking of such ZIP file.
    
    Closes T3369

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/229/ for more details.

Harbormaster completed remote builds in B21896: Diff 20914.Jun 10 2021, 1:40 PM

anlambert requested review of this revision.Jun 10 2021, 1:40 PM

anlambert added inline comments.Jun 10 2021, 1:40 PM

swh/core/tests/test_tarball.py
250	For the record, I did not manage to find a command enabling to generate a ZIP archive with compression type 6 on linux so I used such an an archive found in the wild instead as test data.

vlorentz accepted this revision.Jun 10 2021, 2:12 PM

vlorentz added a subscriber: vlorentz.

vlorentz added inline comments.

swh/core/tarball.py
96–100	to show the original traceback

This revision is now accepted and ready to land.Jun 10 2021, 2:12 PM

Apply @vlorentz comment

Build is green

Patch application report for D5850 (id=20928)

Rebasing onto 5aaaa522d5...

Current branch diff-target is up to date.

Changes applied before test

commit cbf07d953e4ac00280f5f62296e8b8585f77d7ac
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Wed Jun 9 14:17:59 2021 +0200

    tarball: Add support to unpack ZIP archive with implode compression
    
    Python zipfile module does not support the unpacking of ZIP archive
    with legacy compression type 6 (implode).
    
    Fallback using the unzip command in that case as it supports the
    unpacking of such ZIP file.
    
    Closes T3369

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/230/ for more details.

Harbormaster completed remote builds in B21910: Diff 20928.Jun 10 2021, 3:27 PM