tarball: Add support to unpack ZIP archive with implode compression
ClosedPublic
Actions

Authored by anlambert on Jun 10 2021, 1:38 PM.

Details

Reviewers

vlorentz
ardumont

Group Reviewers

Reviewers

Maniphest Tasks

T3369: swh.core.tarballl.uncompress: Some zip files fail to be uncompressed

Commits

rDCOREcbf07d953e4a: tarball: Add support to unpack ZIP archive with implode compression

Summary

Python zipfile module does not support the unpacking of ZIP archive
with legacy compression type 6 (implode).

Fallback using the unzip command in that case as it supports the
unpacking of such ZIP file.

Closes T3369

Diff Detail

Repository

rDCORE Foundations and core functionalities

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Jun 10 2021, 1:38 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJun 10 2021, 1:38 PM

anlambert edited the summary of this revision. (Show Details)Jun 10 2021, 1:39 PM

Build is green

Patch application report for D5850 (id=20914)

Rebasing onto 5aaaa522d5...

Current branch diff-target is up to date.

Changes applied before test

commit f249b40700f2e332f74853f91a66ad25c11a3742
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Wed Jun 9 14:17:59 2021 +0200

    tarball: Add support to unpack ZIP archive with implode compression
    
    Python zipfile module does not support the unpacking of ZIP archive
    with legacy compression type 6 (implode).
    
    Fallback using the unzip command in that case as it supports the
    unpacking of such ZIP file.
    
    Closes T3369

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/229/ for more details.

Harbormaster completed remote builds in B21896: Diff 20914.Jun 10 2021, 1:40 PM

anlambert requested review of this revision.Jun 10 2021, 1:40 PM

anlambert added inline comments.Jun 10 2021, 1:40 PM

swh/core/tests/test_tarball.py
250	For the record, I did not manage to find a command enabling to generate a ZIP archive with compression type 6 on linux so I used such an an archive found in the wild instead as test data.

vlorentz accepted this revision.Jun 10 2021, 2:12 PM

vlorentz added a subscriber: vlorentz.

vlorentz added inline comments.

swh/core/tarball.py
96–100	to show the original traceback

This revision is now accepted and ready to land.Jun 10 2021, 2:12 PM

Apply @vlorentz comment

Build is green

Patch application report for D5850 (id=20928)

Rebasing onto 5aaaa522d5...

Current branch diff-target is up to date.

Changes applied before test

commit cbf07d953e4ac00280f5f62296e8b8585f77d7ac
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Wed Jun 9 14:17:59 2021 +0200

    tarball: Add support to unpack ZIP archive with implode compression
    
    Python zipfile module does not support the unpacking of ZIP archive
    with legacy compression type 6 (implode).
    
    Fallback using the unzip command in that case as it supports the
    unpacking of such ZIP file.
    
    Closes T3369

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/230/ for more details.

Harbormaster completed remote builds in B21910: Diff 20928.Jun 10 2021, 3:27 PM