Page MenuHomeSoftware Heritage

swh.core.tarball: Wire support to existing tarball
ClosedPublic

Authored by ardumont on Oct 24 2022, 4:38 PM.

Details

Summary

They are already supported but it's not wired in the module.

Related to T3781#97784

Diff Detail

Event Timeline

Build has FAILED

Patch application report for D8758 (id=31575)

Rebasing onto 7561ed9016...

Current branch diff-target is up to date.
Changes applied before test
commit e863f21d94a936237f7a34db1fcc0cc4c5a490a3
Author: Antoine R. Dumont (@ardumont) <antoine.romain.dumont@gmail.com>
Date:   Mon Oct 24 16:34:55 2022 +0200

    swh.core.tarball: Wire support to existing tarball
    
    They are already supported but it's not wired in the module.
    
    Related to T3781#97784

Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/485/
See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/485/console

Build has FAILED

Patch application report for D8758 (id=31576)

Rebasing onto 7561ed9016...

Current branch diff-target is up to date.
Changes applied before test
commit 671124d9ec3aad6dfd81f23f0df5b9a2731a950d
Author: Antoine R. Dumont (@ardumont) <antoine.romain.dumont@gmail.com>
Date:   Mon Oct 24 16:34:55 2022 +0200

    swh.core.tarball: Wire support to existing tarball
    
    They are already supported but it's not wired in the module.
    
    Related to T3781#97784

Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/486/
See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/486/console

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 24 2022, 4:42 PM
Harbormaster failed remote builds in B32530: Diff 31576!
anlambert added inline comments.
swh/core/tarball.py
253–257

Those can be removed as mime type is used as last resort to guess tarball format (check MIMETYPE_TO_ARCHIVE_FORMAT in the same file).

swh/core/tarball.py
253–257

yes, thanks, i have it locally.
I'm just trying to get why the build fails prior to update the diff.

ardumont marked an inline comment as done.

Fix test

swh/core/tarball.py
253–257

found it.

swh/core/tarball.py
253–257

found it

I meant the issue with the build.

Those can be removed as mime type is used as last resort to guess tarball format
(check MIMETYPE_TO_ARCHIVE_FORMAT in the same file).

I've seen the code you mentioned indeed. And i've double checked it against the new
extensions i've found.

It's exactly as it needs to be processed so great ;)

In [1]: from pathlib import Path

In [2]: dir_ = Path('/var/tmp/nixguix/manifest-files-output/')

In [3]: all_tarballs = list(dir_.iterdir())

In [4]: import magic

In [5]: m = magic.Magic(mime=True)

In [12]: [(m.from_file(str(tarpath)), str(tarpath)) for tarpath in all_tarballs if str(tarpath).endswith(('.whl', '.love', '.pak', '.oxt', '.vsix', '.VSIXPackage'))]
Out[12]:
[('application/zip',
  'manifest-files-output/Microsoft.VisualStudio.Services.VSIXPackage'),
 ('application/zip', 'manifest-files-output/da_DK-2.5.189.oxt'),
 ('application/zip', 'manifest-files-output/vapor_dbf509f.love'),
 ('application/zip', 'manifest-files-output/ms-python-release.vsix'),
 ('application/zip',
  'manifest-files-output/streamlit-0.50.2-py2.py3-none-any.whl'),
 ('application/zip', 'manifest-files-output/NotoFonts.pak')]
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 25 2022, 10:10 AM
Harbormaster failed remote builds in B32533: Diff 31579!

Build is green

Patch application report for D8758 (id=31579)

Rebasing onto 7561ed9016...

Current branch diff-target is up to date.
Changes applied before test
commit 306a74d01a997832a5f16d29711e32ffd35a47c4
Author: Antoine R. Dumont (@ardumont) <antoine.romain.dumont@gmail.com>
Date:   Mon Oct 24 16:34:55 2022 +0200

    swh.core.tarball: Add support for war files as well
    
    It's already supported [1] but not wired in this module.
    
    Related to T3781#97784

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/489/ for more details.

This revision is now accepted and ready to land.Oct 25 2022, 11:04 AM