Paths

Table of Contentst

Differential D8758

swh.core.tarball: Wire support to existing tarball
ClosedPublic
Actions

Authored by ardumont on Oct 24 2022, 4:38 PM.

Details

Reviewers

anlambert

Group Reviewers

Reviewers

Maniphest Tasks

T3781: Replace the Nixguix loader with a lister

Commits

rDCORE306a74d01a99: swh.core.tarball: Add support for war files as well

Summary

They are already supported but it's not wired in the module.

Related to T3781#97784

Diff Detail

Repository

rDCORE Foundations and core functionalities

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Oct 24 2022, 4:38 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptOct 24 2022, 4:38 PM

ardumont mentioned this in D8757: Add support for more tarball/zip extensions.Oct 24 2022, 4:39 PM

Fix typos

Build has FAILED

Patch application report for D8758 (id=31575)

Rebasing onto 7561ed9016...

Current branch diff-target is up to date.

Changes applied before test

commit e863f21d94a936237f7a34db1fcc0cc4c5a490a3
Author: Antoine R. Dumont (@ardumont) <antoine.romain.dumont@gmail.com>
Date:   Mon Oct 24 16:34:55 2022 +0200

    swh.core.tarball: Wire support to existing tarball
    
    They are already supported but it's not wired in the module.
    
    Related to T3781#97784

Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/485/
See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/485/console

Harbormaster failed remote builds in B32529: Diff 31575!Oct 24 2022, 4:40 PM

Build has FAILED

Patch application report for D8758 (id=31576)

Rebasing onto 7561ed9016...

Current branch diff-target is up to date.

Changes applied before test

commit 671124d9ec3aad6dfd81f23f0df5b9a2731a950d
Author: Antoine R. Dumont (@ardumont) <antoine.romain.dumont@gmail.com>
Date:   Mon Oct 24 16:34:55 2022 +0200

    swh.core.tarball: Wire support to existing tarball
    
    They are already supported but it's not wired in the module.
    
    Related to T3781#97784

Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/486/
See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/486/console

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 24 2022, 4:42 PM

Harbormaster failed remote builds in B32530: Diff 31576!

anlambert added a subscriber: anlambert.Oct 24 2022, 4:58 PM

anlambert added inline comments.

swh/core/tarball.py
252–256	Those can be removed as mime type is used as last resort to guess tarball format (check `MIMETYPE_TO_ARCHIVE_FORMAT` in the same file).

ardumont added inline comments.Oct 24 2022, 5:06 PM

swh/core/tarball.py
252–256	yes, thanks, i have it locally. I'm just trying to get why the build fails prior to update the diff.

Fix test

swh/core/tarball.py
252–256	found it.

ardumont added inline comments.Oct 24 2022, 5:18 PM

swh/core/tarball.py

252–256

found it

I meant the issue with the build.

Those can be removed as mime type is used as last resort to guess tarball format
(check MIMETYPE_TO_ARCHIVE_FORMAT in the same file).

I've seen the code you mentioned indeed. And i've double checked it against the new
extensions i've found.

It's exactly as it needs to be processed so great ;)

In [1]: from pathlib import Path

In [2]: dir_ = Path('/var/tmp/nixguix/manifest-files-output/')

In [3]: all_tarballs = list(dir_.iterdir())

In [4]: import magic

In [5]: m = magic.Magic(mime=True)

In [12]: [(m.from_file(str(tarpath)), str(tarpath)) for tarpath in all_tarballs if str(tarpath).endswith(('.whl', '.love', '.pak', '.oxt', '.vsix', '.VSIXPackage'))]
Out[12]:
[('application/zip',
  'manifest-files-output/Microsoft.VisualStudio.Services.VSIXPackage'),
 ('application/zip', 'manifest-files-output/da_DK-2.5.189.oxt'),
 ('application/zip', 'manifest-files-output/vapor_dbf509f.love'),
 ('application/zip', 'manifest-files-output/ms-python-release.vsix'),
 ('application/zip',
  'manifest-files-output/streamlit-0.50.2-py2.py3-none-any.whl'),
 ('application/zip', 'manifest-files-output/NotoFonts.pak')]

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 25 2022, 10:10 AM

Harbormaster failed remote builds in B32533: Diff 31579!

Build is green

Patch application report for D8758 (id=31579)

Rebasing onto 7561ed9016...

Current branch diff-target is up to date.

Changes applied before test

commit 306a74d01a997832a5f16d29711e32ffd35a47c4
Author: Antoine R. Dumont (@ardumont) <antoine.romain.dumont@gmail.com>
Date:   Mon Oct 24 16:34:55 2022 +0200

    swh.core.tarball: Add support for war files as well
    
    It's already supported [1] but not wired in this module.
    
    Related to T3781#97784

See https://jenkins.softwareheritage.org/job/DCORE/job/tests-on-diff/489/ for more details.

Harbormaster completed remote builds in B32533: Diff 31579.Oct 25 2022, 10:12 AM

ardumont requested review of this revision.Oct 25 2022, 10:12 AM

anlambert accepted this revision.Oct 25 2022, 11:04 AM

This revision is now accepted and ready to land.Oct 25 2022, 11:04 AM

Closed by commit rDCORE306a74d01a99: swh.core.tarball: Add support for war files as well (authored by ardumont). · Explain WhyOct 25 2022, 11:57 AM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDCORE306a74d01a99: swh.core.tarball: Add support for war files as well.

ardumont mentioned this in T3781: Replace the Nixguix loader with a lister.Oct 25 2022, 2:50 PM