Page MenuHomeSoftware Heritage
Create Task
Maniphest T3575

Filter out sdist archives that are not of interest
Closed, MigratedEdits Locked
Actions

Description

Some PyPI origins declare sdist archives in formats that are not supported by swh.core.tarball.uncompress,
for instance .deb, .egg, .rpm or .whl. This leads to errors when trying to load such origins and generates a lot of noise in
sentry reports.

As the content of those archives does not match standard sdist format (there is no PKG-INFO file for instance),
we should filter them out from the artifacts to process for a PyPI origin.

Revisions and Commits

rDLDBASE Generic VCS/Package Loader
D6270rDLDBASE732999842159 pypi/loader: Filter out sdist archives not of interest

Related Objects

Event Timeline

anlambert triaged this task as Normal priority.Sep 15 2021, 1:40 PM
anlambert created this task.
anlambert mentioned this in T3576: Add support for Python eggs file processing .Sep 15 2021, 2:47 PM
anlambert renamed this task from Filter out sdist archives that cannot be extracted by swh.core.tarball.uncompress to Filter out sdist archives that are not of interest.Sep 15 2021, 2:52 PM
anlambert updated the task description. (Show Details)
anlambert added a revision: D6270: pypi/loader: Filter out sdist archives not of interest.Sep 15 2021, 3:39 PM
anlambert added a commit: rDLDBASE732999842159: pypi/loader: Filter out sdist archives not of interest.Sep 16 2021, 10:42 AM
anlambert closed this task as Resolved.Oct 15 2021, 5:30 PM
anlambert claimed this task.

Fix has been deployed to production, closing this.

gitlab-migration changed the task status from Resolved to Migrated.Jan 8 2023, 4:35 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal