Ingest source code archives available from the Python Package Index (PyPI)
- source code (Git)
- issues
Details
Details
Description
Jan 8 2023
Jan 8 2023
gitlab-migration closed T4512: Strip first level directory when loading tarballs from PyPI? as Migrated.
gitlab-migration changed the status of T3576: Add support for Python eggs file processing from Invalid to Migrated.
gitlab-migration changed the status of T3575: Filter out sdist archives that are not of interest from Resolved to Migrated.
gitlab-migration changed the status of T2667: Decide what to do with PyPI snapshot metadata from Resolved to Migrated.
gitlab-migration changed the status of T1396: loader-pypi: Snapshot with null branch are badly handled by loader from Resolved to Migrated.
gitlab-migration changed the status of T1206: PyPI loader fails when the metadata is incomplete from Resolved to Migrated.
Oct 19 2022
Oct 19 2022
Sep 8 2022
Sep 8 2022
vlorentz added a comment to T4512: Strip first level directory when loading tarballs from PyPI?.
anlambert added a comment to T4512: Strip first level directory when loading tarballs from PyPI?.
But clearly it's not great. I wonder if we could do something about this in swh-web instead
olasd added a comment to T4512: Strip first level directory when loading tarballs from PyPI?.
I agree that the UX of switching branches from a release to another on snapshots of PyPI origins is not good.
vlorentz added a comment to T4512: Strip first level directory when loading tarballs from PyPI?.
I don't think it would be appropriate to remove that directory; we try to reproduce tarball faithfully. And there might be other entries at the root (eg. when loading .jar, there would typically be only two directories at the root).
vlorentz triaged T4512: Strip first level directory when loading tarballs from PyPI? as Normal priority.
Oct 15 2021
Oct 15 2021
Fix has been deployed to production, closing this.
Sep 15 2021
Sep 15 2021
anlambert renamed T3575: Filter out sdist archives that are not of interest from Filter out sdist archives that cannot be extracted by swh.core.tarball.uncompress to Filter out sdist archives that are not of interest.
Apparently we decided not to archive them so better filtering those files out as proposed in T3575.
anlambert updated the task description for T3576: Add support for Python eggs file processing .
anlambert updated the task description for T3576: Add support for Python eggs file processing .
anlambert updated the task description for T3576: Add support for Python eggs file processing .
Sep 13 2021
Sep 13 2021
anlambert triaged T3570: Upgrade python3-pkginfo debian package to latest upstream version as Normal priority.
Jul 29 2021
Jul 29 2021
Nov 2 2020
Nov 2 2020
Oct 13 2020
Oct 13 2020
vlorentz added a comment to T2667: Decide what to do with PyPI snapshot metadata.
We don't keep the binary indexes from Debian repositories, for instance.
olasd added a comment to T2667: Decide what to do with PyPI snapshot metadata.
So they're metadata specific to files that we don't archive at all because they're not source? That doesn't sound very useful to keep at all. We don't keep the binary indexes from Debian repositories, for instance.
vlorentz added a comment to T2667: Decide what to do with PyPI snapshot metadata.
They are metadata on the file itself (file name, checksums, has signature, upload time, file-specific comment (often empty), yank status), so they have nothing in common
olasd added a comment to T2667: Decide what to do with PyPI snapshot metadata.
In practice, is there many meaningful differences between the wheel metadata and the sdist metadata? If not then I think option 3 would be the most sensible.
vlorentz updated the task description for T2667: Decide what to do with PyPI snapshot metadata.
Oct 6 2020
Oct 6 2020
Oct 1 2019
Oct 1 2019
Sep 24 2019
Sep 24 2019
ardumont added a comment to D2028: [wip] package-loader: Implement a common package loader mechanism.
roh, i did not want to close it...
I just wanted to update the diff... oh, well!
ardumont updated the summary of D2028: [wip] package-loader: Implement a common package loader mechanism.
Nov 30 2018
Nov 30 2018
ardumont closed T1396: loader-pypi: Snapshot with null branch are badly handled by loader as Resolved.
And deployed.
Nov 29 2018
Nov 29 2018
ardumont renamed T1396: loader-pypi: Snapshot with null branch are badly handled by loader from loader-pypi: badly(?) formatted snapshot targets None releases to loader-pypi: Snapshot with null branch are badly handled by loader.
ardumont added a comment to T1396: loader-pypi: Snapshot with null branch are badly handled by loader.
Then I think these snapshots do look as expected, and the surrounding code should be adapted :)
olasd added a comment to T1396: loader-pypi: Snapshot with null branch are badly handled by loader.
Then I think these snapshots do look as expected, and the surrounding code should be adapted :)
anlambert added a comment to T1396: loader-pypi: Snapshot with null branch are badly handled by loader.
Looks related to source packages without the presence of the PKG-INFO file, see debug output of the loader below:
olasd added a comment to T1396: loader-pypi: Snapshot with null branch are badly handled by loader.
I'd expect branches will have a null target if that release only has binary distributions, but that's not the case for configpy. Needs to be investigated further
ardumont triaged T1396: loader-pypi: Snapshot with null branch are badly handled by loader as Normal priority.
Oct 20 2018
Oct 20 2018
Oct 19 2018
Oct 19 2018