Page MenuHomeSoftware Heritage

package-loaders: Add support for extid versions, and bump it for Debian
ClosedPublic

Authored by vlorentz on Dec 6 2021, 3:32 PM.

Details

Summary

The previous commit updated the format of release, and we must not reuse releases
created before, hence the bump.

Diff Detail

Event Timeline

Build has FAILED

Patch application report for D6751 (id=24514)

Could not rebase; Attempt merge onto 89f5ccc7f5...

Auto-merging docs/package-loader-specifications.rst
Merge made by the 'recursive' strategy.
 docs/package-loader-specifications.rst           |  9 +++--
 swh/loader/package/archive/loader.py             |  6 +++-
 swh/loader/package/archive/tests/test_archive.py |  1 +
 swh/loader/package/debian/loader.py              | 22 +++++++-----
 swh/loader/package/debian/tests/test_debian.py   | 44 ++++++++++++++++--------
 swh/loader/package/loader.py                     | 34 ++++++++++++------
 swh/loader/package/nixguix/loader.py             |  4 ++-
 swh/loader/package/npm/loader.py                 |  3 +-
 swh/loader/package/pypi/loader.py                |  3 +-
 swh/loader/package/tests/test_loader.py          | 20 +++++------
 10 files changed, 96 insertions(+), 50 deletions(-)
Changes applied before test
commit 5e9c804180ec6b28e49b00d2dcd1a9a9646bb66d
Merge: 89f5ccc f6350c3
Author: Jenkins user <jenkins@localhost>
Date:   Mon Dec 6 14:32:43 2021 +0000

    Merge branch 'diff-target' into HEAD

commit f6350c35b6dba51e0dfb82f83889f81b8e351f15
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 6 15:32:10 2021 +0100

    package-loaders: Add support for extid versions, and bump it for Debian
    
    The previous commit updated the format of release, and we must not reuse releases
    created before, hence the bump.

commit 2e01f10bb05215ed724d490415e8cce9ccd89b35
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 6 14:03:51 2021 +0100

    debian: Remove the extrinsic version from release names
    
    Use only the intrinsic version (eg. 1.0.0) instead of the extrinsic version
    (eg. stretch/contrib/1.0.0).
    Releases should only contain data from DSC, not external 'pointers' to them.
    
    Additionally, having extrinsic data in releases means the same
    dsc-sha256 extid can point to different releases, which meant the loader
    may reuse a release mentioning a specific suite as a release in a
    different suite.
    With this commit, this won't be a problem anymore, as releases won't
    mention the suite at all, so suites can safely share extids.

commit 958674dc24171ced169397054ffb1502b60f7f43
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 6 13:48:04 2021 +0100

    debian: Fix confusion between the two versions
    
    'version' was documented as the intrinsic version (eg. '0.7.2-3') and
    'full_version' as the one containing the suite name (eg. 'stretch/contrib/0.7.2-3').
    
    In practice, it was the opposite, except in a few incorrect test.
    
    This commit fixes said tests, and renamed 'full_version' to
    'intrinsic_version'.
    
    This is only a refactoring, the behavior is unchanged for now;
    but a future commit will remove the 'version' (which is extrinsic) from
    the release name (which should contain only data intrinsic to the DSC).

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/647/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/647/console

Harbormaster returned this revision to the author for changes because remote builds failed.Dec 6 2021, 3:33 PM
Harbormaster failed remote builds in B25388: Diff 24514!

Looks like this is missing an update to the maven loader for the new extid() return value

rebase (incl. impl for maven)

Build has FAILED

Patch application report for D6751 (id=24539)

Could not rebase; Attempt merge onto 2d9e93a2f2...

Updating 2d9e93a..6ace7d1
Fast-forward
 docs/package-loader-specifications.rst           |  9 +++--
 swh/loader/package/archive/loader.py             |  6 +++-
 swh/loader/package/archive/tests/test_archive.py |  1 +
 swh/loader/package/debian/loader.py              | 22 +++++++-----
 swh/loader/package/debian/tests/test_debian.py   | 44 ++++++++++++++++--------
 swh/loader/package/loader.py                     | 34 ++++++++++++------
 swh/loader/package/maven/loader.py               |  5 ++-
 swh/loader/package/maven/tests/test_maven.py     |  6 +++-
 swh/loader/package/nixguix/loader.py             |  4 ++-
 swh/loader/package/npm/loader.py                 |  3 +-
 swh/loader/package/pypi/loader.py                |  3 +-
 swh/loader/package/tests/test_loader.py          | 20 +++++------
 12 files changed, 105 insertions(+), 52 deletions(-)
Changes applied before test
commit 6ace7d1d7034e1a8761c8bf7ac2a2155ff99b3f7
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 6 15:32:10 2021 +0100

    package-loaders: Add support for extid versions, and bump it for Debian
    
    The previous commit updated the format of release, and we must not reuse releases
    created before, hence the bump.

commit 4632adb5af307dc48d791276a12e8aafa46975de
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 6 14:03:51 2021 +0100

    debian: Remove the extrinsic version from release names
    
    Use only the intrinsic version (eg. 1.0.0) instead of the extrinsic version
    (eg. stretch/contrib/1.0.0).
    Releases should only contain data from DSC, not external 'pointers' to them.
    
    Additionally, having extrinsic data in releases means the same
    dsc-sha256 extid can point to different releases, which meant the loader
    may reuse a release mentioning a specific suite as a release in a
    different suite.
    With this commit, this won't be a problem anymore, as releases won't
    mention the suite at all, so suites can safely share extids.

commit 26996ef3eef494eb187e01b999255cfa101ad758
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 6 13:48:04 2021 +0100

    debian: Fix confusion between the two versions
    
    'version' was documented as the intrinsic version (eg. '0.7.2-3') and
    'full_version' as the one containing the suite name (eg. 'stretch/contrib/0.7.2-3').
    
    In practice, it was the opposite, except in a few incorrect test.
    
    This commit fixes said tests, and renamed 'full_version' to
    'intrinsic_version'.
    
    This is only a refactoring, the behavior is unchanged for now;
    but a future commit will remove the 'version' (which is extrinsic) from
    the release name (which should contain only data intrinsic to the DSC).

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/651/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/651/console

Harbormaster returned this revision to the author for changes because remote builds failed.Dec 7 2021, 11:09 AM
Harbormaster failed remote builds in B25413: Diff 24539!

Build is green

Patch application report for D6751 (id=24542)

Could not rebase; Attempt merge onto 2d9e93a2f2...

Updating 2d9e93a..0fe66b7
Fast-forward
 docs/package-loader-specifications.rst           |  9 +++--
 swh/loader/package/archive/loader.py             |  6 ++-
 swh/loader/package/archive/tests/test_archive.py |  1 +
 swh/loader/package/debian/loader.py              | 22 +++++++----
 swh/loader/package/debian/tests/test_debian.py   | 50 +++++++++++++++---------
 swh/loader/package/loader.py                     | 34 +++++++++++-----
 swh/loader/package/maven/loader.py               |  5 ++-
 swh/loader/package/maven/tests/test_maven.py     |  6 ++-
 swh/loader/package/nixguix/loader.py             |  4 +-
 swh/loader/package/npm/loader.py                 |  3 +-
 swh/loader/package/pypi/loader.py                |  3 +-
 swh/loader/package/tests/test_loader.py          | 20 +++++-----
 12 files changed, 108 insertions(+), 55 deletions(-)
Changes applied before test
commit 0fe66b718638f93dbdeff145d332f3036779e1ff
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 6 15:32:10 2021 +0100

    package-loaders: Add support for extid versions, and bump it for Debian
    
    The previous commit updated the format of release, and we must not reuse releases
    created before, hence the bump.

commit c87c8c79ea660d5de697455d7c1882a216b23df5
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 6 14:03:51 2021 +0100

    debian: Remove the extrinsic version from release names
    
    Use only the intrinsic version (eg. 1.0.0) instead of the extrinsic version
    (eg. stretch/contrib/1.0.0).
    Releases should only contain data from DSC, not external 'pointers' to them.
    
    Additionally, having extrinsic data in releases means the same
    dsc-sha256 extid can point to different releases, which meant the loader
    may reuse a release mentioning a specific suite as a release in a
    different suite.
    With this commit, this won't be a problem anymore, as releases won't
    mention the suite at all, so suites can safely share extids.

commit 26996ef3eef494eb187e01b999255cfa101ad758
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 6 13:48:04 2021 +0100

    debian: Fix confusion between the two versions
    
    'version' was documented as the intrinsic version (eg. '0.7.2-3') and
    'full_version' as the one containing the suite name (eg. 'stretch/contrib/0.7.2-3').
    
    In practice, it was the opposite, except in a few incorrect test.
    
    This commit fixes said tests, and renamed 'full_version' to
    'intrinsic_version'.
    
    This is only a refactoring, the behavior is unchanged for now;
    but a future commit will remove the 'version' (which is extrinsic) from
    the release name (which should contain only data intrinsic to the DSC).

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/653/ for more details.

ardumont added 1 blocking reviewer(s): olasd.

Thanks.

This revision is now accepted and ready to land.Dec 7 2021, 1:23 PM