Differential D8616

cpan: Align loader implementation with latest lister improvements
ClosedPublic
Actions

Authored by anlambert on Oct 4 2022, 5:24 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Maniphest Tasks

T2833: cpan.loader - archive Perl modules from CPAN

Commits

rDLDBASEa13e3e6f35bc: cpan: Align loader implementation with latest lister improvements

Summary

Artifacts info for a package are now provided as loader arguments so
no need to query metacpan Web API anymore.

Also remove package description in each release message as it does not
describe a release but rather the package.

Related to T2833

Depends on D8615
Depends on D8650

Diff Detail

Repository

rDLDBASE Generic VCS/Package Loader

Branch

cpan-loader-improvements

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 32207
Build 50441: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 50440: arc lint + arc unit

Event Timeline

anlambert created this revision.Oct 4 2022, 5:24 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptOct 4 2022, 5:24 PM

Build is green

Patch application report for D8616 (id=31114)

Rebasing onto b26b98810d...

Current branch diff-target is up to date.

Changes applied before test

commit 213add6830e132cb80bee218b09809bec29551ef
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 29 20:36:38 2022 +0200

    cpan: Align loader implementation with latest lister improvements
    
    Artifacts info for a package are now provided as loader arguments so
    no need to query metacpan Web API anymore.
    
    Also remove package description in each release message as it does not
    describe a release but rather the package.
    
    Related to T2833

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/955/ for more details.

Harbormaster completed remote builds in B32084: Diff 31114.Oct 4 2022, 5:28 PM

anlambert requested review of this revision.Oct 4 2022, 5:28 PM

Also remove package description in each release message as it does not
describe a release but rather the package.

Please mention it in the diff/commit title, or make it its own commit. It's a meaningful change that should be visible.

swh/loader/package/cpan/loader.py
44–45

anlambert planned changes to this revision.Oct 7 2022, 12:31 PM

anlambert edited the summary of this revision. (Show Details)Oct 10 2022, 5:00 PM

anlambert added a parent revision: D8650: cpan: Remove module description from release message.

Rebase and move description removal from release message in separate commit and diff

anlambert added a child revision: D8651: cpan: Do not parse intrinsic metadata for getting module author.Oct 10 2022, 5:04 PM

Build is green

Patch application report for D8616 (id=31234)

Could not rebase; Attempt merge onto 028b7c04b9...

Updating 028b7c0..819f9d2
Fast-forward
 swh/loader/package/cpan/loader.py                  |  87 +++++++---------
 swh/loader/package/cpan/tests/data/fake_cpan.sh    |  86 ----------------
 .../v1_release_versions_Internals-CountObjects     |  26 -----
 swh/loader/package/cpan/tests/test_cpan.py         | 111 +++++++++++++++------
 swh/loader/package/cpan/tests/test_tasks.py        |   8 +-
 5 files changed, 119 insertions(+), 199 deletions(-)
 delete mode 100644 swh/loader/package/cpan/tests/data/fake_cpan.sh
 delete mode 100644 swh/loader/package/cpan/tests/data/https_fastapi.metacpan.org/v1_release_versions_Internals-CountObjects

Changes applied before test

commit 819f9d2702c193497a9ed99b17d58192aeb4ab9b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 29 20:36:38 2022 +0200

    cpan: Align loader implementation with latest lister improvements
    
    Artifacts info for a package are now provided as loader arguments so
    no need to query metacpan Web API anymore to get list of versions
    and their related info.
    
    Related to T2833

commit e53a1e17aad238e38690a8a42b3672d106e179ae
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 7 15:33:35 2022 +0200

    cpan: Remove module description from release message
    
    Module description is not related to a particular release so we
    should not add it in release message.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/982/ for more details.

Harbormaster completed remote builds in B32187: Diff 31234.Oct 10 2022, 5:06 PM

thanks

This revision is now accepted and ready to land.Oct 11 2022, 10:31 AM

Rebase and synchronize again with latest cpan lister changes

Build is green

Patch application report for D8616 (id=31255)

Could not rebase; Attempt merge onto 4cb85e153e...

Updating 4cb85e1..a13e3e6
Fast-forward
 swh/loader/package/cpan/loader.py                  |  89 +++++++---------
 swh/loader/package/cpan/tests/data/fake_cpan.sh    |  86 ----------------
 .../v1_release_versions_Internals-CountObjects     |  26 -----
 swh/loader/package/cpan/tests/test_cpan.py         | 113 +++++++++++++++------
 swh/loader/package/cpan/tests/test_tasks.py        |  14 ++-
 5 files changed, 129 insertions(+), 199 deletions(-)
 delete mode 100644 swh/loader/package/cpan/tests/data/fake_cpan.sh
 delete mode 100644 swh/loader/package/cpan/tests/data/https_fastapi.metacpan.org/v1_release_versions_Internals-CountObjects

Changes applied before test

commit a13e3e6f35bcabf856664ad7f116b17ca5a3daaf
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 29 20:36:38 2022 +0200

    cpan: Align loader implementation with latest lister improvements
    
    Artifacts info for a package are now provided as loader arguments so
    no need to query metacpan Web API anymore to get list of versions
    and their related info.
    
    Related to T2833

commit e17ee9e08e84105710852bcc32c81bf149e55d4c
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 7 15:33:35 2022 +0200

    cpan: Remove module description from release message
    
    Module description is not related to a particular release so we
    should not add it in release message.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/990/ for more details.

Harbormaster completed remote builds in B32207: Diff 31255.Oct 11 2022, 2:44 PM

Closed by commit rDLDBASEa13e3e6f35bc: cpan: Align loader implementation with latest lister improvements (authored by anlambert). · Explain WhyOct 12 2022, 11:38 AM

This revision was automatically updated to reflect the committed changes.

anlambert added a commit: rDLDBASEa13e3e6f35bc: cpan: Align loader implementation with latest lister improvements.

franckbret added a subscriber: franckbret.Oct 13 2022, 8:43 AM

franckbret added inline comments.

swh/loader/package/cpan/loader.py
72	api_base_url should be removed too?

vlorentz added inline comments.Oct 13 2022, 10:05 AM

swh/loader/package/cpan/loader.py
72	indeed

anlambert added inline comments.Oct 13 2022, 10:47 AM

swh/loader/package/cpan/loader.py
72	No, it is used in D8652 to compute extrinsic metadata URLs (@vlorentz, you asked me to do that computation in the loader instead of the lister, see D8615).