Differential D8651

cpan: Do not parse intrinsic metadata for getting module author
ClosedPublic
Actions

Authored by anlambert on Oct 10 2022, 5:04 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Maniphest Tasks

T2833: cpan.loader - archive Perl modules from CPAN

Commits

rDLDBASE7b929606a78f: cpan: Do not parse intrinsic metadata for getting module author

Summary

Parsing perl module metadata files trigger a lot of errors due to badly
formatted JSON or YAML and module author info is already provided by
the cpan lister as extra loader arguments so remove that no longer
needed metadata parsing step.

Related to T2833

Depends on D8616

Diff Detail

Repository

rDLDBASE Generic VCS/Package Loader

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Oct 10 2022, 5:04 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptOct 10 2022, 5:04 PM

anlambert added a child revision: D8652: cpan: Collect extrinsic metadata for each module release.Oct 10 2022, 5:04 PM

Build is green

Patch application report for D8651 (id=31235)

Could not rebase; Attempt merge onto 028b7c04b9...

Updating 028b7c0..3028b78
Fast-forward
 swh/loader/package/cpan/loader.py                  | 127 ++++++---------------
 swh/loader/package/cpan/tests/data/fake_cpan.sh    |  86 --------------
 .../v1_release_versions_Internals-CountObjects     |  26 -----
 swh/loader/package/cpan/tests/test_cpan.py         | 111 ++++++++++++------
 swh/loader/package/cpan/tests/test_tasks.py        |   8 +-
 5 files changed, 117 insertions(+), 241 deletions(-)
 delete mode 100644 swh/loader/package/cpan/tests/data/fake_cpan.sh
 delete mode 100644 swh/loader/package/cpan/tests/data/https_fastapi.metacpan.org/v1_release_versions_Internals-CountObjects

Changes applied before test

commit 3028b7894270e2e0fd67c49afba44bd03fdb1e20
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Oct 10 13:32:05 2022 +0200

    cpan: Do not parse intrinsic metadata for getting module author
    
    Parsing perl module metadata files trigger a lot of errors due to badly
    formatted JSON or YAML and module author info is already provided by
    the cpan lister as extra loader arguments so remove that no longer
    needed metadata parsing step.
    
    Related to T2833

commit 819f9d2702c193497a9ed99b17d58192aeb4ab9b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 29 20:36:38 2022 +0200

    cpan: Align loader implementation with latest lister improvements
    
    Artifacts info for a package are now provided as loader arguments so
    no need to query metacpan Web API anymore to get list of versions
    and their related info.
    
    Related to T2833

commit e53a1e17aad238e38690a8a42b3672d106e179ae
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 7 15:33:35 2022 +0200

    cpan: Remove module description from release message
    
    Module description is not related to a particular release so we
    should not add it in release message.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/983/ for more details.

Harbormaster completed remote builds in B32188: Diff 31235.Oct 10 2022, 5:08 PM

anlambert requested review of this revision.Oct 10 2022, 5:08 PM

How come this doesn't affect tests?

In D8651#225221, @vlorentz wrote:

How come this doesn't affect tests?

Because author info have been provided as loader extra arguments in tests in previous diff.

Rebase

Build is green

Patch application report for D8651 (id=31256)

Could not rebase; Attempt merge onto 4cb85e153e...

Updating 4cb85e1..7b92960
Fast-forward
 swh/loader/package/cpan/loader.py                  | 129 ++++++---------------
 swh/loader/package/cpan/tests/data/fake_cpan.sh    |  86 --------------
 .../v1_release_versions_Internals-CountObjects     |  26 -----
 swh/loader/package/cpan/tests/test_cpan.py         | 113 ++++++++++++------
 swh/loader/package/cpan/tests/test_tasks.py        |  14 ++-
 5 files changed, 127 insertions(+), 241 deletions(-)
 delete mode 100644 swh/loader/package/cpan/tests/data/fake_cpan.sh
 delete mode 100644 swh/loader/package/cpan/tests/data/https_fastapi.metacpan.org/v1_release_versions_Internals-CountObjects

Changes applied before test

commit 7b929606a78f38b48ffc6b966d74bc0d7aea8ce3
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Oct 10 13:32:05 2022 +0200

    cpan: Do not parse intrinsic metadata for getting module author
    
    Parsing perl module metadata files trigger a lot of errors due to badly
    formatted JSON or YAML and module author info is already provided by
    the cpan lister as extra loader arguments so remove that no longer
    needed metadata parsing step.
    
    Related to T2833

commit a13e3e6f35bcabf856664ad7f116b17ca5a3daaf
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 29 20:36:38 2022 +0200

    cpan: Align loader implementation with latest lister improvements
    
    Artifacts info for a package are now provided as loader arguments so
    no need to query metacpan Web API anymore to get list of versions
    and their related info.
    
    Related to T2833

commit e17ee9e08e84105710852bcc32c81bf149e55d4c
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 7 15:33:35 2022 +0200

    cpan: Remove module description from release message
    
    Module description is not related to a particular release so we
    should not add it in release message.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/991/ for more details.

Harbormaster completed remote builds in B32208: Diff 31256.Oct 11 2022, 2:45 PM

vlorentz accepted this revision.Oct 17 2022, 1:06 PM

This revision is now accepted and ready to land.Oct 17 2022, 1:06 PM

Closed by commit rDLDBASE7b929606a78f: cpan: Do not parse intrinsic metadata for getting module author (authored by anlambert). · Explain WhyOct 17 2022, 5:32 PM

This revision was automatically updated to reflect the committed changes.

anlambert added a commit: rDLDBASE7b929606a78f: cpan: Do not parse intrinsic metadata for getting module author.