Page MenuHomeSoftware Heritage

cpan: Do not parse intrinsic metadata for getting module author
ClosedPublic

Authored by anlambert on Oct 10 2022, 5:04 PM.

Details

Summary

Parsing perl module metadata files trigger a lot of errors due to badly
formatted JSON or YAML and module author info is already provided by
the cpan lister as extra loader arguments so remove that no longer
needed metadata parsing step.

Related to T2833

Depends on D8616

Diff Detail

Repository
rDLDBASE Generic VCS/Package Loader
Branch
cpan-no-intrinsic-metadata-parsing
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 32188
Build 50407: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 50406: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8651 (id=31235)

Could not rebase; Attempt merge onto 028b7c04b9...

Updating 028b7c0..3028b78
Fast-forward
 swh/loader/package/cpan/loader.py                  | 127 ++++++---------------
 swh/loader/package/cpan/tests/data/fake_cpan.sh    |  86 --------------
 .../v1_release_versions_Internals-CountObjects     |  26 -----
 swh/loader/package/cpan/tests/test_cpan.py         | 111 ++++++++++++------
 swh/loader/package/cpan/tests/test_tasks.py        |   8 +-
 5 files changed, 117 insertions(+), 241 deletions(-)
 delete mode 100644 swh/loader/package/cpan/tests/data/fake_cpan.sh
 delete mode 100644 swh/loader/package/cpan/tests/data/https_fastapi.metacpan.org/v1_release_versions_Internals-CountObjects
Changes applied before test
commit 3028b7894270e2e0fd67c49afba44bd03fdb1e20
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Oct 10 13:32:05 2022 +0200

    cpan: Do not parse intrinsic metadata for getting module author
    
    Parsing perl module metadata files trigger a lot of errors due to badly
    formatted JSON or YAML and module author info is already provided by
    the cpan lister as extra loader arguments so remove that no longer
    needed metadata parsing step.
    
    Related to T2833

commit 819f9d2702c193497a9ed99b17d58192aeb4ab9b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 29 20:36:38 2022 +0200

    cpan: Align loader implementation with latest lister improvements
    
    Artifacts info for a package are now provided as loader arguments so
    no need to query metacpan Web API anymore to get list of versions
    and their related info.
    
    Related to T2833

commit e53a1e17aad238e38690a8a42b3672d106e179ae
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 7 15:33:35 2022 +0200

    cpan: Remove module description from release message
    
    Module description is not related to a particular release so we
    should not add it in release message.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/983/ for more details.

How come this doesn't affect tests?

How come this doesn't affect tests?

Because author info have been provided as loader extra arguments in tests in previous diff.

Build is green

Patch application report for D8651 (id=31256)

Could not rebase; Attempt merge onto 4cb85e153e...

Updating 4cb85e1..7b92960
Fast-forward
 swh/loader/package/cpan/loader.py                  | 129 ++++++---------------
 swh/loader/package/cpan/tests/data/fake_cpan.sh    |  86 --------------
 .../v1_release_versions_Internals-CountObjects     |  26 -----
 swh/loader/package/cpan/tests/test_cpan.py         | 113 ++++++++++++------
 swh/loader/package/cpan/tests/test_tasks.py        |  14 ++-
 5 files changed, 127 insertions(+), 241 deletions(-)
 delete mode 100644 swh/loader/package/cpan/tests/data/fake_cpan.sh
 delete mode 100644 swh/loader/package/cpan/tests/data/https_fastapi.metacpan.org/v1_release_versions_Internals-CountObjects
Changes applied before test
commit 7b929606a78f38b48ffc6b966d74bc0d7aea8ce3
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Oct 10 13:32:05 2022 +0200

    cpan: Do not parse intrinsic metadata for getting module author
    
    Parsing perl module metadata files trigger a lot of errors due to badly
    formatted JSON or YAML and module author info is already provided by
    the cpan lister as extra loader arguments so remove that no longer
    needed metadata parsing step.
    
    Related to T2833

commit a13e3e6f35bcabf856664ad7f116b17ca5a3daaf
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 29 20:36:38 2022 +0200

    cpan: Align loader implementation with latest lister improvements
    
    Artifacts info for a package are now provided as loader arguments so
    no need to query metacpan Web API anymore to get list of versions
    and their related info.
    
    Related to T2833

commit e17ee9e08e84105710852bcc32c81bf149e55d4c
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Oct 7 15:33:35 2022 +0200

    cpan: Remove module description from release message
    
    Module description is not related to a particular release so we
    should not add it in release message.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/991/ for more details.

This revision is now accepted and ready to land.Oct 17 2022, 1:06 PM