Page MenuHomeSoftware Heritage

pubdev: Modify loader after origin URL modification in related lister
ClosedPublic

Authored by anlambert on Aug 31 2022, 11:38 AM.

Details

Summary

Origin URL for a package listed by the pubdev lister is now in the form
https://pub.dev/packages/<package_name> so we need to reconstruct the
API URL to get package versions from it.

Related to T4465

Depends on D8354

Diff Detail

Repository
rDLDBASE Generic VCS/Package Loader
Branch
pubdev-loader-url-modification
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 31308
Build 48974: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 48973: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8355 (id=30161)

Rebasing onto 68e68e3f92...

Current branch diff-target is up to date.
Changes applied before test
commit 89d0302872b4c7098c304f5cd39aa9467b65fb9d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Aug 31 11:36:22 2022 +0200

    pubdev: Simplify loader implementation
    
    No need to fetch package versions info anymore as it will be
    provided by the pubdev lister as loader extra argument.
    
    Related to T4465

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/849/ for more details.

Align with latest lister changes in D8354

anlambert retitled this revision from pubdev: Simplify loader implementation to pubdev: Modify loader after origin URL modification in related lister.Sep 2 2022, 4:55 PM
anlambert edited the summary of this revision. (Show Details)

Build is green

Patch application report for D8355 (id=30271)

Rebasing onto 68e68e3f92...

Current branch diff-target is up to date.
Changes applied before test
commit 435e4336ed26ce690693953be6562b54bcc5fba7
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Sep 2 16:42:25 2022 +0200

    pubdev: Modify loader after origin URL modification in related lister
    
    Origin URL for a package listed by the pubdev lister is now in the form
    https://pub.dev/packages/<package_name> so we need to reconstruct the
    API URL to get package versions from it.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/854/ for more details.

what are the changes to data files? just pretty-printed?

swh/loader/package/pubdev/loader.py
74–76

you should assert the URL starts with the expected prefix

what are the changes to data files? just pretty-printed?

Yes, they are hard to read when working on tests otherwise. I can make a dedicated commit if you prefer.

swh/loader/package/pubdev/loader.py
74–76

right

Add assertion checking URL prefix

Build is green

Patch application report for D8355 (id=30273)

Rebasing onto 68e68e3f92...

Current branch diff-target is up to date.
Changes applied before test
commit d85615ac58e4a6fc1d8c17b9b7ac82d94521362c
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Sep 2 16:42:25 2022 +0200

    pubdev: Modify loader after origin URL modification in related lister
    
    Origin URL for a package listed by the pubdev lister is now in the form
    https://pub.dev/packages/<package_name> so we need to reconstruct the
    API URL to get package versions from it.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/856/ for more details.

This revision is now accepted and ready to land.Sep 5 2022, 3:53 PM

Build is green

Patch application report for D8355 (id=30299)

Rebasing onto 3ae2df31fe...

Current branch diff-target is up to date.
Changes applied before test
commit 6cdf6d30045cd64c6a8e3674037a8a20ffb96ab6
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Sep 2 16:42:25 2022 +0200

    pubdev: Modify loader after origin URL modification in related lister
    
    Origin URL for a package listed by the pubdev lister is now in the form
    https://pub.dev/packages/<package_name> so we need to reconstruct the
    API URL to get package versions from it.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/859/ for more details.