Page MenuHomeSoftware Heritage

PubDev, loads from pub.dev registry (Dart, Flutter)
ClosedPublic

Authored by franckbret on Aug 24 2022, 1:31 PM.

Details

Summary

Call an http api endpoint per package to get all its versions and
artifacts metadata.

Diff Detail

Repository
rDLDBASE Generic VCS/Package Loader
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D8305 (id=29980)

Rebasing onto 9f8bdceed1...

Current branch diff-target is up to date.
Changes applied before test
commit f4df224834e999053809af450a80bafa48f1f5bf
Author: Franck Bret <franck.bret@octobus.net>
Date:   Wed Aug 24 13:24:46 2022 +0200

    [WIP] PubDev, loads from pub.dev registry (Dart, FLutter)
    
    Call an http api endpoint per package to get all its versions and
    artifacts metadata.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/833/ for more details.

vlorentz added inline comments.
swh/loader/package/pubdev/loader.py
74

Is this decorator useful here?

133–136

https://dart.dev/tools/pub/pubspec#authorauthors describes an "authors". Does this API have it too? If yes, read it too and if there is more than one, use the first and ignore others (they will be part of metadata anyway)

162–165

https://dart.dev/tools/pub/pubspec#authorauthors says the intrinsic author field is deprecated (and definitely should not take precedence over the one returned by the API)

Some improvments after review

Build is green

Patch application report for D8305 (id=29989)

Rebasing onto 9f8bdceed1...

Current branch diff-target is up to date.
Changes applied before test
commit da6552b4fad581e96d287916f730163118eb8d93
Author: Franck Bret <franck.bret@octobus.net>
Date:   Wed Aug 24 13:24:46 2022 +0200

    PubDev, loads from pub.dev registry (Dart, FLutter)
    
    Call an http api endpoint per package to get all its versions and
    artifacts metadata.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/836/ for more details.

franckbret added inline comments.
swh/loader/package/pubdev/loader.py
74

Not sure, I take inspiration from pypi lister where it decorate same method.
Removing it.

133–136

Will try to find at least one package where there are "authors" and add a fixture + tests to ensure the implementation is correct

franckbret retitled this revision from [WIP] PubDev, loads from pub.dev registry (Dart, Flutter) to PubDev, loads from pub.dev registry (Dart, Flutter).Aug 24 2022, 5:09 PM
franckbret marked 2 inline comments as done.

Testing authors and empty author

Add tests and data fixtures to ensure we manage the case where a list of authors exists and the one where author is empty

Build is green

Patch application report for D8305 (id=30022)

Rebasing onto 9f8bdceed1...

Current branch diff-target is up to date.
Changes applied before test
commit 5f9895cc6749f0d8bfc83d35b998119ef275c5e4
Author: Franck Bret <franck.bret@octobus.net>
Date:   Wed Aug 24 13:24:46 2022 +0200

    PubDev, loads from pub.dev registry (Dart, FLutter)
    
    Call an http api endpoint per package to get all its versions and
    artifacts metadata.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/839/ for more details.

Add pubdev entry to package loader specifications documentation

Build is green

Patch application report for D8305 (id=30023)

Rebasing onto 9f8bdceed1...

Current branch diff-target is up to date.
Changes applied before test
commit 1a948a534f82d2a46eef535786171b5a8fc0b018
Author: Franck Bret <franck.bret@octobus.net>
Date:   Wed Aug 24 13:24:46 2022 +0200

    PubDev, loads from pub.dev registry (Dart, FLutter)
    
    Call an http api endpoint per package to get all its versions and
    artifacts metadata.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/840/ for more details.

docs/package-loader-specifications.rst
126

The description is not part of the branch name.

swh/loader/package/pubdev/loader.py
151

There cannot be more than one file per version, so there is no need to include the file name in the branch name.

Remove filename from release name

Using filename for building the release name was useless because there cannot be more than one file per version

Build is green

Patch application report for D8305 (id=30035)

Rebasing onto 9f8bdceed1...

Current branch diff-target is up to date.
Changes applied before test
commit 700bdc2f3cb113f5281e97400074aa7d690c8bb3
Author: Franck Bret <franck.bret@octobus.net>
Date:   Wed Aug 24 13:24:46 2022 +0200

    PubDev, loads from pub.dev registry (Dart, FLutter)
    
    Call an http api endpoint per package to get all its versions and
    artifacts metadata.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/841/ for more details.

Thanks, looks good.

One last thing: could you find smaller alternative for the test files? These are quite big, especially packages_pdf_versions_3.8.2.tar.gz

Thanks, looks good.

One last thing: could you find smaller alternative for the test files? These are quite big, especially packages_pdf_versions_3.8.2.tar.gz

Yes sure, will make a script to generate them

Add a fake_pubdev.sh script that generate fake http api response and fake package archives for test purpose

Build is green

Patch application report for D8305 (id=30049)

Rebasing onto 9f8bdceed1...

Current branch diff-target is up to date.
Changes applied before test
commit aa26fbd98ef8cfbec92a69ccde4ae1d799ff7d52
Author: Franck Bret <franck.bret@octobus.net>
Date:   Wed Aug 24 13:24:46 2022 +0200

    PubDev, loads from pub.dev registry (Dart, FLutter)
    
    Call an http api endpoint per package to get all its versions and
    artifacts metadata.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/842/ for more details.

This revision is now accepted and ready to land.Aug 26 2022, 10:10 AM