Page MenuHomeSoftware Heritage

Added LaunchpadLister
ClosedPublic

Authored by legau on Apr 7 2020, 5:34 PM.

Details

Summary

Related to T1734

From abandonned D2799

Diff Detail

Repository
rDLS Listers
Branch
arcpatch-D2974
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 11824
Build 17936: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 17935: arc lint + arc unit

Event Timeline

@ardumont It seems the build error did not go away

@ardumont It seems the build error did not go away

/me grunts
indeed.

meanwhile:

cd swh-lister
git checkout master
git pull
arc patch D2974
tox -r -- -x -s

Everything is fine.

----------- coverage: platform linux, python 3.7.3-final-0 -----------
Name                                                                                   Stmts   Miss Branch BrPart  Cover
------------------------------------------------------------------------------------------------------------------------
.tox/py3/lib/python3.7/site-packages/swh/lister/__init__.py                               20      2      6      1    88%
.tox/py3/lib/python3.7/site-packages/swh/lister/bitbucket/__init__.py                      4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/bitbucket/lister.py                       44      1     10      1    96%
.tox/py3/lib/python3.7/site-packages/swh/lister/bitbucket/models.py                        6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/bitbucket/tasks.py                        27      4      4      1    84%
.tox/py3/lib/python3.7/site-packages/swh/lister/bitbucket/tests/__init__.py                0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/bitbucket/tests/conftest.py                1      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/bitbucket/tests/test_lister.py            62      0      6      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/bitbucket/tests/test_tasks.py             51      0      8      1    98%
.tox/py3/lib/python3.7/site-packages/swh/lister/cgit/__init__.py                           4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cgit/lister.py                            67      2     22      5    92%
.tox/py3/lib/python3.7/site-packages/swh/lister/cgit/models.py                             6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cgit/tasks.py                              6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cgit/tests/__init__.py                     0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cgit/tests/conftest.py                     1      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cgit/tests/test_lister.py                 45      0      4      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cgit/tests/test_tasks.py                  17      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cli.py                                    58     35     16      1    32%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/__init__.py                           0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/abstractattribute.py                  5      0      2      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/indexing_lister.py                  105     15     40     10    83%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/lister_base.py                      180     29     54     11    81%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/lister_transports.py                103     28     18      4    72%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/models.py                            35      0      2      1    97%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/page_by_page_lister.py               49     18     14      4    56%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/simple_lister.py                     38      4     10      2    88%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/tests/__init__.py                     0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/tests/conftest.py                    20      0      2      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/tests/test_abstractattribute.py      41      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/tests/test_indexing_lister.py        62      1      6      0    99%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/tests/test_lister.py                247      4     38      3    97%
.tox/py3/lib/python3.7/site-packages/swh/lister/core/tests/test_model.py                  56      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cran/__init__.py                           4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cran/lister.py                            41      4      2      1    88%
.tox/py3/lib/python3.7/site-packages/swh/lister/cran/models.py                             6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cran/tasks.py                              6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cran/tests/__init__.py                     0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cran/tests/conftest.py                     6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cran/tests/test_lister.py                 40      0      4      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/cran/tests/test_tasks.py                  17      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/__init__.py                        27      0     10      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/lister.py                         111      7     32      7    90%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/models.py                         105      5     18      2    94%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/tasks.py                            6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/tests/__init__.py                   0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/tests/conftest.py                  28      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/tests/test_init.py                 39      0      8      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/tests/test_lister.py               16      0      2      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/tests/test_models.py               16      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/tests/test_tasks.py                16      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/debian/utils.py                           43     43     12      0     0%
.tox/py3/lib/python3.7/site-packages/swh/lister/github/__init__.py                         4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/github/lister.py                          38      0      8      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/github/models.py                           7      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/github/tasks.py                           27      4      4      1    84%
.tox/py3/lib/python3.7/site-packages/swh/lister/github/tests/__init__.py                   0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/github/tests/conftest.py                   1      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/github/tests/test_lister.py               43      0      4      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/github/tests/test_tasks.py                51      1      8      2    95%
.tox/py3/lib/python3.7/site-packages/swh/lister/gitlab/__init__.py                         4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gitlab/lister.py                          45     11     12      3    68%
.tox/py3/lib/python3.7/site-packages/swh/lister/gitlab/models.py                           6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gitlab/tasks.py                           27      2      2      0    93%
.tox/py3/lib/python3.7/site-packages/swh/lister/gitlab/tests/__init__.py                   0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gitlab/tests/conftest.py                   1      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gitlab/tests/test_lister.py               42     10      4      0    74%
.tox/py3/lib/python3.7/site-packages/swh/lister/gitlab/tests/test_tasks.py                79      2     16      4    94%
.tox/py3/lib/python3.7/site-packages/swh/lister/gnu/__init__.py                            4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gnu/lister.py                             25      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gnu/models.py                              6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gnu/tasks.py                               6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gnu/tests/__init__.py                      0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gnu/tests/conftest.py                      1      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gnu/tests/test_lister.py                  28      0      6      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gnu/tests/test_tasks.py                   17      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gnu/tests/test_tree.py                    63      0      8      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/gnu/tree.py                               93      0     40      1    99%
.tox/py3/lib/python3.7/site-packages/swh/lister/launchpad/__init__.py                      4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/launchpad/lister.py                       66     13     20      5    74%
.tox/py3/lib/python3.7/site-packages/swh/lister/launchpad/models.py                        6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/launchpad/tasks.py                        13      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/launchpad/tests/__init__.py                0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/launchpad/tests/conftest.py               32      0      8      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/launchpad/tests/test_lister.py            16      0      2      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/launchpad/tests/test_tasks.py             21      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/npm/__init__.py                            4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/npm/lister.py                             63     16     12      0    68%
.tox/py3/lib/python3.7/site-packages/swh/lister/npm/models.py                             18      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/npm/tasks.py                              33     15      4      0    49%
.tox/py3/lib/python3.7/site-packages/swh/lister/npm/tests/conftest.py                      6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/npm/tests/test_lister.py                  54      0      2      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/npm/tests/test_tasks.py                   35      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/packagist/__init__.py                      4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/packagist/lister.py                       30      0      2      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/packagist/models.py                        5      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/packagist/tasks.py                         6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/packagist/tests/__init__.py                0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/packagist/tests/conftest.py                6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/packagist/tests/test_lister.py            49      0      6      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/packagist/tests/test_tasks.py             17      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/phabricator/__init__.py                    4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/phabricator/lister.py                     86      5     36      8    89%
.tox/py3/lib/python3.7/site-packages/swh/lister/phabricator/models.py                      7      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/phabricator/tasks.py                       6      1      0      0    83%
.tox/py3/lib/python3.7/site-packages/swh/lister/phabricator/tests/__init__.py              0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/phabricator/tests/conftest.py              6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/phabricator/tests/test_lister.py          71      0     10      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/phabricator/tests/test_tasks.py            6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/pypi/__init__.py                           4      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/pypi/lister.py                            30      0      2      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/pypi/models.py                             5      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/pypi/tasks.py                              6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/pypi/tests/__init__.py                     0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/pypi/tests/conftest.py                     6      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/pypi/tests/test_lister.py                 14      0      2      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/pypi/tests/test_tasks.py                  17      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/tests/__init__.py                          0      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/tests/test_cli.py                         27      0      6      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/tests/test_utils.py                       18      0      0      0   100%
.tox/py3/lib/python3.7/site-packages/swh/lister/utils.py                                   8      0      6      1    93%
------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                   3314    282    580     80    89%

================================================================================================== 125 passed, 1 warning in 98.68s (0:01:38) ==================================================================================================
___________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
  flake8: commands succeeded
  mypy: commands succeeded
  py3: commands succeeded
  congratulations :)

Remains mostly nitpicks to fix (type, indentation, docstring updates...)

swh/lister/launchpad/lister.py
62

...checks: bool = False...

64

indentation is kinda off by 1 char.

(Hopefully we'll have black soon doing that for us),

69

you can remove the type from the docstring.

84

-> Dict[str, Any]:

94

That's returning a dict with the listing status.

swh/lister/launchpad/tests/conftest.py
35

add some space for visibility.

36

same here.

Also, add yourself to the CONTRIBUTORS file (at the top of the repository ;)
(I think it's alphabetic order on the firstname there.)

swh/lister/launchpad/models.py
1

-2020 really (also in other modules ;)

swh/lister/launchpad/tests/data/response0.json
108

Maybe also reduce a bit that file, no need for so many entries.

swh/lister/launchpad/lister.py
64

What do you mean by that ?

swh/lister/launchpad/lister.py
64
The
 filter

i would expect:

The
filter

or let this as is for now.

someone is blackifying our repositories, so i'd expect this to be dealt with on your next rebase.

Ok, so looks good to me.

This needs some last adjustments due to the new black behavior and we are good to go.

Please:

  • Squash your 2 commits into 1.
  • update and rebase on latest master (you should not have conflicts as it's all new files ;)
  • let black do its blackify stuff : pre-commit run --all-files (should trigger black on your diff)
  • and then amend your commit and update this diff

Thanks in advance.

Cheers,

This revision is now accepted and ready to land.Apr 8 2020, 5:25 PM

Build is green

Patch application report for D2974 (id=10705)

Rebasing onto 29325c1d94...

Current branch diff-target is up to date.
Changes applied before test
commit 1fff5ae2be436666cdb58fdf1942f5f66557febd
Author: Léni Gauffier <lenigauffier@gmail.com>
Date:   Sun Apr 12 00:22:20 2020 +0200

    Added LaunchpadLister

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/9/ for more details.

This revision was automatically updated to reflect the committed changes.

\m/

Thanks!

Note:

  • your commit in master worked (as expected)
  • For some reason this diff build finally passed! ¯\_(ツ)_/¯

Cheers,