Depends on D4705
Details
Details
- Reviewers
- None
- Group Reviewers
Reviewers - Commits
- rDLS9944295729a2: Implement phabricator lister using the new pattern class
Yeah, we need one of these...
Diff Detail
Diff Detail
- Repository
- rDLS Listers
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 17847 Build 27580: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 27579: arc lint + arc unit
Event Timeline
Comment Actions
Build is green
Patch application report for D4706 (id=16673)
Could not rebase; Attempt merge onto d2f4781669...
Updating d2f4781..11efbff Fast-forward swh/lister/__init__.py | 9 +- swh/lister/github/__init__.py | 3 +- swh/lister/github/lister.py | 324 +- swh/lister/github/models.py | 17 - swh/lister/github/tasks.py | 34 +- .../data/https_api.github.com/empty_response.json | 1 - .../data/https_api.github.com/first_response.json | 1 - .../data/https_api.github.com/next_response.json | 6702 ------------------- .../data/https_api.github.com/repositories,since=0 | 6706 -------------------- swh/lister/github/tests/test_lister.py | 466 +- swh/lister/github/tests/test_tasks.py | 62 +- swh/lister/pattern.py | 270 + swh/lister/phabricator/__init__.py | 3 +- swh/lister/phabricator/lister.py | 263 +- swh/lister/phabricator/models.py | 17 - swh/lister/phabricator/tasks.py | 4 +- swh/lister/phabricator/tests/data/__init__.py | 0 swh/lister/phabricator/tests/test_lister.py | 138 +- swh/lister/tests/test_cli.py | 30 +- swh/lister/tests/test_pattern.py | 163 + 20 files changed, 1318 insertions(+), 13895 deletions(-) delete mode 100644 swh/lister/github/models.py delete mode 100644 swh/lister/github/tests/data/https_api.github.com/empty_response.json delete mode 120000 swh/lister/github/tests/data/https_api.github.com/first_response.json delete mode 100644 swh/lister/github/tests/data/https_api.github.com/next_response.json delete mode 100644 swh/lister/github/tests/data/https_api.github.com/repositories,since=0 create mode 100644 swh/lister/pattern.py delete mode 100644 swh/lister/phabricator/models.py create mode 100644 swh/lister/phabricator/tests/data/__init__.py create mode 100644 swh/lister/tests/test_pattern.py
Changes applied before test
commit 11efbff87fba35b205f7e103d9311b0dcffef448 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Wed Jul 22 12:05:56 2020 +0200 Implement phabricator lister using the new pattern class commit e420330aa5067053081f0dd5068de86e697154da Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Mon Dec 7 18:12:15 2020 +0100 Implement a base pattern for listers with no state storage commit 2c7edd50e8f72b6f6f28e200aeb18a9fdfa42eed Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Wed Dec 9 18:15:28 2020 +0100 Reimplement the GitHub lister using the new pattern class This replaces the test data with some manually generated answers, which allows us to test a few more cases for instantiating the lister. This also expands test coverage to test behavior on rate-limited requests. commit 4dab6476a0916337a38d3311b2e3b908d19f438f Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Wed Nov 25 20:40:40 2020 +0100 Add a helper to instantiate a new-style lister from a config file This helper will be used in the task entry points. commit 45761dcb5f782ee3045184f4dbc208ce08a4fe25 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Thu Jul 16 11:59:08 2020 +0200 Hook up listers implemented with the new pattern to the CLI We stop depending on the ListerBase implementation. The main hoop we're jumping through is the config override mechanism in swh.lister.get_lister, as it's really specifc to the ListerBase `override_config` argument, which is dropped in pattern.Lister (in favor of explicit arguments at lister instantiation). We implement a small shim in swh.lister.pattern.Lister to give backwards-compatibility for the new pattern to get_lister. This generic configuration override mechanism will probably be completely removed when the configuration mechanism is reworked. We'll see. commit 011928d262546c3e2af139efea854f6ea80a09e0 Author: Nicolas Dandrimont <nicolas@dandrimont.eu> Date: Mon Jul 6 10:27:57 2020 +0200 Introduce a simpler base pattern for lister implementations. This new pattern uses the lister support features introduced in swh.scheduler to replace the database management done in previous iterations of the listers.
See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/75/ for more details.