To prepare for further changes in this area.
Details
Details
- Reviewers
olasd ardumont - Group Reviewers
Reviewers - Commits
- rDSCH59b4cb3f1c7a: Reorganize ListedOrigin fixtures to generate multiple visit_types
existing tests pass unchanged
Diff Detail
Diff Detail
- Repository
- rDSCH Scheduling utilities
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 18329 Build 28310: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 28309: arc lint + arc unit
Event Timeline
Comment Actions
Build is green
Patch application report for D4854 (id=17185)
Could not rebase; Attempt merge onto a62003397d...
Updating a620033..23d1b3c Fast-forward swh/scheduler/cli/__init__.py | 7 +- swh/scheduler/cli/origin.py | 135 +++++++++++++++++++++++++++++++++ swh/scheduler/model.py | 9 +++ swh/scheduler/tests/common.py | 10 +-- swh/scheduler/tests/conftest.py | 45 ++++++++--- swh/scheduler/tests/test_cli_origin.py | 103 +++++++++++++++++++++++++ swh/scheduler/tests/test_model.py | 31 +++++++- 7 files changed, 320 insertions(+), 20 deletions(-) create mode 100644 swh/scheduler/cli/origin.py create mode 100644 swh/scheduler/tests/test_cli_origin.py
Changes applied before test
commit 23d1b3c1883c3c955b5dd5ba1cc2270c93e156d6
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Wed Jan 13 15:25:56 2021 +0100
Reorganize ListedOrigin fixtures to generate multiple visit_types
commit da347f7f4c401a43ec34de76365ad323d0ff7b77
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Tue Jan 12 17:10:39 2021 +0100
Introduce a `swh scheduler origin schedule-next` cli
This creates one-shot tasks in the classic scheduler for the next visits
to run according to the visit scheduling policy.
commit 42957c9e96e6c7d8070e0b6c786c273e8c1602a0
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Tue Jan 12 17:28:33 2021 +0100
Rename test task types to names that match real tasks
The success of tests using these task types would depend on the test run
order, because these task types are (currently) being created by
swh/scheduler/sql/50-data.sql, but the table is truncated after the
first test completes.
commit d1393c54da99c45175dd0b6a69734d17fc887960
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Tue Jan 12 16:16:31 2021 +0100
Introduce a `swh scheduler origin grab-next` cli
This returns, as CSV, the next origins to be visited according to the
passed scheduling policy.See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/106/ for more details.
| swh/scheduler/tests/conftest.py | ||
|---|---|---|
| 62 | why not plain listed_origins_by_type.values() ? | |
Comment Actions
one question inline but otherwise, lgtm.
(also, i'm assuming the need for the visit_types as pytest fixture is needed for later).
| swh/scheduler/tests/conftest.py | ||
|---|---|---|
| 62 | It converts List[List[ListedOrigin]] to List[ListedOrigin]. It's like a fold, with [] as the initial value of the accumulator | |
Comment Actions
Build is green
Patch application report for D4854 (id=17215)
Could not rebase; Attempt merge onto a62003397d...
Updating a620033..59b4cb3 Fast-forward swh/scheduler/cli/__init__.py | 7 +- swh/scheduler/cli/origin.py | 136 +++++++++++++++++++++++++++++++++ swh/scheduler/model.py | 9 +++ swh/scheduler/tests/common.py | 10 +-- swh/scheduler/tests/conftest.py | 45 ++++++++--- swh/scheduler/tests/test_cli_origin.py | 103 +++++++++++++++++++++++++ swh/scheduler/tests/test_model.py | 31 +++++++- 7 files changed, 321 insertions(+), 20 deletions(-) create mode 100644 swh/scheduler/cli/origin.py create mode 100644 swh/scheduler/tests/test_cli_origin.py
Changes applied before test
commit 59b4cb3f1c7a081e0d28b11d15888d38a9de151e
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Wed Jan 13 15:25:56 2021 +0100
Reorganize ListedOrigin fixtures to generate multiple visit_types
commit 4f5338f2aba360fed2e524cbcdd23b11bacfb79d
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Tue Jan 12 17:10:39 2021 +0100
Introduce a `swh scheduler origin schedule-next` cli
This creates one-shot tasks in the classic scheduler for the next visits
to run according to the visit scheduling policy.
commit 3dd1d5f28d329620a65ee00749d24401b6d8cf00
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Tue Jan 12 17:28:33 2021 +0100
Rename test task types to names that match real tasks
The success of tests using these task types would depend on the test run
order, because these task types are (currently) being created by
swh/scheduler/sql/50-data.sql, but the table is truncated after the
first test completes.
commit 5d7b002ac403565e348ac8fe4dd56d015cf29cae
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Tue Jan 12 16:16:31 2021 +0100
Introduce a `swh scheduler origin grab-next` cli
This returns, as CSV, the next origins to be visited according to the
passed scheduling policy.See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/119/ for more details.