The subcommand bypasses the legacy task-based mechanism to directly send
new origin visits to celery
Details
Details
- Reviewers
ardumont - Group Reviewers
Reviewers - Commits
- rDSCH2efad289833e: Add a swh scheduler origin send-to-celery subcommand
simple command line test added to check the basic behavior
Depends on D6145
Diff Detail
Diff Detail
- Repository
- rDSCH Scheduling utilities
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 23204 Build 36204: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 36203: arc lint + arc unit
Event Timeline
Comment Actions
Build is green
Patch application report for D6146 (id=22245)
Could not rebase; Attempt merge onto cc76a573e7...
Updating cc76a57..2efad28 Fast-forward swh/scheduler/backend.py | 9 ++++++- swh/scheduler/cli/origin.py | 46 ++++++++++++++++++++++++++++++++++ swh/scheduler/interface.py | 3 +++ swh/scheduler/tests/test_cli_origin.py | 34 +++++++++++++++++++++++++ swh/scheduler/tests/test_scheduler.py | 16 ++++++++++++ 5 files changed, 107 insertions(+), 1 deletion(-)
Changes applied before test
commit 2efad289833e971594833b9ed825b9acead8d254
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Tue Jun 1 20:04:11 2021 +0200
Add a swh scheduler origin send-to-celery subcommand
The subcommand bypasses the legacy task-based mechanism to directly send
new origin visits to celery
commit 5e8007fdbfeb612ea394f97eeba25a1c4e529b7e
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date: Tue Jun 1 15:48:05 2021 +0200
Add table sampling option to grab_next_visits
Running common operations on all git origins is pretty intense. Using
table sampling gives us the opportunity to at least schedule some jobs
in (decently small) time.See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/447/ for more details.