Page MenuHomeSoftware Heritage

docker: Explicit the scheduler runner for save-code-now
ClosedPublic

Authored by ardumont on Jun 10 2021, 3:22 PM.

Details

Summary

Related to T3367

Depends on D5826 (and the override in the docker-compose.override.yml for
swh-scheduler-runner and swh-scheduler-runner-priority)

Test Plan

doco up -d Then play with the save code now and ensure the priority tasks are indeed
prioritize with the right new scheduler (it is):

swh-scheduler-runner-priority_1  | wait-for-it: amqp:5672 is available after 0 seconds
swh-scheduler-runner-priority_1  | INFO:swh.scheduler.celery_backend.runner:Grabbed 1 tasks load-git (priority)
swh-scheduler-runner-priority_1  | INFO:swh.scheduler.cli.admin.runner:Scheduled 1 tasks

The other ones don't bulge unless you start listing stuff:

swh-scheduler-runner_1           |  postgresql is up!
swh-scheduler-runner_1           | (1 row)
swh-scheduler-runner_1           |
swh-scheduler-runner_1           | wait-for-it: waiting for swh-scheduler:5008 without a timeout
swh-scheduler-runner_1           | wait-for-it: swh-scheduler:5008 is available after 0 seconds
swh-scheduler-runner_1           | Starting the swh-scheduler start-runner
swh-scheduler-runner_1           | wait-for-it: waiting for amqp:5672 without a timeout
swh-scheduler-runner_1           | wait-for-it: amqp:5672 is available after 0 seconds
swh-scheduler-runner_1           | INFO:swh.scheduler.celery_backend.runner:Grabbed 1 tasks index-origin-metadata
swh-scheduler-runner_1           | INFO:swh.scheduler.cli.admin.runner:Scheduled 1 tasks
...

Diff Detail

Repository
rDENV Development environment
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.
docker/docker-compose.yml
158

The difference is here, the runner is in charge of scheduling solely the load-{git,svn,hg,archive-files} with priority.
The other (initial) runner won't do it anymore, it will only scheduled the rests of the tasks without any priority.

This revision is now accepted and ready to land.Jun 10 2021, 4:00 PM