HomeSoftware Heritage

Celery runner: only schedule tasks when the buffer is less than 80% full

Description

Celery runner: only schedule tasks when the buffer is less than 80% full

The queries to pick up tasks from the scheduler sometimes degenerate when the
number of tasks fetched is too low, which hangs the runner for all other tasks.
Adding this lower bound helps postgresql use proper optimizations to pull tasks.

Details

Provenance
olasdAuthored on May 19 2020, 11:30 AM
olasdPushed on May 19 2020, 11:48 AM
Differential Revision
D3164: Celery runner: only schedule tasks when the buffer is less than 80% full
Parents
rDSCHb83990613370: Disable the azure http logger in the celery worker base config
Branches
Unknown
Tags
Unknown
References
tag: v0.1.0
Build Status
Buildable 12429
Build 18862: test-and-buildJenkins console · Jenkins