HomeSoftware Heritage

Celery runner: only schedule tasks when the buffer is less than 80% full

Authored by olasd on May 19 2020, 11:30 AM.

Description

Celery runner: only schedule tasks when the buffer is less than 80% full

The queries to pick up tasks from the scheduler sometimes degenerate when the
number of tasks fetched is too low, which hangs the runner for all other tasks.
Adding this lower bound helps postgresql use proper optimizations to pull tasks.