HomeSoftware Heritage

celery: acknowledge tasks as soon as they're received

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

celery: acknowledge tasks as soon as they're received

With late acknowledgements, RabbitMQ will re-send tasks to clients even
if they can't ever complete the task (e.g. when the task gets killed
because the machine is out of memory).

This problem only increases over time, leading to complete starvation of
the ingestion system.

Now that we have multiple mechanisms to issue retries of tasks, we can
use early acknowledgements for tasks instead, which should mitigate the
ongoing starvation, at the expense of having to retry tasks externally.

Commit No Longer Exists

This commit no longer exists in the repository.