HomeSoftware Heritage

celery_backend.listener: make the event listener reliable

Description

celery_backend.listener: make the event listener reliable

We reimplement the event listener using kombu directly instead of
Celery, which allows us to create persistent queues and manually ack the
messages once we have received them.

We bundle messages by packets of 1000 or 1/second when sending them to
the database. We only ack the messages once the database transcation has
ended.

Details

Provenance
olasdAuthored on Feb 22 2016, 3:07 PM
olasdPushed on Feb 22 2016, 3:09 PM
Parents
rDSCHc4e1b63c5dae: backend: implement the timestamp argument for the task_run functions
Branches
Unknown
Tags
Unknown
References
tag: v0.0.3