Page MenuHomeSoftware Heritage

Implement listener on top of pika instead of celery
ClosedPublic

Authored by olasd on Mon, Mar 23, 11:04 AM.

Details

Summary

The old listener based on celery has had some reliability issues with
the increased throughput of messages lately. A trivial re-implementation on top
of pika seems to be working better in production.

Test Plan

this has been running in prod for 2 weeks

Diff Detail

Repository
rDSCH Scheduling utilities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

olasd created this revision.Mon, Mar 23, 11:04 AM
ardumont accepted this revision.Mon, Mar 23, 11:42 AM
ardumont added a subscriber: ardumont.

I guess we'll add tests on this later :)

Thanks!

swh/scheduler/celery_backend/pika_listener.py
2

-2020?

This revision is now accepted and ready to land.Mon, Mar 23, 11:42 AM
olasd updated this revision to Diff 10205.Mon, Mar 23, 1:01 PM

Update copyright years

olasd marked an inline comment as done.Mon, Mar 23, 1:02 PM
This revision was automatically updated to reflect the committed changes.