Page MenuHomeSoftware Heritage

Implement listener on top of pika instead of celery
ClosedPublic

Authored by olasd on Mar 23 2020, 11:04 AM.

Details

Summary

The old listener based on celery has had some reliability issues with
the increased throughput of messages lately. A trivial re-implementation on top
of pika seems to be working better in production.

Test Plan

this has been running in prod for 2 weeks

Diff Detail

Repository
rDSCH Scheduling utilities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont added a subscriber: ardumont.

I guess we'll add tests on this later :)

Thanks!

swh/scheduler/celery_backend/pika_listener.py
1

-2020?

This revision is now accepted and ready to land.Mar 23 2020, 11:42 AM