Page MenuHomeSoftware Heritage

Implement listener on top of pika instead of celery
ClosedPublic

Authored by olasd on Mar 23 2020, 11:04 AM.

Details

Summary

The old listener based on celery has had some reliability issues with
the increased throughput of messages lately. A trivial re-implementation on top
of pika seems to be working better in production.

Test Plan

this has been running in prod for 2 weeks

Diff Detail

Repository
rDSCH Scheduling utilities
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 11280
Build 17050: tox-on-jenkinsJenkins
Build 17049: arc lint + arc unit

Event Timeline

ardumont added a subscriber: ardumont.

I guess we'll add tests on this later :)

Thanks!

swh/scheduler/celery_backend/pika_listener.py
1

-2020?

This revision is now accepted and ready to land.Mar 23 2020, 11:42 AM