HomeSoftware Heritage

Delay the unsubscribe to the end of handle_messages

Description

Delay the unsubscribe to the end of handle_messages

to prevent some possible race condition leading to kafka errors like:

rd_kafka_assignment_partition_stopped: Assertion `rktp->rktp_started' failed

This could occur when, at the time of unsubscribing from a partition,
another partition is also depleted. Since the unsubscription consist in
resubscribing to all partitions except unsubscribes ones, we could try
to subscribe to such a depleted parition, leading to the error message
listed above.

Details

Provenance
douarddaAuthored on Mar 25 2022, 3:52 PM
douarddaPushed on Mar 30 2022, 11:34 AM
Differential Revision
D7463: Delay the unsubscribe to the end of handle_messages
Parents
rDDATASETe01daba4d601: Move exporter config entries in dedicated sections
Branches
Unknown
Tags
Unknown
Build Status
Buildable 28003
Build 43857: test-and-buildJenkins console · Jenkins