HomeSoftware Heritage

Handle BufferErrors in writer.kafka.Producer

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

Handle BufferErrors in writer.kafka.Producer

When the local rdkafka producer queue is full, the producer would crash
with a BufferError. While this condition is rare, it can happen when
running a backfill operation.

When such a condition arises, retry sending the message a few times with
an increasing backoff. When giving up, use the common KafkaDeliveryError
mechanism rather than the plain BufferError.

Details

Provenance
olasdAuthored on Feb 25 2021, 12:12 PM
olasdPushed on Feb 25 2021, 6:48 PM
Build Status
Buildable 19502
Build 30253: test-and-buildJenkins console · Jenkins

Commit No Longer Exists

This commit no longer exists in the repository.