HomeSoftware Heritage

Use better kafka producer semantics in the journal writers

Authored by olasd on Mar 2 2020, 5:32 PM.

Description

Use better kafka producer semantics in the journal writers

What we really want is for the broker to acknowledge all messages before we go
on to the next step. That's accomplished by flushing the producer rather than
enabling idempotence (which has other side-effects, such as only-once delivery,
which we don't really care about as all our consumers are, in effect,
idempotent).

Setting acks to all means that the broker acknowledges that all in-sync replicas
have persisted the message, which is a stronger guarantee than what we had
before.