HomeSoftware Heritage

Use better kafka producer semantics in the journal writers

Description

Use better kafka producer semantics in the journal writers

What we really want is for the broker to acknowledge all messages before we go
on to the next step. That's accomplished by flushing the producer rather than
enabling idempotence (which has other side-effects, such as only-once delivery,
which we don't really care about as all our consumers are, in effect,
idempotent).

Setting acks to all means that the broker acknowledges that all in-sync replicas
have persisted the message, which is a stronger guarantee than what we had
before.

Details

Provenance
olasdAuthored on Mar 2 2020, 5:32 PM
olasdPushed on Mar 11 2020, 1:15 PM
Differential Revision
D2809: Use better kafka producer semantics in the journal writers
Parents
rDJNLb1b125fb5ece: Make the number of messages processed at a time by journal clients configurable
Branches
Unknown
Tags
Unknown
Build Status
Buildable 11066
Build 16672: test-and-buildJenkins