Page MenuHomeSoftware Heritage

Improve kafka deployment
Closed, ResolvedPublic

Description

with

  • enough storage
  • HA
  • replication

Event Timeline

zack triaged this task as Normal priority.Mar 25 2019, 3:07 PM
olasd claimed this task.May 14 2019, 7:49 PM
olasd closed this task as Resolved.
olasd added a subscriber: olasd.

So that took a few tries in puppet, but adding new brokers to the kafka deployment should now be seamless.

The three elasticsearch nodes have been added as brokers to the current kafka cluster.

The existing topics are being rebalanced and replicated to span the four nodes of the cluster, each with 2 replicas.