Page MenuHomeSoftware Heritage

Write a storage backend that writes to kafka
Closed, MigratedEdits Locked


a simple a possible, write-only for now

Event Timeline

zack triaged this task as Normal priority.Mar 25 2019, 3:07 PM

The intent is to start feeding kafka with some "production-grade" data, in a more reliable fashion that the current PostgreSQL notify mechanism. This will also help us get a better sense of the shape of the data, to make the analysis in T1602 more relevant.

As a first step, we would like to investigate somehow multiplexing writes between PostgreSQL and Kafka.