Grafana dashboards show that most indexer consumers are working, but they are lagging behind, and the gap is increasing.
* https://grafana.softwareheritage.org/goto/RJkpEXVVz?orgId=1 origin-intrinsic-metadata (on `origin_visit_status` topic)
* https://grafana.softwareheritage.org/goto/c_iYPX4Vz?orgId=1 content-fossology-license (on `content` topic)
* https://grafana.softwareheritage.org/goto/g1VPPX4Vz?orgId=1 content-mimetype (on `content`) topic
* however, the extrinsic-metadata indexer is fine: https://grafana.softwareheritage.org/goto/3YQsPX44z?orgId=1
The plot of lag derivative shows a progressive slowdown, so it's probably not due to a specific configuration change.
I do not know what is causing this, though. Two possible suspects:
1. rdkafka frequently disconnecting from the brokers (or generally, having connection isues): https://sentry.softwareheritage.org/share/issue/76ed328b2ae6465face2ea4bb5f32187/
2. slow storage and/or objstorage (which would make sense, as the extrinsic-metadata indexer is super-fast, and is also the only one not to use the storage and objstorage)