According the the initial proposal [1], prometheus will remain the historical reference of the counter values.
swh-counters needs to expose an api endpoint to allow prometheus to retrieve and store them in a timeseries.
[1] https://hedgedoc.softwareheritage.org/UHZYNPrAS_q6LhgwT5UhoQ#target-proposal
| Status | Assigned | Task | ||
|---|---|---|---|---|
| Migrated | gitlab-migration | T2912 Next generation archive counters | ||
| Migrated | gitlab-migration | T3164 Expose counters in prometheus format |
The current serie were the counters are stored is named sql_swh_archive_object_count, the serie for swh-counters could be swh_archive_object_count
The current tag are:
All these tag can be computed by swh-counters except person, not exposed in a kafka topic.
The counters are now exposed throught a /metrics enpoint and ingested by prometheus.
They are well tagged per environment so we will be able to isolate the counters for each one:
| Element | Value |
|---|---|
| swh_archive_object_total{col="value",environment="staging",instance="counters0.internal.staging.swh.network",job="swh-counters",object_type="content"} | 150059502 |
| swh_archive_object_total{col="value",environment="staging",instance="counters0.internal.staging.swh.network",job="swh-counters",object_type="origin"} | 581640 |
| swh_archive_object_total{col="value",environment="staging",instance="counters0.internal.staging.swh.network",job="swh-counters",object_type="revision"} | 31682911 |
The diff D5332 will allow to monitor the others topics