Add some metrics to swh-loader-core
- fetch duration (histogram)
- number of objects of each kind added to the archive (counters)
Add some metrics to swh-loader-core
Status | Assigned | Task | ||
---|---|---|---|---|
Open | None | T1408 More/better Metrics | ||
Resolved | olasd | T1535 Deploy prometheus-statsd-exporter to gather per-worker metrics | ||
Open | None | T1461 Add loader-related metrics to swh-loader-core |
I think the second point mostly happened: the storage is returning statistics to the loader, but the loaders don't generally collect them.
I'm not sure what kind of granularity we really want for the loader-level object counters (do we want per-loader breakdowns of objects added? per-visit?)
Once that's specified this is probably a candidate for an easy hack (as most of the data is there, it just needs to be collected / sent to statsd).