Once the swh-storage metric has been integrated in prometheus, the stats export should use it.
Description
Status | Assigned | Task | ||
---|---|---|---|---|
Migrated | gitlab-migration | T1356 Kill munin | ||
Migrated | gitlab-migration | T1355 Move the object counter from munin to prometheus | ||
Migrated | gitlab-migration | T1437 Rewrite the munin stats export for the website to use prometheus | ||
Migrated | gitlab-migration | T1436 Integrate swh-storage metrics in prometheus |
Event Timeline
Prometheus data has been exported to a json file similar to the format produced by the Muni/RRD based toolchain.
Results are visible on https://www-dev.softwareheritage.org/archive/
(vs https://www.softwareheritage.org/archive/ for original graphs)
The new toolchain works but we can see a few problems:
- The graph stops at the first hole in the Prometheus dataset (21 march 2019)
- We are missing historical values from October 2015 to December 2018 (not present in Prometheus)
- Even for the first serie of Prometheus data, the form of the graph is too flat
For reference, a "Source files" graph produced by Prometheus itself can be visualized here:
http://pergamon.internal.softwareheritage.org:9090/graph?g0.range_input=1y&g0.end_input=2019-07-16%2015%3A13&g0.stacked=0&g0.expr=sql_swh_archive_object_count%7Bobject_type%3D%22content%22%7D&g0.tab=0
For the "March 2019 problem", the json output generated from the Prometheus API itself misses the more recent data points.
Depending on Prometheus for all data is not a hard requirement.
If it turns out to be impossible to inject historical data in Prometheus, the json creation script itself is authorized to provide the missing timecounter values.