Once the swh-storage metric has been integrated in prometheus, the stats export should use it.
Description
Description
| Status | Assigned | Task | ||
|---|---|---|---|---|
| Migrated | gitlab-migration | T1356 Kill munin | ||
| Migrated | gitlab-migration | T1355 Move the object counter from munin to prometheus | ||
| Migrated | gitlab-migration | T1437 Rewrite the munin stats export for the website to use prometheus | ||
| Migrated | gitlab-migration | T1436 Integrate swh-storage metrics in prometheus |
Event Timeline
Comment Actions
Prometheus data has been exported to a json file similar to the format produced by the Muni/RRD based toolchain.
Results are visible on https://www-dev.softwareheritage.org/archive/
(vs https://www.softwareheritage.org/archive/ for original graphs)
The new toolchain works but we can see a few problems:
- The graph stops at the first hole in the Prometheus dataset (21 march 2019)
- We are missing historical values from October 2015 to December 2018 (not present in Prometheus)
- Even for the first serie of Prometheus data, the form of the graph is too flat
For reference, a "Source files" graph produced by Prometheus itself can be visualized here:
http://pergamon.internal.softwareheritage.org:9090/graph?g0.range_input=1y&g0.end_input=2019-07-16%2015%3A13&g0.stacked=0&g0.expr=sql_swh_archive_object_count%7Bobject_type%3D%22content%22%7D&g0.tab=0
Comment Actions
For the "March 2019 problem", the json output generated from the Prometheus API itself misses the more recent data points.
Comment Actions
Depending on Prometheus for all data is not a hard requirement.
If it turns out to be impossible to inject historical data in Prometheus, the json creation script itself is authorized to provide the missing timecounter values.