Sorry the description was not completely clear
The durations metrics are still updated, but not the operations count ones
In 2 metrics at 10 minutes of interval:
vsellier@saam ~ % ls -al /tmp/m[12] -rw-r--r-- 1 vsellier vsellier 176850 Mar 30 07:25 /tmp/m1 -rw-r--r-- 1 vsellier vsellier 176846 Mar 30 07:34 /tmp/m2
swh.storage 1.2.0 increased a bunch of timeouts and made some queries smarter so it's entirely plausible that the number of errors has dropped drastically.
What exact metric are you saying isn't updating? In your diff, the swh_storage_request_duration_seconds_count{endpoint="index"} metric seems to be increasing normally
SGTM, thanks!
I have the feeling that, in terms of API extensibility, we'll want to be returning both the OriginVisit and its latest OriginVisitStatus.
If we don't do that, we might find ourselves in a situation where we want to combine the fields from both objects into one, which feels a bit clunky when we could just return a proper composite type.
I have the feeling that, in terms of API extensibility, we'll want to be returning both the OriginVisit and its latest OriginVisitStatus.
swh-model 5.0.0 released, which finalizes these changes
Closing this, because we already fixed everything we can for now.
In T3841#80779, @olasd wrote:I think it's fine to remove the entries when we don't need them anymore (i.e. the object has been restored). Worst case, it'll be re-added at the next iteration of the script :-)
In T3841#80779, @olasd wrote:You'll need a column for which datastore has the corrupted object.
You'll need a column for which datastore has the corrupted object.
I wrote a script to scrub postgres and kafka: https://forge.softwareheritage.org/source/snippets/browse/master/vlorentz/recheck_consistency.py
migration done.
belvedere migration status: first index on directory created, ongoing index creation for revision, and then release.
staging, prod: storage deployed and service restarted (it's not dependent on the sql migration to be complete).
We don't care anymore, this will be handled by T3753.
The slide of the restrospective of the experiment are available at : https://hedgedoc.softwareheritage.org/VOP9qh1MTqm4DjPQfFgNbQ
It was not easy to know if it's a lot of call or long running calls because it's regular sample and we don't have this granularity.