HomeSoftware Heritage

Rework the retry and reporting system in replay.py

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

Rework the retry and reporting system in replay.py

split the 'copy_object()' in 2 parts, 'get_object()' and 'put_object()'
and make each of these decorated by '@content_replay_retry' (instead of
the whole 'copy_object()'), so that a failing 'put_object' does not
trigger getting the object from the src objstorage again.

Also, only log in statsd the end result (error or success) instead of
logging each attempt. We don't need these intermediate results in
statsd, and it makes them much harder to use in a dashboard.

Last, for the sake of implementation ease, use function name as
"operation" tag in stastds reports (so "get_object", "put_object" and
"obj_in_storage") instead of "copy" and "in_dst".

Doing so, the 'operation' argument of the ReplayError exception has been
dropped.

Details

Provenance
douarddaAuthored on Nov 25 2021, 5:49 PM
douarddaPushed on Dec 2 2021, 1:11 PM
Differential Revision
D6692: Rework the retry and reporting system in replay.py
Build Status
Buildable 25318
Build 39574: test-and-buildJenkins console · Jenkins

Commit No Longer Exists

This commit no longer exists in the repository.