HomeSoftware Heritage

Rework the retry and reporting system in replay.py

Description

Rework the retry and reporting system in replay.py

split the 'copy_object()' in 2 parts, 'get_object()' and 'put_object()'
and make each of these decorated by '@content_replay_retry' (instead of
the whole 'copy_object()'), so that a failing 'put_object' does not
trigger getting the object from the src objstorage again.

Also, only log in statsd the end result (error or success) instead of
logging each attempt. We don't need these intermediate results in
statsd, and it makes them much harder to use in a dashboard.

Last, for the sake of implementation ease, use function name as
"operation" tag in stastds reports (so "get_object", "put_object" and
"obj_in_storage") instead of "copy" and "in_dst".

Doing so, the 'operation' argument of the ReplayError exception has been
dropped.

Details

Provenance
douarddaAuthored on Nov 25 2021, 5:49 PM
douarddaPushed on Dec 2 2021, 1:11 PM
Differential Revision
D6692: Rework the retry and reporting system in replay.py
Parents
rDOBJSRPL1d8ea80c7d01: Add tests for expected statsd reports during a content replay session
Branches
Unknown
Tags
Unknown
Build Status
Buildable 25318
Build 39574: test-and-buildJenkins console · Jenkins