Page MenuHomeSoftware Heritage

WIP backend: add retrying on IO error with remote services
AbandonedPublic

Authored by tenma on Nov 30 2020, 11:38 AM.

Details

Summary

Wrapper function on objects that let choose methods to retry on IO errors.

This function would not live here of course by rather in core if useful.

Not tested yet, had some troubles finding how to decorate methods of objects by decorating only the object, and decorate where it is used rather than where defined.

Inspiration taken from RetryingProxyStorage. First, need feedback on the approach, before discussing implementation.

Related to T2862

Test Plan

local tests then proper pytest

Diff Detail

Event Timeline

tenma edited the summary of this revision. (Show Details)
tenma edited the summary of this revision. (Show Details)
tenma edited the summary of this revision. (Show Details)

Build was aborted

Patch application report for D4630 (id=16427)

Rebasing onto 4e1fb5e04c...

First, rewinding head to replay your work on top of it...
Applying: backend: add retrying on IO error with remote services
Using index info to reconstruct a base tree...
M	swh/vault/backend.py
Falling back to patching base and 3-way merge...
Auto-merging swh/vault/backend.py
Changes applied before test
commit ebd1aaef2425859fc70202361072d1859b8f99a1
Author: tenma <tenma+swh@mailbox.org>
Date:   Mon Nov 30 11:30:12 2020 +0100

    backend: add retrying on IO error with remote services
    
    Decorator-like function on objects that let choose methods to retry on IO
    error.

Link to build: https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/58/
See console output for more information: https://jenkins.softwareheritage.org/job/DVAU/job/tests-on-diff/58/console

ardumont added a subscriber: ardumont.

@tenma
This is partially (for the storage part) superseded by D4677.
So might be, this need to be closed now or in planned changes.

Another strategy preferred: using retrying proxies for storage (existing and activated) and scheduler (to be done).