Page MenuHomeSoftware Heritage

Make the retry proxy work on all functions.
ClosedPublic

Authored by vlorentz on Nov 6 2020, 12:53 PM.

Details

Summary

The metadata migration script kept crashing otherwise.

This also remove tests that are now redundant, as they all test the exact same code in the same scenario

Diff Detail

Event Timeline

Build is green

Patch application report for D4434 (id=15701)

Could not rebase; Attempt merge onto 84984a600c...

Updating 84984a60..5794e040
Fast-forward
 requirements-swh-journal.txt    |   2 +-
 swh/storage/backfill.py         |  10 +-
 swh/storage/retry.py            |  80 ++---
 swh/storage/tests/test_retry.py | 640 ----------------------------------------
 swh/storage/writer.py           |  21 +-
 5 files changed, 42 insertions(+), 711 deletions(-)
Changes applied before test
commit 5794e0403f2ad69c9ccd98733bceee908d1bf21e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 6 12:52:50 2020 +0100

    Make the retry proxy work on all functions.
    
    The metadata migration script kept crashing otherwise.

commit 9d1237b4eae1d64ca784e3ebb7f0f390e551b3a6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Nov 5 15:28:49 2020 +0100

    Set the value_sanitizer argument of get_journal_writer.
    
    The next version of swh-journal will remove the default value.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1052/ for more details.

ardumont added a subscriber: ardumont.

Explaining why you dropped, i gather redundant tests, from the retry module, in the diff description would be nice.

And looks good to me.

swh/storage/retry.py
83

Does that filter on callables only?

This revision is now accepted and ready to land.Nov 6 2020, 1:31 PM
swh/storage/retry.py
83

I don't understand the question

swh/storage/retry.py
83

well, i'm not sure what the conditional does... so i'm asking.

So my guess is that it's filtering on callables (well methods) to install and decorates such method with the retry behavior.

swh/storage/retry.py
83

ah, yes. Now that we have protocols, hasattr(attribute, "__call__") is equivalent to isinstance(attribute, typing.Callable)

Build is green

Patch application report for D4434 (id=15753)

Could not rebase; Attempt merge onto 24cdc85c15...

Updating 24cdc85c..aded45b9
Fast-forward
 requirements-swh-journal.txt    |   2 +-
 swh/storage/backfill.py         |  10 +-
 swh/storage/retry.py            |  80 ++---
 swh/storage/tests/test_retry.py | 640 ----------------------------------------
 swh/storage/writer.py           |  21 +-
 5 files changed, 42 insertions(+), 711 deletions(-)
Changes applied before test
commit aded45b9b27d93df8195fde5f462250e2e84798b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Nov 6 12:52:50 2020 +0100

    Make the retry proxy work on all functions.
    
    The metadata migration script kept crashing otherwise.

commit 2e7d489eb245d4c3b684360d6f70f061804f77a9
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Nov 5 15:28:49 2020 +0100

    Set the value_sanitizer argument of get_journal_writer.
    
    The next version of swh-journal will remove the default value.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1060/ for more details.