Page MenuHomeSoftware Heritage

Use generic HTTP retry policy by default and rename dedicated decorator
ClosedPublic

Authored by anlambert on Sep 21 2022, 5:02 PM.

Details

Summary

Instead of retrying HTTP requests only for 429 status code by default,
prefer to use the generic retry policy enabling to also retry for status
codes >= 500 but also on ConnectionError exceptions.

Rename throttling_retry decorator to http_retry to reflect this change.

Diff Detail

Repository
rDLS Listers
Branch
http-retry-generic-policy
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 31716
Build 49626: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 49625: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8519 (id=30680)

Rebasing onto bd35d54398...

Current branch diff-target is up to date.
Changes applied before test
commit 6284d34b1725ce9c5e9f76359a2760da871edb7b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Sep 21 16:56:34 2022 +0200

    Use generic HTTP retry policy by default and rename dedicated decorator
    
    Instead of retrying HTTP requests only for 429 status code by default,
    prefer to use the generic retry policy enabling to also retry for status
    codes >= 500 but also on ConnectionError exceptions.
    
    Rename throttling_retry decorator to http_retry to reflect this change.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/671/ for more details.

ardumont added a subscriber: ardumont.

Ok

I'm not so sure in real life, 500 -> 500 -> 200 will happen that much often but let's see.

This revision is now accepted and ready to land.Sep 22 2022, 9:25 AM

Build is green

Patch application report for D8519 (id=30739)

Rebasing onto 9b3e565cf7...

Current branch diff-target is up to date.
Changes applied before test
commit 9c55acd286091acb6f6094e9fe1c95aca1fdeeec
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Sep 21 16:56:34 2022 +0200

    Use generic HTTP retry policy by default and rename dedicated decorator
    
    Instead of retrying HTTP requests only for 429 status code by default,
    prefer to use the generic retry policy enabling to also retry for status
    codes >= 500 but also on ConnectionError exceptions.
    
    Rename throttling_retry decorator to http_retry to reflect this change.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/684/ for more details.