Differential D8519

Use generic HTTP retry policy by default and rename dedicated decorator
ClosedPublic
Actions

Authored by anlambert on Sep 21 2022, 5:02 PM.

Details

Reviewers

ardumont

Group Reviewers

Reviewers

Commits

rDLS9c55acd28609: Use generic HTTP retry policy by default and rename dedicated decorator

Summary

Instead of retrying HTTP requests only for 429 status code by default,
prefer to use the generic retry policy enabling to also retry for status
codes >= 500 but also on ConnectionError exceptions.

Rename throttling_retry decorator to http_retry to reflect this change.

Diff Detail

Repository

rDLS Listers

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Sep 21 2022, 5:02 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptSep 21 2022, 5:02 PM

Build is green

Patch application report for D8519 (id=30680)

Rebasing onto bd35d54398...

Current branch diff-target is up to date.

Changes applied before test

commit 6284d34b1725ce9c5e9f76359a2760da871edb7b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Sep 21 16:56:34 2022 +0200

    Use generic HTTP retry policy by default and rename dedicated decorator
    
    Instead of retrying HTTP requests only for 429 status code by default,
    prefer to use the generic retry policy enabling to also retry for status
    codes >= 500 but also on ConnectionError exceptions.
    
    Rename throttling_retry decorator to http_retry to reflect this change.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/671/ for more details.

Harbormaster completed remote builds in B31658: Diff 30680.Sep 21 2022, 5:07 PM

anlambert requested review of this revision.Sep 21 2022, 5:07 PM

anlambert added a child revision: D8520: Refactor and deduplicate HTTP requests code in listers.Sep 21 2022, 8:09 PM

I'm not so sure in real life, 500 -> 500 -> 200 will happen that much often but let's see.

This revision is now accepted and ready to land.Sep 22 2022, 9:25 AM

Rebase

Build is green

Patch application report for D8519 (id=30739)

Rebasing onto 9b3e565cf7...

Current branch diff-target is up to date.

Changes applied before test

commit 9c55acd286091acb6f6094e9fe1c95aca1fdeeec
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Sep 21 16:56:34 2022 +0200

    Use generic HTTP retry policy by default and rename dedicated decorator
    
    Instead of retrying HTTP requests only for 429 status code by default,
    prefer to use the generic retry policy enabling to also retry for status
    codes >= 500 but also on ConnectionError exceptions.
    
    Rename throttling_retry decorator to http_retry to reflect this change.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/684/ for more details.

Harbormaster completed remote builds in B31716: Diff 30739.Sep 26 2022, 10:55 AM

Closed by commit rDLS9c55acd28609: Use generic HTTP retry policy by default and rename dedicated decorator (authored by anlambert). · Explain WhySep 26 2022, 11:36 AM

This revision was automatically updated to reflect the committed changes.

anlambert added a commit: rDLS9c55acd28609: Use generic HTTP retry policy by default and rename dedicated decorator.

franckbret mentioned this in D8531: Puppet: Lister for Puppet modules.Sep 26 2022, 12:01 PM

franckbret mentioned this in D8529: RubyGems: List origins from https://rubygems.org.Sep 26 2022, 12:10 PM

franckbret mentioned this in D8528: Nuget: Lister for NuGet the package manager for .NET.Sep 26 2022, 12:13 PM

franckbret mentioned this in D8517: Conda: List origins from anaconda.com, the Package, dependency and environment management for any language.Sep 26 2022, 12:17 PM

franckbret mentioned this in D8338: Hackage: List origins from hackage.haskell.org, The Haskell Package Repository.Sep 26 2022, 12:27 PM