HomeSoftware Heritage

sourceforge: retry for all retryable exceptions

Description

sourceforge: retry for all retryable exceptions

Since this lister is doing a lot more requests than most other, it makes
sense that issues would arise more often. We want the lister to continue
even if the website is having issues and not break on the first 500 or
closed connection it encounters.

This change introduces a mechanism to retry all exceptions worth
retrying and uses it for the SourceForge lister. Other listers might
benefit from this, but this is out of scope here.

Tests had to be adjusted to stub the sleep function since retries happened
way more often.

Details

Provenance
AlphareAuthored on May 26 2021, 12:05 PM
AlpharePushed on May 26 2021, 12:39 PM
Differential Revision
D5785: sourceforge: retry for all retryable exceptions
Parents
rDLS8f3bbacd5eee: sourceforge: don't abort on error for project
Branches
Unknown
Tags
Unknown
References
tag: v1.3.2
Tasks
T3310: Deploy sourceforge lister on staging
Build Status
Buildable 21647
Build 33643: test-and-buildJenkins console · Jenkins