HomeSoftware Heritage

loader: Fix rsync failures by retrying associated commands

Description

loader: Fix rsync failures by retrying associated commands

Fetching CVS repository data using rsync often fails with the following error
(especially with archived repositories hosted on sourceforge):

rsync error: some files/attrs were not transferred (see previous errors) (code 23)

It seems the only way to mitigate that issue is to retry the rsync command
until it succeeds.

So add a rsync_retry decorator and apply it to a new method in the loader
wrapping the call to subprocess.run executing the rsync command.

Also use rsync option to compress file data during the transfer.

Details

Provenance
anlambertAuthored on Apr 14 2022, 5:12 PM
anlambertPushed on Apr 14 2022, 5:40 PM
Differential Revision
D7586: loader: Fix rsync failures by retrying associated commands
Parents
rDLDCVSc82c3268656d: Add .git-blame-ignore-revs file with automatic reformatting commits
Branches
Unknown
Tags
Unknown
References
tag: v0.1.1
Build Status
Buildable 28489
Build 44544: test-and-buildJenkins console · Jenkins