HomeSoftware Heritage

retry: Add retry behavior on pipeline storage with flushing failure

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

retry: Add retry behavior on pipeline storage with flushing failure

Currently, wrong "hash collisions" are happening a lot on ingestion [1] [2] [3]
The last loading step (flush) is failing on most loaders (git, npm, etc...).

This commits adds the retry behavior to the current pipeline storage deployed.
Which should decrease the frequency of that error.

The remaining hash collision which won't subside should be then real hash
collisions.

[1] https://sentry.softwareheritage.org/share/issue/102aace238fe4ba6b49bcc5531f7c2bf/

[2] https://sentry.softwareheritage.org/share/issue/8e8b48a1d94c465b8109e76311ecdbe7/

[3] https://sentry.softwareheritage.org/share/issue/d4f1208b7eec4b43b11e38494ff039cc/

Details

Provenance
ardumontAuthored on Jan 30 2020, 12:22 PM
ardumontPushed on Jan 30 2020, 1:39 PM
Differential Revision
D2603: pipeline storage: Add retry behavior on flushing failures
Build Status
Buildable 10373
Build 15430: test-and-buildJenkins

Commit No Longer Exists

This commit no longer exists in the repository.