Differential D2052

core/loader: Retry contents loading on HashCollision errors
ClosedPublic
Actions

Authored by anlambert on Oct 1 2019, 10:55 AM.

Tags

None

Subscribers

None

Details

Reviewers

Group Reviewers

Commits

rDLDBASE2a0b2edb23e9: core/loader: Retry contents loading on HashCollision errors

Summary

Here is a simple workaround to avoid HashCollision errors thay may appear when different workers are loading
the exact same content in parallel database transactions.

This clearly does not fix the storage issue but at least it should greatly reduce the amount of HashCollision
errors reported by the vcs loaders in production.

Related T2019

Diff Detail

Repository

rDLDBASE Generic VCS/Package Loader

Branch

hash-collision-retry

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 8034
Build 11582: tox-on-jenkins	Jenkins
Build 11581: arc lint + arc unit

Event Timeline

anlambert created this revision.Oct 1 2019, 10:55 AM

Herald added a reviewer: Reviewers. · View Herald TranscriptOct 1 2019, 10:55 AM

Build is green
See https://jenkins.softwareheritage.org/job/DLDBASE/job/tox/166/ for more details.

Harbormaster completed remote builds in B8034: Diff 6890.Oct 1 2019, 10:56 AM

ardumont accepted this revision.Oct 1 2019, 10:57 AM

This revision is now accepted and ready to land.Oct 1 2019, 10:57 AM

Closed by commit rDLDBASE2a0b2edb23e9: core/loader: Retry contents loading on HashCollision errors (authored by anlambert). · Explain WhyOct 1 2019, 11:33 AM

This revision was automatically updated to reflect the committed changes.

anlambert added a commit: rDLDBASE2a0b2edb23e9: core/loader: Retry contents loading on HashCollision errors.

Revision Contents
Changeset List

Path

Size

swh/

loader/

core/

3 lines

Diff 6890

swh/loader/core/loader.py

Loading...