Differential D2052

core/loader: Retry contents loading on HashCollision errors
ClosedPublic
Actions

Authored by anlambert on Oct 1 2019, 10:55 AM.

Tags

None

Subscribers

None

Details

Reviewers

Group Reviewers

Commits

rDLDBASE2a0b2edb23e9: core/loader: Retry contents loading on HashCollision errors

Summary

Here is a simple workaround to avoid HashCollision errors thay may appear when different workers are loading
the exact same content in parallel database transactions.

This clearly does not fix the storage issue but at least it should greatly reduce the amount of HashCollision
errors reported by the vcs loaders in production.

Related T2019

Diff Detail

Repository

rDLDBASE Generic VCS/Package Loader

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Oct 1 2019, 10:55 AM

Herald added a reviewer: Reviewers. · View Herald TranscriptOct 1 2019, 10:55 AM

Build is green
See https://jenkins.softwareheritage.org/job/DLDBASE/job/tox/166/ for more details.

Harbormaster completed remote builds in B8034: Diff 6890.Oct 1 2019, 10:56 AM

ardumont accepted this revision.Oct 1 2019, 10:57 AM

This revision is now accepted and ready to land.Oct 1 2019, 10:57 AM

Closed by commit rDLDBASE2a0b2edb23e9: core/loader: Retry contents loading on HashCollision errors (authored by anlambert). · Explain WhyOct 1 2019, 11:33 AM

This revision was automatically updated to reflect the committed changes.

anlambert added a commit: rDLDBASE2a0b2edb23e9: core/loader: Retry contents loading on HashCollision errors.

Revision Contents
Changeset List

Path

Size

swh/

loader/

core/

3 lines

Diff 6900

swh/loader/core/loader.py

Loading...