Fix objects loading when using generator parameters
ClosedPublic
Actions

Authored by anlambert on Jan 16 2020, 5:24 PM.

Details

Reviewers

ardumont

Group Reviewers

Reviewers

Commits

rDSTOCcc29708564c3: storage: Fix objects loading when using generator parameters
rDSTOcc29708564c3: storage: Fix objects loading when using generator parameters

Summary

Some objects (directories, origins, releases, revisions) will not be added into the storage
if they are provided as generator parameters to the Storage::*_add methods instead of lists.

This is why the swh-vault tests are currently failing as since the removal of the retry
mechanism in swh-loader-core, generators are provided as parameters to the Storage::*_add
methods while previously lists of objects were provided.

As a simple fix, ensure to transform generators into lists before processing the objects.

Diff Detail

Repository

rDSTO Storage manager

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Jan 16 2020, 5:24 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJan 16 2020, 5:24 PM

anlambert added a reviewer: ardumont.Jan 16 2020, 5:24 PM

Build is green
See https://jenkins.softwareheritage.org/job/DSTO/job/tox/862/ for more details.

Harbormaster completed remote builds in B10153: Diff 9092.Jan 16 2020, 5:27 PM

anlambert edited the summary of this revision. (Show Details)Jan 16 2020, 5:27 PM

ardumont accepted this revision.Jan 16 2020, 5:30 PM

This revision is now accepted and ready to land.Jan 16 2020, 5:30 PM

Update copyright years in test_storage.py

Build is green
See https://jenkins.softwareheritage.org/job/DSTO/job/tox/863/ for more details.

Harbormaster completed remote builds in B10154: Diff 9093.Jan 16 2020, 5:40 PM

And thanks, i was debugging the behavior in both loader and vault.
I was converging on something is wrong in the storage... But i was not there yet ;)

And thanks, i was debugging the behavior in both loader and vault.
I was converging on something is wrong in the storage... But i was not there yet ;)

Indeed, the issue was pretty subtle.
I will tag a new swh-storage version to remove the red light for swh-vault on Jenkins.