Page MenuHomeSoftware Heritage

Fix objects loading when using generator parameters
ClosedPublic

Authored by anlambert on Jan 16 2020, 5:24 PM.

Details

Summary

Some objects (directories, origins, releases, revisions) will not be added into the storage
if they are provided as generator parameters to the Storage::*_add methods instead of lists.

This is why the swh-vault tests are currently failing as since the removal of the retry
mechanism in swh-loader-core, generators are provided as parameters to the Storage::*_add
methods while previously lists of objects were provided.

As a simple fix, ensure to transform generators into lists before processing the objects.

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Jan 16 2020, 5:24 PM
anlambert edited the summary of this revision. (Show Details)Jan 16 2020, 5:27 PM
ardumont accepted this revision.Jan 16 2020, 5:30 PM
This revision is now accepted and ready to land.Jan 16 2020, 5:30 PM
anlambert updated this revision to Diff 9093.Jan 16 2020, 5:37 PM

Update copyright years in test_storage.py

And thanks, i was debugging the behavior in both loader and vault.
I was converging on something is wrong in the storage... But i was not there yet ;)

And thanks, i was debugging the behavior in both loader and vault.
I was converging on something is wrong in the storage... But i was not there yet ;)

Indeed, the issue was pretty subtle.
I will tag a new swh-storage version to remove the red light for swh-vault on Jenkins.

anlambert updated this revision to Diff 9094.Jan 16 2020, 5:54 PM

Remove some not needed empty lines in newly introduced tests.

I will tag a new swh-storage version to remove the red light for swh-vault on Jenkins.

Miam! ;)