Page MenuHomeSoftware Heritage

buffer: add support for snapshots
ClosedPublic

Authored by olasd on Feb 4 2021, 2:41 PM.

Details

Summary

This is mostly a consistency addition, considering that most (if not
all) loaders will only add a single snapshot.

The common pattern of loading objects in topological order (content >
directory > revision > release > snapshot), then flushing the storage,
is now fully consistent; Without this addition, the snapshot addition
would reach the backend storage before all other objects are added,
leading to potential inconsistencies if the flush of other object types
fails.

Depends on D5013.

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D5015 (id=17886)

Could not rebase; Attempt merge onto 9a9f234e0a...

Updating 9a9f234e..5b3e6c9f
Fast-forward
 swh/storage/buffer.py            |   7 ++-
 swh/storage/tests/test_buffer.py | 130 +++++++++++++++++++++++++++++++--------
 2 files changed, 110 insertions(+), 27 deletions(-)
Changes applied before test
commit 5b3e6c9f70f94f4ae7440de453fd75cd1d3bbeb6
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Thu Feb 4 09:56:03 2021 +0100

    buffer: add support for snapshots
    
    This is mostly a consistency addition, considering that most (if not
    all) loaders will only add a single snapshot.
    
    The common pattern of loading objects in topological order (content >
    directory > revision > release > snapshot), then flushing the storage,
    is now fully consistent; Without this addition, the snapshot addition
    would reach the backend storage before all other objects are added,
    leading to potential inconsistencies if the flush of other object types
    fails.

commit 18967ed4a5f2eb67c57ae34ed913073b45bedc90
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Thu Feb 4 10:14:59 2021 +0100

    buffer: add type annotations for tests

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1123/ for more details.

olasd requested review of this revision.Feb 4 2021, 2:46 PM
ardumont added a subscriber: ardumont.

makes sense.

This revision is now accepted and ready to land.Feb 4 2021, 2:59 PM
This revision was automatically updated to reflect the committed changes.