Add a new size-tiering implementation for the objstorage multiplexer
AbandonedPublic
Actions

Authored by olasd on Jun 26 2018, 6:17 PM.

Details

Reviewers

None

Group Reviewers

Reviewers

Summary

This allows us to send objects to different backends according to their size,
for instance different RADOS pools with different settings.

Test Plan

A new unit test has been added to make sure the tiering behavior really works

Repository

rDOBJS Object storage

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 1263
Build 1607: arc lint + arc unit

Since there are also some code for checking bounds are contiguous, don't you think adding a test for it as well would be good?

Other than my questions, this seems good to go.

swh/objstorage/multiplexer/size_tiering_objstorage.py
59	I'm wondering if this should go to the `check_config` method? I get that we don't want to instantiate a wrong ObjStorage if the config is a nogo. But isn't it why the check_config was initiated in the first place?
87	I did not know about this part. Reading the swh.objstorage.multiplexer.multiplexer_objstorage.ObjStorageThread.

Since there are also some code for checking bounds are contiguous, don't you think adding a test for it as well would be good?

Yeah, it would :-) I'll add those tests in a moment.

swh/objstorage/multiplexer/size_tiering_objstorage.py
59	I've toyed with moving these checks in the `check_config` call (I think I put them there initially), but I really do need to initialize the `upper_bounds` attribute at the point of instantiation. I don't like the pattern of setting up a mandatory attribute somewhere else than `__init__`.