Differential D6796 Diff 24859 docs/winery.rst

Changeset View

Standalone View

docs/winery.rst

This file was added.

.. _swh-objstorage-winery:

Winery backend

==============

The Winery backend implements the `Ceph based object storage architecture <https://wiki.softwareheritage.org/wiki/A_practical_approach_to_efficiently_store_100_billions_small_objects_in_Ceph>`__.

Benchmarks

dacharyAuthorUnsubmitted

Done

@olasd the high level description that you suggested to add is here.

dachary: @olasd the high level description that you suggested to add is here.

----------

Setup via Fed4Fire

vlorentzUnsubmitted

Not Done

--------------------

- The `sharedstorage.py` file contains the global index implementation that associates every object id to the shard it contains. A list of shard (either writable or readonly) is stored in a table, with a numeric id to save space. The name of the shard is used to create a database (for write shards) or a RBD image (for read shards).

+ The ``sharedstorage.py`` file contains the global index implementation that associates every object id to the shard it contains. A list of shard (either writable or readonly) is stored in a table, with a numeric id to save space. The name of the shard is used to create a database (for write shards) or a RBD image (for read shards).

The `roshard.py` file contain the lookup function for a read shard and is a thin layer on top of swh-perfect hash.

because this isn't markdown (ditto below)

vlorentz: because this isn't markdown (ditto below)

~~~~~~~~~~~~~~~~~~

* https://www.fed4fire.eu/

Create a base rspec specification.

* /opt/jFed/jFed-Experimenter

* In the General Tab

* Create an experiment (New)

* Add one Physical Node by dragging it

* Right click on the node and choose "Configure Node"

* Select testbed: Grid 5000

* Node => Specific hardware type: dahu-grenoble

* Disk image => Bullseye base

* Save under sample.rspec

* Manually edit to duplicate the nodes

Run the experiment.

* /opt/jFed/jFed-Experimenter

* In the General Tab

* Open Local and load winery-test-environment/fed4fire.rspec

* Edit ceph1 node to check if the Specific hardware type is dahu-grenoble

* Click on Topology Viewer

* Run

* Give a unique name to the experiment

* Start experiment

* Once the provisionning is complete (Testing connectivity to resources on Grid5000) click "Export As"

* Choose "Export Configuration Management Settings"

* Save under /tmp/test.zip

* ( cd winery-test-environment ; ./fed4fire.sh /tmp/test.zip )