Page MenuHomeSoftware Heritage
Differential D8887 Diff 32050 sysadm/mirror-operations/planning.rst

Changeset View

Standalone View

View Options

sysadm/mirror-operations/planning.rst

Show First 20 Lines • Show All 111 Lines • ▼ Show 20 Lines
vault worker swh-vault 1 16GB fast 1TB vault worker swh-vault 1 16GB fast 1TB
vault rabbitmq 1 8GB regular 10GB vault rabbitmq 1 8GB regular 10GB
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
Storage backend Storage backend
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
.. tabbed:: Postgresql .. tab-set::
.. tab-item:: Postgresql
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
SWH Service Tool Instances RAM Storage Type Storage Volume SWH Service Tool Instances RAM Storage Type Storage Volume
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
storage postgresql 1 512GB fast+zfs (lz4) 40TB storage postgresql 1 512GB fast+zfs (lz4) 40TB
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
.. tabbed:: Cassandra (min.)* .. tab-item:: Cassandra (min.)*
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
SWH Service Tool Instances RAM Storage Type Storage Volume SWH Service Tool Instances RAM Storage Type Storage Volume
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
storage cassandra 3 32GB fast 30TB storage cassandra 3 32GB fast 30TB
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
.. tabbed:: Cassandra (typ.)* .. tab-item:: Cassandra (typ.)*
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
SWH Service Tool Instances RAM Storage Type Storage Volume SWH Service Tool Instances RAM Storage Type Storage Volume
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
storage cassandra 6+ 32GB fast 20TB storage cassandra 6+ 32GB fast 20TB
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
Objstorage backend Objstorage backend
^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
.. tab-set::
.. tabbed:: FS .. tab-item:: FS
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
SWH Service Tool Instances RAM Storage Type Storage Volume SWH Service Tool Instances RAM Storage Type Storage Volume
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
objstorage swh-objstorage 1 [#f1]_ 512GB zfs (with lz4) 1PB objstorage swh-objstorage 1 [#f1]_ 512GB zfs (with lz4) 1PB
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
.. tabbed:: Winery - Ceph* .. tab-item:: Winery - Ceph*
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
SWH Service Tool Instances RAM Storage Type Storage Volume SWH Service Tool Instances RAM Storage Type Storage Volume
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
objstorage swh-objstorage 2 [#f2]_ 32GB standard 100GB objstorage swh-objstorage 2 [#f2]_ 32GB standard 100GB
winery-db postgresql 2 [#f2]_ 512GB fast 10TB winery-db postgresql 2 [#f2]_ 512GB fast 10TB
ceph-mon ceph 3 4GB fast 60GB ceph-mon ceph 3 4GB fast 60GB
ceph-osd ceph 12+ 4GB mix fast+HDD 1PB (total) ceph-osd ceph 12+ 4GB mix fast+HDD 1PB (total)
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
.. tabbed:: Seaweedfs* .. tab-item:: Seaweedfs*
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
SWH Service Tool Instances RAM Storage Type Storage Volume SWH Service Tool Instances RAM Storage Type Storage Volume
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
objstorage swh-objstorage 3 32GB standard 100GB objstorage swh-objstorage 3 32GB standard 100GB
seaweed LB nginx 1 32GB fast 100GB seaweed LB nginx 1 32GB fast 100GB
seaweed-master seaweedfs 3 8GB standard 10GB seaweed-master seaweedfs 3 8GB standard 10GB
seaweed-filer seaweedfs 3 32GB fast 1TB seaweed-filer seaweedfs 3 32GB fast 1TB
seaweed-volume seaweedfs 3+ 32GB standard 1PB (total) seaweed-volume seaweedfs 3+ 32GB standard 1PB (total)
================ ====================== ========= ===== ============== ============== ================ ====================== ========= ===== ============== ==============
.. rubric:: Notes .. rubric:: Notes
.. [#f1] An swh-objstorage using :py:mod:`simple filesystem .. [#f1] An swh-objstorage using :py:mod:`simple filesystem
<swh.objstorage.backends.pathslicing>` as backend can actually be <swh.objstorage.backends.pathslicing>` as backend can actually be
split on several machines using the split on several machines using the
:py:mod:`swh.objstorage.multiplexer` backend. :py:mod:`swh.objstorage.multiplexer` backend.
.. [#f2] The swh-objstorage RPC service and the index database can be hosted on .. [#f2] The swh-objstorage RPC service and the index database can be hosted on
the same machine. the same machine.
About · Developer information · Legal