diff --git a/docs/infrastructure/hypervisors.rst b/docs/infrastructure/hypervisors.rst index 403779f..05fe26a 100644 --- a/docs/infrastructure/hypervisors.rst +++ b/docs/infrastructure/hypervisors.rst @@ -1,26 +1,29 @@ =========== Hypervisors =========== Software Heritage uses a few hypervisors configured in a Proxmox cluster List of Proxmox nodes ===================== - beaubourg: Xeon E7-4809 server, 16 cores/512 GB RAM, bought in 2015 - hypervisor3: EPYC 7301 server, 32 cores/256 GB RAM, bought in 2018 +- orsay: Opteron 6172, 48 cores/128 GB RAM, refurbished (2010 vintage) + +Orsay is not a production machine, its purpose is to run throw-away development/staging VMs. Per-node storage ================ The servers each have physically installed 2.5" SSDs (SAS or SATA), configured in mdadm RAID10 pools. A device mapper layer on top of these pools allows Proxmox to easily manage VM disk images. Network storage =============== A :ref:`ceph_cluster` is setup as a shared storage resource. It can be used to temporarily transfer VM disk images from one hypervisor node to another, or to directly store virtual machine disk images. diff --git a/docs/infrastructure/storage_site_azure_euwest.rst b/docs/infrastructure/storage_site_azure_euwest.rst index 7bf85d5..249c00e 100644 --- a/docs/infrastructure/storage_site_azure_euwest.rst +++ b/docs/infrastructure/storage_site_azure_euwest.rst @@ -1,38 +1,32 @@ Azure Euwest ============ virtual machines ---------------- - dbreplica0: contains a read-only instance of the *softwareheritage* database - dbreplica1: contains a read-only instance of the *softwareheritage-indexer* database -- kafka01 to 06 +- kafka01 to 06: journal nodes - mirror-node-1 to 3 -- storage0 -- vangogh (vault implementation) -- webapp0 -- worker01 to 13 +- storage0: storage and object storage services used by the Azure workers +- vangogh: vault service and r/w database for the vault workers +- webapp0: webapp mirror using storage0 services to expose results +- worker01 to 10 and worker13: indexer workers +- worker11 to 12: vault workers (cooking) The PostgreSQL databases are populated using wal streaming from *somerset*. storage accounts ---------------- 16 Azure storage account (0euwestswh to feuwestswh) are dedicated to blob containers for object storage. The first hexadecimal digit of an account name is also the first digit of its content hashes. -Blobs are storred in location names of the form *6euwestswh/contents* +Blobs are stored in location names of the form *6euwestswh/contents* Other storage accounts: - archiveeuwestswh: mirrors of dead software forges like *code.google.com* - swhvaultstorage: cooked archives for the *vault* server running in azure. - swhcontent: object storage content (individual blobs) - - -TODO: describe kafka* virtual machines -TODO: describe mirror-node* virtual machines -TODO: describe storage0 virtual machine -TODO: describe webapp0 virtual machine -TODO: describe worker* virtual machines diff --git a/docs/infrastructure/storage_site_rocquencourt_virtual.rst b/docs/infrastructure/storage_site_rocquencourt_virtual.rst index 99bf1a7..664f735 100644 --- a/docs/infrastructure/storage_site_rocquencourt_virtual.rst +++ b/docs/infrastructure/storage_site_rocquencourt_virtual.rst @@ -1,43 +1,43 @@ Virtual machines at Rocquencourt ================================ The following virtual machines are hosted on Proxmox hypervisors located at Rocquencourt. All of them use local storage on their virtual hard drive. VMs without NFS mount points ---------------------------- - munin0 - tate, used for public and private (intranet) wikis - getty - thyssen - jenkins-debian1.internal.softwareheritage.org - logstash0 - kibana0 - saatchi - louvre Containers and VMs with nfs storage: ------------------------------------ - somerset.internal.softwareheritage.org is a lxc container running on *beaubourg* It serves as a host for the *softwareheritage* and *softwareheritage-indexer* databases. -- worker01 to worker16.internal.softwareheritage.org -- pergamon -- moma +- worker01 to worker16.internal.softwareheritage.org: loader and lister workers +- pergamon: internal system administration services (puppet master, grafana, dns resolver, etc...) +- moma: webapp and deposit services exposed publicly These VMs access one or more of these NFS volumes located on uffizi:: uffizi:/srv/softwareheritage/objects uffizi:/srv/storage/space uffizi:/srv/storage/space/annex uffizi:/srv/storage/space/annex/public uffizi:/srv/storage/space/antelink uffizi:/srv/storage/space/oversize-objects uffizi:/srv/storage/space/personal uffizi:/srv/storage/space/postgres-backups/somerset uffizi:/srv/storage/space/provenance-index uffizi:/srv/storage/space/swh-deposit