Page MenuHomeSoftware Heritage

[production] Provision enough space for the search ES cluster to ingest all intrinsic metadata
Open, NormalPublic

Description

11:25 <+vsellier> we will have to check the elasticsearch behavior with production volume
11:26 <+vsellier> the size increases have been important on staging
11:26 <+vsellier> around x5
11:26 <+vsellier> (the index size)
11:27 <+vsellier> if the ratio is the same for production, the bump could be from 250g to 1.2T
11:28 <+vsellier> the search-esnode are not yet sized for this
11:31 <vlorentz> shouldn't we stop the metadata ingestion until they are?
11:31 <+vsellier> (they are vms on proxmox, we will have to think where we can find so much space)
11:32 <+vsellier> the search journal client is not deployed so no problem for the moment, nothing is sent to ES
11:33 <+vsellier> (I mean not deployed in production, it is only on staging)

Event Timeline

vlorentz triaged this task as Normal priority.Thu, Feb 11, 1:17 PM
vlorentz created this task.
vsellier renamed this task from Provision enough space for the search ES cluster to ingest all intrinsic metadata to [production] Provision enough space for the search ES cluster to ingest all intrinsic metadata.Mon, Feb 15, 10:02 AM