Page MenuHomeSoftware Heritage

High disk utilization on dbreplica0.euwest.azure
Open, NormalPublic

Description

Disk utilization of postgresql's volume on dbreplica0 is currently at 100% and iowait is high (400%). This is likely to be the current bottleneck of the indexers.

Currently, it uses 4 disks of 2TB each (a priori using RAID 0). Each of these is an SSD Premium, so according to Azure's doc, that's 4×7500IOPS.

We might want to switch another set of disks, like 8 disks of 1TB each. This would allow more IOPS (8×5000 according to the doc) for approximately the same price.

There is also the possibility of "SSD Ultra" disks, which allow for provisioning higher IOPS. It looks like provisioning for the same IOPS and MB/s we currently have with "SSD Premium" disks would cost approximately the same. But then we could increase the IOPS and decrease MB/s as we want.
Unfortunately, these disks are only available in the US for now, which would increase latency somewhere in our pipeline.

Event Timeline

vlorentz created this task.Feb 5 2019, 2:48 PM
vlorentz triaged this task as Normal priority.
vlorentz added a subscriber: ardumont.