Page MenuHomeSoftware Heritage

Enlarge disk on dbreplica1
Closed, ResolvedPublic

Description

dbreplica1 overflowed its storage. We need to add some more

Event Timeline

olasd created this task.Nov 28 2018, 2:35 PM
olasd triaged this task as Unbreak Now! priority.
olasd added a comment.Nov 28 2018, 3:04 PM

Target size: 8 TB striped across 8 disks (from 2 TB on a single disk).

First off the machine was too small. Resized from E2s_v2 (4 data disks) to E8s_v3 (16 data disks).

Disk creation:

for i in {1..8}; do az vm disk attach -g euwest-db --vm-name dbreplica1 --disk dbreplica1_pgdata$i --new --size-gb 1024; done

[wait for disks to appear; /dev/sdd to /dev/sdk appear]

Create a template with fdisk (single partition, full disk, type=linux raid autodetect (fd)) then duplicate it

fdisk /dev/sdd
for disk in {e..k}; do sfdisk -d /dev/sdd | sfdisk /dev/sd$disk; done

Create a raid between the disks

mdadm --create /dev/md0 --level 0 --raid-devices 8 /dev/sd{d..k}1

Add the raid to the LVM volume

pvcreate /dev/md0
vgextend data-vg01 /dev/md0

Move data from the old PV to the new PV

pvmove /dev/sdc /dev/md0

[in progress...]

ftigeot closed this task as Resolved.Dec 3 2018, 3:07 PM
ftigeot claimed this task.
ftigeot added a subscriber: ftigeot.

The pvmove command was done this morning.

Next steps I did pairing with ardumont@:

  • Remove old physical device vgreduce -a data-vg01
  • Extend logical volume group lvextend -l"+100%pvs" /dev/mapper/data--vg01-data--lv01 /dev/md0
  • Resize the filesystem resize2fs /dev/mapper/data--vg01-data--lv01