Page MenuHomeSoftware Heritage

Enlarge disk on dbreplica1
Closed, ResolvedPublic

Description

dbreplica1 overflowed its storage. We need to add some more

Event Timeline

olasd triaged this task as Unbreak Now! priority.Nov 28 2018, 2:35 PM
olasd created this task.
olasd added a comment.Nov 28 2018, 3:04 PM

Target size: 8 TB striped across 8 disks (from 2 TB on a single disk).

First off the machine was too small. Resized from E2s_v2 (4 data disks) to E8s_v3 (16 data disks).

Disk creation:

for i in {1..8}; do az vm disk attach -g euwest-db --vm-name dbreplica1 --disk dbreplica1_pgdata$i --new --size-gb 1024; done

[wait for disks to appear; /dev/sdd to /dev/sdk appear]

Create a template with fdisk (single partition, full disk, type=linux raid autodetect (fd)) then duplicate it

fdisk /dev/sdd
for disk in {e..k}; do sfdisk -d /dev/sdd | sfdisk /dev/sd$disk; done

Create a raid between the disks

mdadm --create /dev/md0 --level 0 --raid-devices 8 /dev/sd{d..k}1

Add the raid to the LVM volume

pvcreate /dev/md0
vgextend data-vg01 /dev/md0

Move data from the old PV to the new PV

pvmove /dev/sdc /dev/md0

[in progress...]

ftigeot closed this task as Resolved.Dec 3 2018, 3:07 PM
ftigeot claimed this task.
ftigeot added a subscriber: ftigeot.

The pvmove command was done this morning.

Next steps I did pairing with ardumont@:

  • Remove old physical device vgreduce -a data-vg01
  • Extend logical volume group lvextend -l"+100%pvs" /dev/mapper/data--vg01-data--lv01 /dev/md0
  • Resize the filesystem resize2fs /dev/mapper/data--vg01-data--lv01