Page MenuHomeSoftware Heritage

Migrate to zfs
Closed, MigratedEdits Locked

Description ran out of disk space.

somerset is piling up WALs waiting on this machine to get space to apply them.

Instead of adding more disks to the current lvm, we should move the data to zfs.

Event Timeline

olasd changed the task status from Open to Work in Progress.Sep 8 2020, 3:16 PM
olasd triaged this task as High priority.
olasd created this task.
  • enabled non-free
  • installed build-essential
  • installed linux-image-cloud-amd64, linux-headers-cloud-amd64 from bpo
  • installed zfs-dkms from bpo
  • use resource disk as cache:
    • in /etc/waagent.conf, set Resourcedisk.Format=n
    • prepare /etc/systemd/system/zfs-cache-azure.service

ExecStartPre=-zpool remove data sdb
ExecStart=zpool add data cache sdb
ExecStop=zpool remove data sdb

  • reboot on bpo kernel with zfs module enabled

I've replaced zfs-cache-azure.service with the following:


ExecStartPre=-zpool remove data /dev/disk/azure/resource
ExecStart=zpool add data cache /dev/disk/azure/resource
ExecStop=zpool remove data /dev/disk/azure/resource


(turns out the resource disk pops up as sda on this machine...)

For now, the machine is set up with the following layout:

  • 2 special allocation class devices, 256GiB each
  • 6 x 1 TB HDDs for data
  • the azure resource disk as L2ARC

I've followed the instructions on for the dataset tuning - slide 82 onwards


This layout manages 300 MB/s write speeds, we'll see if that's enough...

The migration of the data to zfs is complete, and the database has been restarted.

The old LVM now needs to be cleaned up.