The I/O read performances on banco seems to be much better (50% more, 1.5x) than those on uffizi, even after discounting the slow down due to libvirt (see T283). The only significant difference seems to be XFS tuning, which has been tuned for performances to match strip sizes on uffizi (where it is slower!) but not on banco.
We need to review XFS performance tuning, and improve uffizi's I/O performances to match banco.
Banco's benchs:
zack@banco$ bonnie++ -d /srv/softwareheritage/objects/01/bonnie++
Seq Out | Seq In | Random | |||
Char | Block | Rewrite | Char | Block | Seeks |
K/s - CPU | K/s - CPU | K/s - CPU | K/s - CPU | K/s - CPU | /s - CPU |
1211 - 99% | 952310 - 98% | 569833 - 66% | 2612 - 99% | 1503015 - 88% | 576.2 - 22% |
8600us | 17858us | 325ms | 7576us | 336ms | 133ms |
Seq Create | Random Create | ||||
Create | Read | Delete | Create | Read | Delete |
/sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU |
32351 - 81% | +++++ - +++% | +++++ - +++% | 32666 - 82% | +++++ - +++% | +++++ - +++% |
22834us | 136us | 11007us | 14554us | 12us | 16350us |
zack@banco$ bonnie++ -d /srv/softwareheritage/objects/a1/bonnie++
Seq Out | Seq In | Random | |||
Char | Block | Rewrite | Char | Block | Seeks |
K/s - CPU | K/s - CPU | K/s - CPU | K/s - CPU | K/s - CPU | /s - CPU |
1230 - 99% | 988938 - 98% | 558315 - 67% | 2561 - 97% | 1566189 - 88% | 586.5 - 22% |
7934us | 15883us | 346ms | 21770us | 140ms | 98680us |
Seq Create | Random Create | ||||
Create | Read | Delete | Create | Read | Delete |
/sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU |
+++++ - +++% | +++++ - +++% | +++++ - +++% | +++++ - +++% | +++++ - +++% | +++++ - +++% |
6700us | 131us | 10769us | 301us | 15us | 104us |
Uffizi's bench (via louvre, bare metal):
zack@louvre$ mount|grep /mnt /dev/mapper/vg--data-0 on /mnt type xfs (rw,relatime,attr2,inode64,noquota) zack@louvre$ bonnie++ -d /mnt/bonnie
Seq Out | Seq In | Random | |||
Char | Block | Rewrite | Char | Block | Seeks |
K/s - CPU | K/s - CPU | K/s - CPU | K/s - CPU | K/s - CPU | /s - CPU |
1404 - 99% | 633900 - 99% | 478570 - 75% | 2820 - 99% | 919112 - 91% | 345.1 - 27% |
7770us | 244ms | 359ms | 10967us | 513ms | 111ms |
Seq Create | Random Create | ||||
Create | Read | Delete | Create | Read | Delete |
/sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU |
28184 - 88% | +++++ - +++% | +++++ - +++% | 28916 - 84% | +++++ - +++% | +++++ - +++% |
478us | 151us | 14701us | 444us | 15us | 43015us |
zack@louvre$ mount | grep /mnt /dev/mapper/vg--data-a on /mnt type xfs (rw,relatime,attr2,inode64,noquota) zack@louvre$ bonnie++ -d /mnt/bonnie
Seq Out | Seq In | Random | |||
Char | Block | Rewrite | Char | Block | Seeks |
K/s - CPU | K/s - CPU | K/s - CPU | K/s - CPU | K/s - CPU | /s - CPU |
1317 - 99% | 655989 - 99% | 480398 - 75% | 2571 - 95% | 948253 - 92% | 350.6 - 25% |
9479us | 297ms | 10843ms | 38326us | 358ms | 136ms |
Seq Create | Random Create | ||||
Create | Read | Delete | Create | Read | Delete |
/sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU | /sec - CPU |
28950 - 85% | +++++ - +++% | +++++ - +++% | 29147 - 88% | +++++ - +++% | +++++ - +++% |
478us | 112us | 13591us | 355us | 30us | 24317us |