100k files sample

head -100000 0-ext4 > sample.0
sed 's@^\(..\)\(..\)\(..\)@/srv/storage/0/\1/\2/\3/\1\2\3@' sample.0 > sample.0.xfs
sed 's@^\(.....\)@/srv/storage/0-ext4/\1/\1@' sample.0 > sample.0.ext4

Read all files

olasd@banco:~$ time while read f; do cat $f; done >/dev/null < sample.0.xfs 

real    106m10.771s
user    0m42.064s
sys     0m32.796s

olasd@banco:~$ time while read f; do cat $f; done >/dev/null < sample.0.ext4
   
real    73m46.195s
user    0m46.780s
sys     0m36.308s

Stat all files

olasd@banco:~$ head -200000 0-ext4 | tail -100000 > sample.1
olasd@banco:~$ wc -l sample.1
100000 sample.1
olasd@banco:~$ sed 's@^\(..\)\(..\)\(..\)@/srv/storage/0/\1/\2/\3/\1\2\3@' sample.1 > sample.1.xfs
olasd@banco:~$ sed 's@^\(.....\)@/srv/storage/0-ext4/\1/\1@' sample.1 > sample.1.ext4
olasd@banco:~$ time while read f; do stat $f; done >/dev/null < sample.1.xfs 

real    88m22.129s
user    0m45.576s
sys     0m39.192s
olasd@banco:~$ time while read f; do stat $f; done >/dev/null < sample.1.ext4

real    32m14.480s
user    0m33.428s
sys     0m36.204s

In T450#7115, @olasd wrote:

olasd@banco:~$ head -200000 0-ext4 | tail -100000 > sample.1

Just to be sure we're on the same page: this was to avoid stat()-ing the same files you've "just" read() with the previous test, right?

Aside from that, this looks promising: ~30% gain in read() (where the performance boost is likedly capped by the large files in there), ~300% gain in stat(). It's a pretty decent speed up. Although it's not earth shattering—or maybe I was expecting too much out of this.

Do you have other tests in the working?
If not, I'd like to run these numbers through Josh, to see if they match his expectations before taking a decision about what to do.

In T450#7116, @zack wrote:

In T450#7115, @olasd wrote:

olasd@banco:~$ head -200000 0-ext4 | tail -100000 > sample.1

Just to be sure we're on the same page: this was to avoid stat()-ing the same files you've "just" read() with the previous test, right?

Yeah, to try to avoid cache effects.

Aside from that, this looks promising: ~30% gain in read() (where the performance boost is likedly capped by the large files in there), ~300% gain in stat(). It's a pretty decent speed up. Although it's not earth shattering—or maybe I was expecting too much out of this.

Do you have other tests in the working?
If not, I'd like to run these numbers through Josh, to see if they match his expectations before taking a decision about what to do.

No other tests in the pipeline; I'm open to ideas :)