[Linux-cluster] GFS write performance with small files

[Alexander.Laamanen at tecnomen.com]

Hi,
 
We did some GFS benchmarking using an application that writes many small
files. There seems to be a big overhead (in the amount of actual data
written to the device) with these small files. 
Info:
- Mount options: noatime,lockproto=lock_nolock
- GFS version: 20050506 of RHEL4 branch
- Number of files written: 10000
 
File size | block write speed | fs write speed
----------------------------------------------
10k       | 13-14 MB/s        | 2.2 MB/s
50k       | 18-19 MB/s        | 8.3 MB/s
100k      | 21-22 MB/s        | 13.5 MB/s

- "fs write speed" is the write speed as seen by the application
- "block write speed" is the write speed to the underlying block device, as
reposted by iostat (and verified with the raid controller's monitoring
utility).

So from this it seems that GFS is writing ~50k "extra data" for each file
through the block device. Note: this "extra data" is not taken away from the
free disk space as reposted by df.

Any ideas why this is happening? Any ideas of parameters that could be
tuned?

( ext3 on the same block device does not have this issue.)

Thanks,

- Alexander