[Linux-cluster] GFS slower than NFS ???

Wendy Cheng wcheng at redhat.com
Tue Jul 10 02:12:55 UTC 2007


R Wahyudi wrote:

> Before : We have 2 mail storage system which is shared using NFS over 
> 100MB Ethernet.
> 50% of user data divided equally on each storage server, and each 
> storage server NFS-mount the other
> storage server so that it can provide  100% of data.
> A number of SMTP,POP, and IMAP servers mounting the 2 storage servers 
> using NFS.
>
> After:
> We consolidated the storage server using  HP StorageWorks8100 EVA,
> and we have 2 POP/IMAP server which mount the disk from the 
> StorageWorks via 2GB Fiber - iSCSI.
> These 2 server are GFS clustered.
>
> To my disappointment's, the "After" setup was slower than the before.
> Doing "ls -lah" on a directory with 300+ files take an average of 25 
> seconds,
> while it took less than 1 second on previous setup.


The "ls -la" command is known to be a performance killer for cluster 
filesystems like GFS. It is not an GFS specific issues (a google search 
for POSIX "statlite" and "readdirplus" should give you plenty of 
examples). In general, we would like to

1. Caution users whether "ls -la" is really a good performance indicator 
for their applications.
2. Avoid having one gigantic directory holding many many small files. 
Re-structuring them into different sub-directories should see sizable 
performance improvement.

-- Wendy




More information about the Linux-cluster mailing list