[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] Why GFS is so slow? What it is waiting for?

Ja S wrote:
Hi, All:

I used to post this question before, but have not
received any comments yet. Please allow me post it

I have a subdirectory containing more than 30,000
small files on a SAN storage (GFS1+DLM, RAID10). No
user application knows the existence of the
subdirectory. In other words, the subdirectory is free
of accessing.
Short answer is to remember "ls" and "ls -la" are very different commands. "ls" is a directory read (that reads from one single file) but "ls -la" needs to get file attributes (file size, modification times, ownership, etc) from *each* of the files from the subject directory. In your case, it needs to read more than 30,000 inodes to get them. The "ls -la" is slower for *any* filesystem but particularly troublesome for a cluster filesystem such as GFS due to:

1. Cluster locking overheads (it needs readlocks from *each* of the files involved). 2. Depending on when and how these files are created. During file creation time and if there are lock contentions, GFS has a tendency to spread the file locations all over the disk. 3. You use iscsi such that dlm lock traffic and file block access are on the same fabric ? If this is true, you will more or less serialize the lock access.

Hope above short answer will ease your confusion.

-- Wendy
However, it took ages to list the subdirectory on an
absolute idle cluster node. See below:

# time ls -la | wc -l

real    3m5.249s
user    0m0.628s
sys     0m5.137s

There are about 3 minutes spent on somewhere. Does
anyone have any clue what the system was waiting for?

Thanks for your time and wish to see your valuable
comments soon.


Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Linux-cluster mailing list
Linux-cluster redhat com

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]