[Linux-cluster] Why GFS is so slow? What it is waiting for?

[Andrew A. Neuschwander]

I've looked at this problem a bit as well. My system is a 4Gb FC SAN with
a bonded GigE DLM dedicated network. Stat'ing 30,000 files in 3 minutes on
GFS isn't unreasonable considering that it must get and release the gfs
locks. In this scenario, you are averaging about 6ms per file stat. When
we did our tests, all of our subsystems (FC, Net, CPU, Memory, Disk) were
near idle. I think the 6ms is simply the accumulated latency of all the
subsystems involved. There is a lot of work happening in that short period
of time.

-A
-- 
Andrew A. Neuschwander, RHCE
Linux Systems/Software Engineer
College of Forestry and Conservation
The University of Montana
http://www.ntsg.umt.edu
andrew at ntsg.umt.edu - 406.243.6310


On Thu, May 8, 2008 4:29 pm, Bob Peterson wrote:
> On Thu, 2008-05-08 at 14:27 -0700, Ja S wrote:
>> Hi, All:
>>
>> I used to post this question before, but have not
>> received any comments yet. Please allow me post it
>> again.
>>
>> I have a subdirectory containing more than 30,000
>> small files on a SAN storage (GFS1+DLM, RAID10). No
>> user application knows the existence of the
>> subdirectory. In other words, the subdirectory is free
>> of accessing.
>>
>> However, it took ages to list the subdirectory on an
>> absolute idle cluster node. See below:
>>
>> # time ls -la | wc -l
>> 31767
>>
>> real    3m5.249s
>> user    0m0.628s
>> sys     0m5.137s
>>
>> There are about 3 minutes spent on somewhere. Does
>> anyone have any clue what the system was waiting for?
>>
>>
>> Thanks for your time and wish to see your valuable
>> comments soon.
>>
>> Jas
>
> Hi Jas,
>
> I believe the answer to your question is in the FAQ:
>
> http://sources.redhat.com/cluster/wiki/FAQ/GFS#gfs_slow
>
> Regards,
>
> Bob Peterson
> Red Hat Clustering & GFS
>
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
>
>