[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Re: [Linux-cluster] gfs export over nfs is very slow


I found that we are also experiencing performance issues using GFS. Earlier we thought EXT3 and GFS were performing equally so NFS is the issue. But I found that some tests were done incorrectly and EXT3 over NFS is twice faster than GFS over NFS. We formatted a SAN volume as EXT3 and benchmarked it on NFS client. We formatted the same SAN volume as GFS and benchmarked again. GFS + NFS is very slow.

I have also read NFS tuning guides and tried several options. But no change whatsoever.

Are there any other ways to debug this issue. It's a top most priority for us now.


On Fri, 02 Jun 2006 Olivier Thibault wrote :
>I have upgraded FC5, and it's now much better.
>For information, here is a bonnie++ test result, on gfs exported via nfs, gigabit ethernet lan.
>Version 1.01d      ------Sequential Output------ --Sequential Input- --Random-
>                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
>Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
>poisson          4G 21383  19 21582  6  4026  75 24101  21 22974  3 259.8  1
>                    ------Sequential Create------ --------Random Create--------
>                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
>              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP /sec %CP
>                  16  158  1  517  91  248  2  157  1  3243  22  238  2
>Locally, the same test is more than twice faster.
>Does someone knows if there are optimizations for gfs and nfs, other than ones found in NFS Howto ?
>Best regards,
>Olivier Thibault a écrit :
>>Raj Kumar a écrit :
>>  > Hi,
>>  >
>>  > We are using GFS6.0 (no cluster suite) and NFS exports of the file system. I am getting a transfer rate of about 35MB/sec. We have a high speed SAN. Actually the transfer rate can be little higher but we attribute the slow rate to NFS itself since we see the same numbers for EXT3 also.
>>  >
>>  > Regards,
>>  > Raj
>>  >
>>  >
>>Thank you for your answer.
>>I am upgrading to last GFS/DLM/CMAN kernel stuff and will retry.
>>I've ran bonnie++ with ext3 exported over nfs and it is really speeder even if it's not what i expected. I got about 22 MB/s (r/w).
>>But i saw that nfsd was consuming a lot of CPU. The system load was 15 !!
>>I've also ran test with Suse SLES9 xfs exported over nfs. I got 40MB/s, which is what aim to get with GFS ...
>>I don't understand ...
>>Is there anybody who export gfs over nfs with FC5 ?
>>Thanks by advance
>>>On Tue, 30 May 2006 Olivier Thibault wrote :
>>>>I am testing RHCS on Fedora Core 5.
>>>>I have a shared gfs volume mounted on two nodes (using clvmd and lock_dlm).
>>>>Locally, everything is ok.
>>>>If I export the gfs volume via nfs, i obtain *very poor* performance.
>>>>For exemple, from a nfs client with dd, it take 90 seconds to create a 16 MB file !!!
>>>> From the cluster's nodes, the performances a good, and i made some tests exporting xfs over nfs, and it was good too.
>>>>So what's wrong with nfs+gfs ?
>>>>I would be very interested to know how guys who use this have configured it, and what performances they have.
>>>>Thanks for any advices.
>>>>Best regards
>>>>-- Olivier THIBAULT
>>>>Laboratoire de Mathématiques et Physique Théorique (UMR CNRS 6083)
>>>>Université François Rabelais
>>>>Parc de Grandmont - 37200 TOURS
>>>>Tél: +33 2 47 36 69 12
>>>>Fax: +33 2 47 36 69 56
>>>>-- Linux-cluster mailing list
>>>>Linux-cluster redhat com
>>>-- Linux-cluster mailing list
>>>Linux-cluster redhat com
>Linux-cluster mailing list
>Linux-cluster redhat com

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]