[Linux-cluster] gfs export over nfs is very slow

[Raj Kumar]

Hi,

I found that we are also experiencing performance issues using GFS. Earlier we thought EXT3 and GFS were performing equally so NFS is the issue. But I found that some tests were done incorrectly and EXT3 over NFS is twice faster than GFS over NFS. We formatted a SAN volume as EXT3 and benchmarked it on NFS client. We formatted the same SAN volume as GFS and benchmarked again. GFS + NFS is very slow.

I have also read NFS tuning guides and tried several options. But no change whatsoever.

Are there any other ways to debug this issue. It's a top most priority for us now.

Thanks,
Raj



On Fri, 02 Jun 2006 Olivier Thibault wrote :
>Hi,
>
>I have upgraded FC5, and it's now much better.
>For information, here is a bonnie++ test result, on gfs exported via nfs, gigabit ethernet lan.
>Version 1.01d       ------Sequential Output------ --Sequential Input- --Random-
>                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
>Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
>poisson          4G 21383  19 21582   6  4026  75 24101  21 22974   3 259.8   1
>                     ------Sequential Create------ --------Random Create--------
>                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
>               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP /sec %CP
>                  16   158   1   517  91   248   2   157   1  3243  22  238   2
>
>
>Locally, the same test is more than twice faster.
>Does someone knows if there are optimizations for gfs and nfs, other than ones found in NFS Howto ?
>
>Best regards,
>
>Olivier
>
>Olivier Thibault a écrit :
>>Hi,
>>
>>Raj Kumar a écrit :
>>  > Hi,
>>  >
>>  > We are using GFS6.0 (no cluster suite) and NFS exports of the file system. I am getting a transfer rate of about 35MB/sec. We have a high speed SAN. Actually the transfer rate can be little higher but we attribute the slow rate to NFS itself since we see the same numbers for EXT3 also.
>>  >
>>  > Regards,
>>  > Raj
>>  >
>>  >
>>
>>Thank you for your answer.
>>I am upgrading to last GFS/DLM/CMAN kernel stuff and will retry.
>>I've ran bonnie++ with ext3 exported over nfs and it is really speeder even if it's not what i expected. I got about 22 MB/s (r/w).
>>But i saw that nfsd was consuming a lot of CPU. The system load was 15 !!
>>I've also ran test with Suse SLES9 xfs exported over nfs. I got 40MB/s, which is what aim to get with GFS ...
>>I don't understand ...
>>
>>Is there anybody who export gfs over nfs with FC5 ?
>>
>>Thanks by advance
>>
>>Olivier
>>
>>>On Tue, 30 May 2006 Olivier Thibault wrote :
>>>>Hi,
>>>>
>>>>
>>>>I am testing RHCS on Fedora Core 5.
>>>>I have a shared gfs volume mounted on two nodes (using clvmd and lock_dlm).
>>>>Locally, everything is ok.
>>>>If I export the gfs volume via nfs, i obtain *very poor* performance.
>>>>For exemple, from a nfs client with dd, it take 90 seconds to create a 16 MB file !!!
>>>> From the cluster's nodes, the performances a good, and i made some tests exporting xfs over nfs, and it was good too.
>>>>So what's wrong with nfs+gfs ?
>>>>I would be very interested to know how guys who use this have configured it, and what performances they have.
>>>>
>>>>Thanks for any advices.
>>>>
>>>>Best regards
>>>>
>>>>-- Olivier THIBAULT
>>>>Laboratoire de Mathématiques et Physique Théorique (UMR CNRS 6083)
>>>>Université François Rabelais
>>>>Parc de Grandmont - 37200 TOURS
>>>>Tél: +33 2 47 36 69 12
>>>>Fax: +33 2 47 36 69 56
>>>>
>>>>-- Linux-cluster mailing list
>>>>Linux-cluster at redhat.com
>>>>https://www.redhat.com/mailman/listinfo/linux-cluster
>>>
>>>
>>>------------------------------------------------------------------------
>>>
>>>-- Linux-cluster mailing list
>>>Linux-cluster at redhat.com
>>>https://www.redhat.com/mailman/listinfo/linux-cluster
>>
>
>--
>Linux-cluster mailing list
>Linux-cluster at redhat.com
>https://www.redhat.com/mailman/listinfo/linux-cluster
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20060602/e62eccf1/attachment.htm>