[Linux-cluster] More GFS2 tuning...

[Corey Kovacs]

On Mon, Feb 16, 2009 at 7:26 PM, Jeff Sturm <jeff.sturm at eprize.com> wrote:
>> -----Original Message-----
>> From: linux-cluster-bounces at redhat.com
>> [mailto:linux-cluster-bounces at redhat.com] On Behalf Of Corey Kovacs
>> Sent: Monday, February 16, 2009 1:55 PM
>> To: linux-cluster at redhat.com
>> Subject: [Linux-cluster] More GFS2 tuning...
>>
>> By my reckoning, I should be able to see 400MB or more
>> sustained throughput using this setup. If this is a pipe
>> dream, someone let me know quick before I go nutz.
>
> What do you get from the raw device?  (I.e. if you remove GFS/NFS from
> the picture.)

Haven't tried this yet, will do it in the morning.

>
>> The bo values start at around 200MB, then drop down to 0 in
>> most cases for a few seconds, then spike to ~700MB/s then
>> eases back down to 200, 150 and back down to 0. It looks very
>> much like a cacheing issue to me.
>
> Linux virtual memory does some funny things with fs caching.  Try some
> tests with O_DIRECT to bypass the buffer cache.  On RHEL 5 systems, you
> can achieve that with "dd ... oflag=direct" and varying block sizes.
>
>> I've read that GFS2 is supposed to be "self tuning" but I
>> don't think these are necessarily GFS2 issues.

This is a new one for me, I'll try this in the morning.


> Agreed.  If you can experiment with the hardware, what do you get from
> other fs types?  (such as ext3)
>

These tests are on the roadmap as well.


>> Anyone have something similar? What I/O rates are people getting?
>
> I don't have any FC hardware quite as nice as yours, but multipathing
> AoE over a pair of GigE connections we can get 200MB/s raw, sequential
> throughput.  (I.e. about the limits of the interconnects.)
>
> My GFS filesystems are mostly a collection of very small (~1MB or less)
> files, so, it's hard to say how they are performing.  I'm much more
> concerned about the rate of file creates over GFS than raw throughput
> right now...
>
> Jeff
>
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
>

Thanks


-Corey