[Linux-cluster] which is better gfs2 and ocfs2?

Michael Lackner michael.lackner at unileoben.ac.at
Wed Mar 9 14:53:40 UTC 2011

I guess not all usage scenarios are comparable, but I once
tried to use GFS2 as well as OCFS2 to share a FC SAN to three
nodes using 8GBit FC and 1GBit Ethernet for the cluster
communication. Additionally, i compared it to a trial version
of Dataplows SAN File System (SFS). I was also supposed to
compare it to Quantum StorNext, but there just wasn't enough
time for that.

OS was CentOS 5.3 at that time.

So I tried a lot of performance tuning settings for all three,
and it was like this:

1.) SFS was the fastest, but caused reproducible kernel panics.
Those were fixed by Dataplow, but then SFS produced corrupted data
when writing large files. Unusable in that state, so we gave up.
SFS uses NFS for lock management. Noteworthy: Writing data on the
machine with the NFS lock manager also crippled the I/O performance
for all the other nodes in a VERY, VERY bad way..

2.) GFS2 was the slowest, and despite all the tunings I tried, it
never came close to anything that any local FS would provide in
terms of speed (compared to EXT3 and XFS). The statfs() calls
pretty much crippled the FS. Multiple I/O streams on multiple nodes:
Not a good idea it seems..  Sometimes you have to wait for minutes
for the FS to just give you any feedback, when you're hammering
it with let's say 30 sequential write streams across 3 nodes, with
the streams equally distributed among them.

3.) OCFS2 was slightly faster than GFS2, especially when it came
to statfs(), like ls -l. It did not slow down that much. But overall,
it was still just far too slow.

Our solution: Hook up the SAN on one node only, and share via NFS
over GBit Ethernet. Overall, we are getting better results even
with the obvious network overhead, especially when doing a lot of
I/O on multiple clients.

Our original goal was to provide a high-speed centralized storage
solution for multiple nodes without having to use ethernet. This
failed completely unfortunately.

Hope this helps, it's just my experience though. As usual, mileage
may vary...

yue wrote:
> which is better gfs2 and ocfs2?
> i want to share fc-san, do you know which is better?
> stablility,performmance?

Michael Lackner
Lehrstuhl für Informationstechnologie, Montanuniversität Leoben
IT Administration
michael.lackner at mu-leoben.at | +43 (0)3842/402-1505

More information about the Linux-cluster mailing list