[Linux-cluster] Why GFS is so slow? What it is waiting for?

[Ja S]

Hi, Andrew:

Thank you very much for the help. Yes, your
explanation really makes sense. I buy it.

But I would like to discuss it a little bit further.
The following message was part of my previous reply to
Wendy. Just paste it here for your convenience. 


# stat abc/
  File: `abc/'
  Size: 8192            Blocks: 6024       IO Block:
4096   directory
Device: fc00h/64512d    Inode: 1065226     Links: 2
Access: (0770/drwxrwx---)  Uid: (    0/    root)  
Gid: (    0/    root)
Access: 2008-05-08 06:18:58.000000000 +0000
Modify: 2008-04-15 03:02:24.000000000 +0000
Change: 2008-04-15 07:11:52.000000000 +0000

# cd abc/
# time ls | wc -l 
31764

real    0m44.797s
user    0m0.189s
sys     0m2.276s


>From the test results, it seems that the system really
only used 2.276 seconds to perform the disk IO, read
the directory and count the number of files.

I am not sure whether I missed anything or not. I
really cannot understand how the system took about 42
seconds to process the lock on the single directory.

Any further comments? 

Thanks again in advance,

Jas


--- "Andrew A. Neuschwander" <andrew at ntsg.umt.edu>
wrote:

> I've looked at this problem a bit as well. My system
> is a 4Gb FC SAN with
> a bonded GigE DLM dedicated network. Stat'ing 30,000
> files in 3 minutes on
> GFS isn't unreasonable considering that it must get
> and release the gfs
> locks. In this scenario, you are averaging about 6ms
> per file stat. When
> we did our tests, all of our subsystems (FC, Net,
> CPU, Memory, Disk) were
> near idle. I think the 6ms is simply the accumulated
> latency of all the
> subsystems involved. There is a lot of work
> happening in that short period
> of time.
> 
> -A
> -- 
> Andrew A. Neuschwander, RHCE
> Linux Systems/Software Engineer
> College of Forestry and Conservation
> The University of Montana
> http://www.ntsg.umt.edu
> andrew at ntsg.umt.edu - 406.243.6310
> 
> 
> On Thu, May 8, 2008 4:29 pm, Bob Peterson wrote:
> > On Thu, 2008-05-08 at 14:27 -0700, Ja S wrote:
> >> Hi, All:
> >>
> >> I used to post this question before, but have not
> >> received any comments yet. Please allow me post
> it
> >> again.
> >>
> >> I have a subdirectory containing more than 30,000
> >> small files on a SAN storage (GFS1+DLM, RAID10).
> No
> >> user application knows the existence of the
> >> subdirectory. In other words, the subdirectory is
> free
> >> of accessing.
> >>
> >> However, it took ages to list the subdirectory on
> an
> >> absolute idle cluster node. See below:
> >>
> >> # time ls -la | wc -l
> >> 31767
> >>
> >> real    3m5.249s
> >> user    0m0.628s
> >> sys     0m5.137s
> >>
> >> There are about 3 minutes spent on somewhere.
> Does
> >> anyone have any clue what the system was waiting
> for?
> >>
> >>
> >> Thanks for your time and wish to see your
> valuable
> >> comments soon.
> >>
> >> Jas
> >
> > Hi Jas,
> >
> > I believe the answer to your question is in the
> FAQ:
> >
> >
>
http://sources.redhat.com/cluster/wiki/FAQ/GFS#gfs_slow
> >
> > Regards,
> >
> > Bob Peterson
> > Red Hat Clustering & GFS
> >
> >
> > --
> > Linux-cluster mailing list
> > Linux-cluster at redhat.com
> >
>
https://www.redhat.com/mailman/listinfo/linux-cluster
> >
> >
> 
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
>
https://www.redhat.com/mailman/listinfo/linux-cluster
> 



      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ