[Linux-cluster] clvmd without GFS?

[Matt Mitchell]

Cahill, Ben M wrote:
>  
> 
> 
>>-----Original Message-----
>>From: linux-cluster-bounces at redhat.com 
>>[mailto:linux-cluster-bounces at redhat.com] On Behalf Of Matt Mitchell
>>Sent: Thursday, October 28, 2004 3:16 PM
>>To: David Teigland
>>Cc: linux-cluster at redhat.com
>>Subject: Re: [Linux-cluster] clvmd without GFS?
>>
> 
> 
>>7) Ugh.  OK, let's try it on the other node:
>>
>>greenville:/mnt/xs_media# time sh -c 'ls 
>>100032/mls/fmls_stills | wc -l'
>>298407
>>
>>real    77m38.569s
>>user    0m8.850s
>>sys     0m35.006s
>>
>>
>>Both systems are sitting there idle now.  What did I do by 
>>unmounting and remounting the GFS partition?
>>
>>For the record that is just over 12GB of data in those 298407 files. 
>>Partition is 3% full (as reported by df).
>>
>>Help...?
> 
> 
> I'd be curious to know if it makes a difference if you mount using -o
> noatime option (see man mount)?  The default access-time update
> threshold for GFS is 3600 seconds (1 hour).  This can cause a bunch of
> write transactions to happen, even if you're just doing a read operation
> such as ls.  Since your exercise is taking over an hour, this might be
> thrashing the atime updates, but I don't know how much that might be
> adding.

Looks like that is the case for the second time slowness; it also 
probably explains why the directory seemed relatively snappy immediately 
after I finished populating it but not at all the next morning.

hudson:/mnt/xs_media# mount
[ snipped ]
/dev/mapper/xs_gfsvg-xs_media on /mnt/xs_media type gfs (rw,noatime)
hudson:/mnt/xs_media# time sh -c 'ls 100032/mls/fmls_stills | wc -l'
298407

real    74m15.781s
user    0m5.546s
sys     0m40.529s
hudson:/mnt/xs_media# time sh -c 'ls 100032/mls/fmls_stills | wc -l'
298407

real    3m37.052s
user    0m5.502s
sys     0m12.643s

For sake of comparison, here is the same thing after unmounting both 
nodes and remounting only hudson, again with noatime (so it is not 
touching any disk blocks):
hudson:/mnt/xs_media# time sh -c 'ls 100032/mls/fmls_stills | wc -l'
298407

real    3m59.533s
user    0m5.501s
sys     0m51.741s

(Now I am trying to unmount the partition again, and it's hanging.)

So it is definitely dlm behaving badly...?

-m