[Linux-cluster] clvmd without GFS?
Matt Mitchell
mmitchell at virtualproperties.com
Thu Oct 28 19:16:22 UTC 2004
David Teigland wrote:
> Lots of small files can certainly expose some of the performance
> limitations of gfs. "Hours" sounds very odd, though, so I ran a couple
> sanity tests on my own test hardware.
>
> One node mounted with lock_dlm, the directory has 100,000 4k files,
> running "time ls -l | wc -l".
>
> - dual P3 700 MHz, 256 MB, some old FC disks in a JBOD
> 5 min 30 sec
>
> - P4 2.4 GHz, 512 MB, iscsi to a netapp
> 2 min 30 sec
>
> Having more nodes mounted didn't change this. (Four nodes of the first
> kind all running this at the same time averaged about 17 minutes each.)
Here are some more data points from the latest test I have tried. I was
feeling emboldened by the speed of the writing I was seeing, so I tried
loading up a few more files.
The setup: 2 nodes, lock_dlm. Both are P3/866s with 1 GB of RAM apiece.
One of the nodes (hudson) has two CPUs.
The process:
1) Mount disks.
2) Copy image files into subdirectory. Other node is idle.
3)
hudson:/mnt/xs_media# time sh -c 'ls 100032/mls/fmls_stills | wc -l'
298407
real 7m40.726s
user 0m5.541s
sys 1m58.229s
While that's not stunningly great I consider it acceptable performance.
It's a lot of dentries to crawl through.
4) Feeling frisky, I decided to do a "real" test by unmounting and
remounting the FS in order to clear caches, etc. The other host did not
get touched in this interval.
5)
hudson:/mnt/xs_media# time sh -c 'ls 100032/mls/fmls_stills | wc -l'
298407
real 74m43.284s
user 0m5.533s
sys 0m40.146s
Order of magnitude slower.
6) OK, I think, that might be the time to acquire locks the first time.
Doing it again should be fast:
hudson:/mnt/xs_media# time sh -c 'ls 100032/mls/fmls_stills | wc -l'
298407
real 75m29.150s
user 0m5.528s
sys 0m40.724s
7) Ugh. OK, let's try it on the other node:
greenville:/mnt/xs_media# time sh -c 'ls 100032/mls/fmls_stills | wc -l'
298407
real 77m38.569s
user 0m8.850s
sys 0m35.006s
Both systems are sitting there idle now. What did I do by unmounting
and remounting the GFS partition?
For the record that is just over 12GB of data in those 298407 files.
Partition is 3% full (as reported by df).
Help...?
-m
More information about the Linux-cluster
mailing list