[Linux-cluster] gfs tuning
td3201 at gmail.com
Mon Jun 16 19:38:55 UTC 2008
On Mon, Jun 16, 2008 at 2:16 PM, Ross Vandegrift <ross at kallisti.us> wrote:
> On Mon, Jun 16, 2008 at 11:45:51AM -0500, Terry wrote:
>> I have 4 GFS volumes, each 4 TB. I am seeing pretty high load
>> averages on the host that is serving these volumes out via NFS. I
>> notice that gfs_scand, dlm_recv, and dlm_scand are running with high
>> CPU%. I truly believe the box is I/O bound due to high awaits but
>> trying to dig into root cause. 99% of the activity on these volumes
>> is write. The number of files is around 15 million per TB. Given
>> the high number of writes, increasing scand_secs will not help. Any
>> other optimizations I can do?
> Are you running multi-threaded/multi-process writes to the same files
> on various nodes?
> During benchmarking and testing a cluster I recently built, I noticed
> a very large performance hit when performing multi-threaded I/O to
> overlapping areas of the filesystem.
> If you can randomize the order that different nodes are accessing
> the filesystem, you'll go a long way to reducing contention. That
> will improve your performance.
> However, I suspect with NFS you won't have too much choice, since
> file access will be governed by client read/write patterns...
> Ross Vandegrift
> ross at kallisti.us
I won't have a choice, unfortunately. Here is what I set so far:
gfs_tool settune $i statfs_slots 128
gfs_tool settune $i scand_secs 30
gfs_tool settune $i glock_purge 50
More information about the Linux-cluster