[rhelv6-list] RHEL6.2 XFS brutal performence with lots of files

[Pasi Kärkkäinen]

On Fri, Jun 22, 2012 at 07:45:22PM -0500, Daryl Herzmann wrote:
> On Tue, Jun 5, 2012 at 3:10 PM, Jussi Silvennoinen
> <jussi_rhel6 at silvennoinen.net> wrote:
> >> I've been noticing lots of annoying problems with XFS performance with
> >> RHEL6.2 on 64bit.  I typically have 20-30 TB file systems with data
> >> structured in directories based on day of year, product type, for example,
> >>
> >>  /data/2012/06/05/product/blah.gif
> >>
> >> Doing operations like tar or rm over these directories bring the system to
> >> a grinding halt.  Load average goes vertical and eventually the power button
> >> needs to be pressed in many cases :( A hack workaround is to break apart the
> >> task into smaller chunks and let the system breath in between operations...
> >>
> >> Anyway, I read Ric Wheeler's "Billion Files" with great interest
> >>
> >>
> >> http://www.redhat.com/summit/2011/presentations/summit/decoding_the_code/thursday/wheeler_t_0310_billion_files_2011.pdf
> >>
> >> It appears there are 'known issues' with XFS and RHEL6.1.  It does not
> >> appear these issues were addressed in RHEL 6.2?
> >>
> >> Does anybody know if these issues were addressed in the upcoming RHEL 6.3?
> >> My impression is that upstream fixes for this only recently (last 6 months?)
> >> appeared in the mainline kernel.
> >>
> >> Perhaps I am missing some tuning that could be done to help with this?
> >
> >
> > Enabling lazy-count does wonders for workloads that involve massive amounts
> > of metadata. Unfortunately it's a mkfs-time option only AFAIK.
> 
> Thanks, but it was already enabled...
> 

Did you try monitoring the system while the load avg starts increasing? 
What's using all the resources? Is it a memleak somewhere? 

-- Pasi