[dm-devel] slab-nomerge (was Re: [git pull] device mapper changes for 4.3)

[Jesper Dangaard Brouer]

On Thu, 3 Sep 2015 16:02:47 +1000 Dave Chinner <dchinner at redhat.com> wrote:

> On Wed, Sep 02, 2015 at 06:21:02PM -0700, Linus Torvalds wrote:
> > On Wed, Sep 2, 2015 at 5:51 PM, Mike Snitzer <snitzer at redhat.com> wrote:
> > >
> > > What I made possible with SLAB_NO_MERGE is for each subsystem to decide
> > > if they would prefer to not allow slab merging.
> > 
> > .. and why is that a choice that even makes sense at that level?
> > 
> > Seriously.
> > 
> > THAT is the fundamental issue here.
> 
> It makes a lot more sense than you think, Linus.
> 
[...]
> 
> On the surface, this looks like a big win but it's not - it's
> actually a major problem for slab reclaim and it manifests when
> there are large bursts of allocation activity followed by sudden
> reclaim activity.  When the slab grows rapidly, we get the majority
> of objects on a page being of one type, but a couple will be of a
> different type. Than under memory pressure, the shrinker can then
> only free the majority of objects on a page, guaranteeing the slab
> will remain fragmented under memory pressure.  Continuing to run the
> shrinker won't result in any more memory being freed from the merged
> slab and so we are stuck with unfixable slab fragmentation.
> 
> However, if the slab with a shrinker only contains one kind of
> object, when it becomes fragmented due to variable object lifetime,
> continued memory pressure will cause it to keep shrinking and hence
> will eventually correct the fragmentation problem. This is a much
> more robust configuration - the system will self correct without
> user intervention being necessary.
> 
> IOWs, slab merging prevents us from implementing effective active
> fragmentation management algorithms and hence prevents us  from
> reducing slab fragmentation via improved shrinker reclaim
> algorithms.  Simply put: slab merging reduces the effectiveness of
> shrinker based slab reclaim.

I'm buying into the problem of variable object lifetime sharing the
same slub.

With the SLAB bulk free API I'm introducing, we can speedup slub
slowpath, by free several objects with a single cmpxchg_double, BUT
these objects need to belong to the same page.
 Thus, as Dave describe with merging, other users of the same size
objects might end up holding onto objects scattered across several
pages, which gives the bulk free less opportunities.

That would be a technical argument for introducing a SLAB_NO_MERGE flag
per slab.  But I want to do some measurement before making any
decision. And it might be hard to show for my use-case of SKB free,
because SKB allocs will likely be dominating 256 bytes slab anyhow.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer