[dm-devel] slab-nomerge (was Re: [git pull] device mapper changes for 4.3)

[Christoph Lameter]

On Thu, 3 Sep 2015, Jesper Dangaard Brouer wrote:

> > IOWs, slab merging prevents us from implementing effective active
> > fragmentation management algorithms and hence prevents us  from
> > reducing slab fragmentation via improved shrinker reclaim
> > algorithms.  Simply put: slab merging reduces the effectiveness of
> > shrinker based slab reclaim.
>
> I'm buying into the problem of variable object lifetime sharing the
> same slub.

Well yeah I see the logic of the argument but what I have seen in practice
is that the access to objects becomes rather random over time. inodes and
denties are used by multiple underlying volumes/mountpoints etc. They are
expired individually etc etc. The references to objects become garbled
over time anyways.

What I would be interested in is some means by which locality of objects
of different caches can be explicitly specified. This would allow the
placing together of multiple objects in the same page frame. F.e. dentries
and inodes and other metadata of a filesystem that is related. This would
enhance the locality of the data and allow better defragmentation. But we
are talking here about a totally different allocator design.

> With the SLAB bulk free API I'm introducing, we can speedup slub
> slowpath, by free several objects with a single cmpxchg_double, BUT
> these objects need to belong to the same page.
>  Thus, as Dave describe with merging, other users of the same size
> objects might end up holding onto objects scattered across several
> pages, which gives the bulk free less opportunities.

This happens regardless as far as I can tell. On boot up you may end up
for a time in special situations where that is true.

> That would be a technical argument for introducing a SLAB_NO_MERGE flag
> per slab.  But I want to do some measurement before making any
> decision. And it might be hard to show for my use-case of SKB free,
> because SKB allocs will likely be dominating 256 bytes slab anyhow.

With the skbs you would want to place the skb data together with the
packet data and other network related objects right? Maybe we can think
out an allocator that can store objects related to a specific action in a
page frame that can then be tossed as a whole.