[dm-devel] slab-nomerge (was Re: [git pull] device mapper changes for 4.3)

[Christoph Lameter]

On Sat, 5 Sep 2015, Dave Chinner wrote:

> > Inodes and dentries have constructors. These slabs are not mergeable and
> > will never be because they have cache specific code to be executed on the
> > object.
>
> I also said that the fact that they are not merged is really by
> chance, not by good management. They are not being merged because of
> the constructor, not because they have a shrinker. hell, I even said
> that if it comes down to it, we don't even need SLAB_NO_MERGE
> because we can create dummy constructors to prevent merging....

Right. There is no chance here though. Its intentional to not merge slab
where we could get into issues.

Would be interested to see how performance changes if the inode/dentries
would become mergeable.

> *Some* shrinkers act on mergable slabs because they have no
> constructor. e.g. the xfs_dquot and xfs_buf shrinkers.  I want to
> keep them separate just like the inode cache is kept separate
> because they have workload based demand peaks in the millions of
> objects and LRU based shrinker reclaim, just like inode caches do.

But then we are not sure why we would do that. Certainly merging can
increases the stress on the per node locks for a slab cache as the example
by Jesper shows (and this can be dealt with by increasing per cpu
resources). On the other hand this also leads to rapid defragmentation
because the free objects from partial pages produced by the frees of
one of the merged slabs can get reused quickly for another purpose.

> I really don't see the issue here - explicitly encoding and
> documenting the behaviour we've implicitly been relying on for years
> is something we do all the time. Code clarity and documented
> behaviour is a *good thing*.

The question first has to be answered why keeping them separate is such a
good thing without also having an explicit way of telling the allocator to
keep certain objects in the same slab page if possible. Otherwise we get
this randomizing effect that nullifies the idea that sequential
freeing/allocation would avoid fragmentation.

I have in the past be in favor of adding such a flag to avoid merging but
I am slowly getting to the point that this may not be wise anymore. There
is too much arguing from gut reactions here and relying on assumptions
about internal operations of slabs (thinking to be able to exploit the
fact that linearly allocated objects come from the same slab page coming
from you is one of these).

Defragmentation IMHO requires a targeted approach were either objects that
are in the way can be moved out of the way or there is some type of
lifetime marker on objects that allows the memory allocators to know that
these objects can be freed all at once when a certain operation is
complete.