[dm-devel] slab-nomerge (was Re: [git pull] device mapper changes for 4.3)

[Dave Chinner]

On Thu, Sep 03, 2015 at 09:09:24AM +0300, Pekka Enberg wrote:
> Hi Andrew,
> 
> On Wed, 2 Sep 2015 22:10:12 -0500 (CDT) Christoph Lameter <cl at linux.com> wrote:
> >> > But I'd still like some pointers/help on what makes slab merging so
> >> > beneficial.  I'm sure Christoph and others have justification.  But if
> >> > not then yes the default to slab merging probably should be revisited.
> >>
> >> ...
> >>
> >> Check out the linux-mm archives for these dissussions.
> 
> On Thu, Sep 3, 2015 at 7:55 AM, Andrew Morton <akpm at linux-foundation.org> wrote:
> > Somewhat OT, but...  The question Mike asks should be comprehensively
> > answered right there in the switch-to-merging patch's changelog.
> >
> > The fact that it is not answered in the appropriate place and that
> > we're reduced to vaguely waving at the list archives is a fail.  And a
> > lesson!
> 
> Slab merging is a technique to reduce memory footprint and memory
> fragmentation. Joonsoo reports 3% slab memory reduction after boot
> when he added the feature to SLAB:

I'm not sure whether you are trying to indicate that it was
justified inteh commit message or indicate how little justification
there was...

> commit 12220dea07f1ac6ac717707104773d771c3f3077
> Author: Joonsoo Kim <iamjoonsoo.kim at lge.com>
> Date:   Thu Oct 9 15:26:24 2014 -0700
> 
>     mm/slab: support slab merge
> 
>     Slab merge is good feature to reduce fragmentation.  If new creating slab
>     have similar size and property with exsitent slab, this feature reuse it
>     rather than creating new one.  As a result, objects are packed into fewer
>     slabs so that fragmentation is reduced.

A partial page or two in a newly allocated slab in not
"fragmentation". They are simply free objects in the cache that
haven't been allocated yet. Fragmentation occurs when large numbers
of objects are freed so the pages end up mostly empty but
cannot be freed because there is still 1 or 2 objects in use of
them. As such, if there was fragementation and slab merging fixed
it, I'd expect to be seeing a much larger reduction in memory
usage....

>     Below is result of my testing.
> 
>     * After boot, sleep 20; cat /proc/meminfo | grep Slab
> 
>     <Before>
>     Slab: 25136 kB
> 
>     <After>
>     Slab: 24364 kB
> 
>     We can save 3% memory used by slab.

The numbers don't support the conclusion. Memory used from boot to
boot always varies by a small amount - a slight difference in the
number of files accessed by the boot process can account for this.
Also, you can't 't measure slab fragmentation by measuring the
amount of memory used. You have to look at object counts in each
slab and work out the percentage of free vs allocated objects. So
there's no evidence that this 772kb difference in memory footprint
can even be attributed to slab merging.

What about the rest of the slab fragmentation problem space?  It's
not even mentioned in the commit, but that's really what is
important to long running machines.

IOWs, where's description of the problem that needs fixing? What's
the example workload that demonstrates the problem? What's the
before and after measurements of the workloads that generate
significant slab fragmentation?  What's the long term impact of the
change (e.g.  a busy server with a uptime of several weeks)? is the
fragmentation level reduced?  increased? not significant?  What
impact does this have on subsystems with shrinkers that are now
operating on shared slabs? Do the shrinkers still work as
effectively as they used to?  Do they now cause slab fragmentation,
and if they do, does it self correct under continued memory
pressure?

And with the patch being merged without a single reviewed-by or
acked-by, I'm sitting here wondering how we managed to fail software
engineering 101 so badly here?

Cheers,

Dave.
-- 
Dave Chinner
dchinner at redhat.com