[dm-devel] New dm-bufio with shrinker API

[Joe Thornber]

On Mon, Sep 05, 2011 at 12:01:28PM -0400, Mikulas Patocka wrote:
> "cache_size" is the value that you set as a maximum cache size. The 
> default is 2% of memory or 25% of vmalloc area.

ah, could you rename this variable to 'max_allocated' then please, to
match with the 'total_allocated' field (which I presume gives the
current cache size?).

> > With the old block manager the test suite ran nicely with
> > less than 256k, from memory I think I started seeing slow down around
> > 128k.  With bufio I'm seeing consistently larger cache sizes for the
> > same performance.
> 
> So, reduce cache_size to 256k (or whatever value you want to test) and see 
> how it performs.

But then I'm limited to 256k, my point is we want scaling _and_ to use
less memory.  We cannot tell our users to experiment to find the right
setting for this depending on the number of pools they're running and
the usage of each pool.

> > For instance the test_overwriting_various_thin_devices scenario from
> > here
> > (https://github.com/jthornber/thinp-test-suite/blob/master/basic_tests.rb)
> > has a peak use of ~1100k, if I change from using dt with random io
> > pattern to plain old dd then the usage drops to ~900k. Setting the
> > max_age parameter to 1 second had very little effect.
> 
> Reduce cache_size and try it.

Here are the numbers (best of 3 runs):

| Test                        | 256k cache (M/s) | 2M cache (M/s) |
| unprovisioned thin          |             74.4 |             75 |
| provisioned thin            |             72.8 |           72.6 |
| new snap (complete sharing) |             73.7 |           73.8 |
| old snap (no sharing)       |             72.2 |           72.8 |

So I think that proves my point.  We're getting no benefit from that
extra memory, is there a subsystem that could be making better use of
it? (eg, page cache?).  Or are you telling me that nobody else would
have been using that memory?

(This is all just tweaking, bufio is working very well).

- Joe