[dm-devel] dm-userspace (no in-kernel cache version)

[Dan Smith]

FT> As explained, this removes rmap (in-kernel cache) and use mmaped
FT> buffer instead of read/write system calls for user/kernel
FT> communication.

Ok, I got your code to work, and I have run some benchmarks.  I'll cut
directly to the chase...

I used dbench with a single process, for 120 seconds on a dm-userspace
device mapping directly to an LVM device.  I used my example.c and the
example-rb.c provided with the ringbuffer version.  The results are:

  with cache, chardev:  251 MB/s
  no cache, ringbuffer: 243 MB/s
 
I am very pleased with these results.  I assume that your code is not
tuned for performance yet, which means we should be able to squeeze at
least 8 MB/s more out to make it equal (or better).  Even still, the
amount of code it saves is worth the hit, IMHO.

I do have a couple of comments:

1. You said that the ringbuffer saves the need for syscalls on each
   batch read.  This is partially true, but you still use a write() to
   signal completion so that the kernel will read the u->k ringbuffer.
   So, at best, the number of syscalls made is half of my
   read()/write() method.  I think it's possible that another
   signaling mechanism could be used, which would eliminate this call.
   I do think eliminating the copying with the ringbuffer approach is
   very nice; I like it a lot.

2. I was unable to get your code to perform well with multiple threads
   of dbench.  While my code sustains performance with 16 threads, the
   non-cache/ringbuffer version slows to a crawl (~1MB/s with 16
   procs).  I noticed that the request list grows to over a 100,000
   entries at times, which means that the response from userspace
   requires searching that linearly, which may be the issue.

I am going to further study your changes, but I think in the end that
I will incorporate most or all of them.  Some work will need to be
done to incorporate support for some of the newer features (endio, for
example), but I'll start looking into that.

Thanks a lot!

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms at us.ibm.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20060912/9955f351/attachment.sig>