[Libvir] RFC [0/3]: Re-factor QEMU daemon protocol to use XDR

[Daniel P. Berrange]

I've been thinking about a way to move forward on the remote management
capabilities in libvirt since our last list discussion didn't come to any
clear agreement.

The current situation

  - QEMUD daemon uses a hand-crafted protocol basically just dumping
    structs on the wire. Key points:
        - Fairly simple to understand code for marshalling/demarshalling
        - Very low overhead on wire
        - Not safe endianess
        - Not safe wrt to architecture data alignment rules
        - No authentication/encryption
        - Server driver specific to QEMU

  - The generic libvirt daemon uses SunRPC as the underlying protocol.
        - Protocol versioning 
        - Safe wrt to endianess
        - Safe wrt to architecture data alignment rules
        - Added TLS encryption / SSH tunneling
        - Generic for any libvirt backend
        - Doesn't work with non-blocking sockets due to SunRPC limitations
        - SunRPC api is not well known / 'wierd'
        - Call + reply only; no async signals from server

Looking at the current impl of the generic libvirt daemon there are some
things which strike me:

  1. We have an extra layer of marshalling/demarshalling calls. ie the methods
     in the libvirt driver, call to the SunRPC marshalling methods, which
     populate an XDR generated struct, which call the XDR routines to write
     the data.
  2. The main loop doesn't allow us to listen for other non-SunRPC file
     handles (eg the QEMU/dnsmasq processes).
  3. We've basically re-written the the SunRPC svc*_create methods to
     get support for TLS / SSH

We are going to need to address 2 by creating our own mainloop impl. For
point 1 we can have the libvirt driver populate the XDR struct directly
and eliminate the SunRPC generated marshalling stubs completely.  If we
did that, and given points 2 & 3, we're basically left using very little
of the SunRPC capabilities at all. Pretty much only thing it is doing for
us at that point is the protocol versioning, and giving us a horrific API
which doesn't support non-blocking IO so forces us to be multi-threads or
multi-process.

So, I'm thinking why don't we just use XDR directly & be done with it. As an
experiment I took the existing QEMU daemon & driver and re-factored the
hand written binary protocol to use XDR instead. 

Looking at the diffstat - there was basically no appreciable difference
in codesize after the re-factoring, and we get the added benefit of being
endianness & alignment safe.

 qemud/Makefile.am   |   15 +
 qemud/conf.c        |    1 
 qemud/dispatch.c    |  538 ++++++++++++++++------------------------------
 qemud/dispatch.h    |    2 
 qemud/internal.h    |   21 +
 qemud/protocol.h    |  329 ----------------------------
 qemud/protocol.x    |  608 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemud/qemud.c       |  189 +++++++++++-----
 qemud/uuid.c        |    1 
 src/Makefile.am     |   11 
 src/internal.h      |    2 
 src/qemu_internal.c |  593 ++++++++++++++++++++++++++------------------------
 12 files changed, 1288 insertions(+), 1022 deletions(-)

The particular changes I'll explain in the next 3 mails where I attach the
patches.

Meanwhile though, I'm thinking we could take approach of incremental code
refactoring, pulling in all the various capabilities from the remote patch
(except for the SunRPC demarshalling itself) to the QEMU daemon. At each 
stage having a working daemon & client but with ever growing capabilities,
until it provides full remote management. 

  1. Add an extra initial message type to do a major/minor version number
     exchange. Based on the negotiated versions, the server can activate
     or deactivate particular requests it allows from the client.

  2. Add in the URI parsing routines to the qemud_interal.c driver, but
     only allow use of QEMU initially.

  3. Add in the TLS + SSH  client & server socket setup code to give an
     authenticated & secured remote channel

  4. Re-factor qemud/dispatch.c so that it calls into the regular libvirt
     API for the Xen case, but keeps calling qemud/driver.c for the QEMU
     case.


BTW, these patches are not intended to be applied to CVS - they're for 
discussion purposes at this stage. That said, they are fully functional
so you can apply them to your checkout & you should be able run all the
existing virsh commands against the libvirt_qemud. Just make sure you
patch & rebuild both the client & server ;-)

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|