[libvirt] [RFC]: Secure migration

[Daniel P. Berrange]

On Tue, Mar 03, 2009 at 01:46:25PM +0100, Chris Lalancette wrote:
> --------------------------------------------------------------------------------
> 1)  virsh on the controller connects to the src, and initiates the migration
> command.  In turn, this causes the controller to also connect to the dst.  Now,
> during the "Prepare" step on the dst, we setup a qemu container to listen to
> some port on localhost.  Next, the "Perform" step is called on
> the src machine.  This causes an external program to be forked on the src; this
> external program connects to the dst (using virConnectOpen), then waits for
> migration data from the qemu instance.  As the migration data comes in, it uses
> a new RPC call (something like "writeMigrationData") to pass the data to the
> dst.  The dst then takes that data, and writes it out to the waiting qemu container.
> 
> Pros: Uses existing RPC mechanism, so you don't have to open a new port on the
> destination
> 
> Cons: There is a "hidden" dependency between the src and dst, which may be
> difficult for users to get right (more on this below).
>
> 2)  virsh on the controller connects to the src, and initiates the migration
> command.  In turn, this causes the controller to also connect to the dst.  Now,
> during the "Prepare" step on the dst, we setup a qemu container to listen to
> some port (call it 1234) on localhost.  It also forks an external program (or a
> thread) to listen for an incoming gnutls connection.  Next, the "Perform" step
> is call on the src machine.  This forks an external program (or thread) to
> listen for incoming data from a localhost migration, do the gnutls handshake
> with the dst, and dump the data over the gnutls connection to the dst.
> 
> Pros: Works mostly transparently; the user doesn't have to set anything up
> different than they do today for "unsecure" migration
> 
> Cons: Requires opening a new (but still well-known) port in the firewall on the
> dst side.
> --------------------------------------------------------------------------------

I'm having a little  trouble understanding the finer differences between these
two architectures, but the way I imagined is like this

   - libvirtd on source has a unix domain / localhost TCP socket for
     QEMU migrations
   - qemu on src does a 'migrate to unix domain socket'
   - libvirtd on source connects to libvirtd on dest using normal 
     RPC protocol (secured with whatever the admin's configured)
   - libvirtd on dest spawns qemu and lets it restore from stdio
     which is fed from the data it gets over the RPC layer


  +---------source-----  ---+                 +------------dest-----------+
  |                         |                 |                           |
   qemu  --------->  libvirtd --------------> libvirtd -------------> qmeu
        UNIX socket           TCP libvirt RPC         -incoming stdio

I don't believe there is need to spawn any external programs here really
(well, perhaps "-incoming cat' for new QEMU lacking the 'stdio' protocol)


> a) libvirtd is using TLS + the "tls_allowed_dn_list" in
> /etc/libvirt/libvirtd.conf on the dst machine.  They have it configured so that
> they can access the machine via TLS on the controller machine, but not from the
> src machine.

That's not a real problem - that's just a configuration setup thing - the
admin already needs to correctly managed this setup for API usage, so its
no harder to get it right for migration

> b) libvirtd is using SASL digest-md5 on the dst machine.  When the src machine
> tries to connect, it needs a name and password to do so, which it doesn't have.
> (technically, I guess we could proxy that response back to the controller, and
> have the user fill it there, and then return back to the src, and then the dst,
> but it seems roundabout)

There could be a dedicated "migration" user account and password configured
in the config for /etc/libvirtd/  that is used for migration only. You don't
particularly want to relay the credentials via the virsh client, because 
that potentially gives them credentials / authorization capabilities you
dont want them to have.

> c) libvirtd is using SASL gssapi on the dst machine.  When the src machine tries
> to connect to the dst, it needs to have the right configuration (i.e.
> /etc/krb5.conf and /etc/sasl2/libvirt.conf need to work), and it also has to get
> some kind of principal from the kerberos server (but which principal?).

Each server has a kerberos principal, so I'd expect they can directly
auth to each other, assuming suitable ACL configs. 

> Because of the hidden dependency problem, I think solution 2) is actually more
> viable; yes, it also has a dependency (opening a hole in the firewall), but that
> can be documented and will work no matter what other authentication you are
> doing between the controller and src and dst machines.  However, I am open to
> being convinced otherwise.  Thoughts?

These are all just minor auth credentials/acl config tasks that the admin 
has to deal with for normal remote usage already, so I don't see that they
present a particular problem for migration

There is also the possibility of plugging in a one-time-key auth credential
for migrations, where the dest passes the neccessary key to the source
libvirtd, via the client app initiating the migration - this is what the
'cookie' parameter in Prepare/Perform was there for.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|