[libvirt] [RFC]: Secure migration

[Chris Lalancette]

Daniel P. Berrange wrote:
> On Tue, Mar 03, 2009 at 01:46:25PM +0100, Chris Lalancette wrote:
>> --------------------------------------------------------------------------------
>> 1)  virsh on the controller connects to the src, and initiates the migration
>> command.  In turn, this causes the controller to also connect to the dst.  Now,
>> during the "Prepare" step on the dst, we setup a qemu container to listen to
>> some port on localhost.  Next, the "Perform" step is called on
>> the src machine.  This causes an external program to be forked on the src; this
>> external program connects to the dst (using virConnectOpen), then waits for
>> migration data from the qemu instance.  As the migration data comes in, it uses
>> a new RPC call (something like "writeMigrationData") to pass the data to the
>> dst.  The dst then takes that data, and writes it out to the waiting qemu container.
>>
>> Pros: Uses existing RPC mechanism, so you don't have to open a new port on the
>> destination
>>
>> Cons: There is a "hidden" dependency between the src and dst, which may be
>> difficult for users to get right (more on this below).
>>
>> 2)  virsh on the controller connects to the src, and initiates the migration
>> command.  In turn, this causes the controller to also connect to the dst.  Now,
>> during the "Prepare" step on the dst, we setup a qemu container to listen to
>> some port (call it 1234) on localhost.  It also forks an external program (or a
>> thread) to listen for an incoming gnutls connection.  Next, the "Perform" step
>> is call on the src machine.  This forks an external program (or thread) to
>> listen for incoming data from a localhost migration, do the gnutls handshake
>> with the dst, and dump the data over the gnutls connection to the dst.
>>
>> Pros: Works mostly transparently; the user doesn't have to set anything up
>> different than they do today for "unsecure" migration
>>
>> Cons: Requires opening a new (but still well-known) port in the firewall on the
>> dst side.
>> --------------------------------------------------------------------------------
> 
> I'm having a little  trouble understanding the finer differences between these
> two architectures, but the way I imagined is like this
> 
>    - libvirtd on source has a unix domain / localhost TCP socket for
>      QEMU migrations
>    - qemu on src does a 'migrate to unix domain socket'
>    - libvirtd on source connects to libvirtd on dest using normal 
>      RPC protocol (secured with whatever the admin's configured)
>    - libvirtd on dest spawns qemu and lets it restore from stdio
>      which is fed from the data it gets over the RPC layer
> 
> 
>   +---------source-----  ---+                 +------------dest-----------+
>   |                         |                 |                           |
>    qemu  --------->  libvirtd --------------> libvirtd -------------> qmeu
>         UNIX socket           TCP libvirt RPC         -incoming stdio
> 
> I don't believe there is need to spawn any external programs here really
> (well, perhaps "-incoming cat' for new QEMU lacking the 'stdio' protocol)

Yes, this setup is what my option 1 above is.  Option 2 is very similar, except
we use a separate channel not hooked up with the normal RPC mechanism to
transfer the data.

>> a) libvirtd is using TLS + the "tls_allowed_dn_list" in
>> /etc/libvirt/libvirtd.conf on the dst machine.  They have it configured so that
>> they can access the machine via TLS on the controller machine, but not from the
>> src machine.
> 
> That's not a real problem - that's just a configuration setup thing - the
> admin already needs to correctly managed this setup for API usage, so its
> no harder to get it right for migration
> 
>> b) libvirtd is using SASL digest-md5 on the dst machine.  When the src machine
>> tries to connect, it needs a name and password to do so, which it doesn't have.
>> (technically, I guess we could proxy that response back to the controller, and
>> have the user fill it there, and then return back to the src, and then the dst,
>> but it seems roundabout)
> 
> There could be a dedicated "migration" user account and password configured
> in the config for /etc/libvirtd/  that is used for migration only. You don't
> particularly want to relay the credentials via the virsh client, because 
> that potentially gives them credentials / authorization capabilities you
> dont want them to have.
> 
>> c) libvirtd is using SASL gssapi on the dst machine.  When the src machine tries
>> to connect to the dst, it needs to have the right configuration (i.e.
>> /etc/krb5.conf and /etc/sasl2/libvirt.conf need to work), and it also has to get
>> some kind of principal from the kerberos server (but which principal?).
> 
> Each server has a kerberos principal, so I'd expect they can directly
> auth to each other, assuming suitable ACL configs. 
> 
>> Because of the hidden dependency problem, I think solution 2) is actually more
>> viable; yes, it also has a dependency (opening a hole in the firewall), but that
>> can be documented and will work no matter what other authentication you are
>> doing between the controller and src and dst machines.  However, I am open to
>> being convinced otherwise.  Thoughts?
> 
> These are all just minor auth credentials/acl config tasks that the admin 
> has to deal with for normal remote usage already, so I don't see that they
> present a particular problem for migration

Yes, they are certainly all solvable from the admin's point-of-view, so they are
not show stoppers.  The thing is that I think admins will have a difficult time
discovering what the problems are when migration doesn't work for them.  There
are just so many combinations that it's very easy for the admin to get one of
them wrong, and then it may be difficult to figure out exactly what they need to
do to get it working.  On the other hand, having a dedicated channel makes it
relatively easy; if the admin is having problems, then the answer is going to be
"open port XYZ on the destination", and that will usually solve the problem.

-- 
Chris Lalancette