[libvirt] libvirt mdev migration, mdevctl integration

[Daniel P. Berrangé]

On Tue, Nov 26, 2019 at 11:50:05AM +0100, Cornelia Huck wrote:
> On Mon, 25 Nov 2019 17:47:26 +0000
> Daniel P. Berrangé <berrange at redhat.com> wrote:
> 
> > On Mon, Nov 25, 2019 at 06:14:33PM +0100, Cornelia Huck wrote:
> > > On Mon, 18 Nov 2019 19:00:25 +0000
> > > Daniel P. Berrangé <berrange at redhat.com> wrote:
> > >   
> > > > On Mon, Nov 18, 2019 at 10:06:34AM -0700, Alex Williamson wrote:  
> 
> > > > > An issue here though is that the json will also include the parent
> > > > > device, which we obviously cannot assume is the same (particularly the
> > > > > bus address) on the migration target.  We can allow commandline
> > > > > overrides for the parent just as we do above for the UUID when defining
> > > > > the mdev device from json, but it's an open issue who is going to be
> > > > > smart enough (perhaps dumb enough) to claim this responsibility.  It
> > > > > would be interesting to understand how libvirt handles other host
> > > > > specific information during migration, for instance if node or processor
> > > > > affinities are part of the VM XML, how is that translated to the
> > > > > target?  I could imagine that we could introduce a simple "first
> > > > > available" placement in mdevctl, but maybe there should minimally be a
> > > > > node allocation preference with optional enforcement (similar to
> > > > > numactl), or maybe something above libvirt needs to take this
> > > > > responsibility to prepare the target before we get ourselves into
> > > > > trouble.    
> > > > 
> > > > I don't think we need to solve placement in libvirt.
> > > > 
> > > > The guest XML will just reference the mdev via a UUID that
> > > > was used with virNodeDeviceDefineXML. 
> > > > 
> > > > The virNodeDeviceDefineXML call where the mdev is first defined
> > > > will set the details of the mdev creation for this specific host.
> > > > The XML used with virNodeDeviceDefineXML can be different on the
> > > > source + target hosts. As long as the UUID is the same in both
> > > > hosts, the VM will associate with it correctly.  
> > > 
> > > I wonder how to sync up with different placements, but maybe I'm just
> > > missing something.
> > > 
> > > Looking at this from the vfio-ccw angle, we can easily have the same
> > > device (as identified by the device number) on different subchannels
> > > (parents). To find out the device number, you need to look at the child
> > > ccw device of the subchannel while it is *not* bound to vfio-ccw, but
> > > to the normal I/O subchannel driver instead. Or ask your admin for the
> > > system definition...  
> > 
> > This just means that whoever/whatever is invoking "virDomainDeviceDefinXML"
> > or "mdevctl create" will pass different parameters on each host. When
> > migrating a guest the mgmt app can indicate which device should be used
> > for the guest on each host. This is similar issue to migrating a guest
> > which uses a ethNNN device that's got different name on each host ,or
> > a /dev/sdNNN that's different on each host, etc
> 
> Ok, so the burden will be on a management layer resp. the admin to make
> sure that the correct device is in place, even if it resides in
> different places in the topology? Makes sense, I guess.

At least for the initial implementation this gives us something clear to
aim for, and enables mgmt apps to do pretty much anything they should
require, albeit at a slightly greater burden for the app dev in the short
term.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|