[Libvir] Tricky provisioning problem with inactive domains

[Daniel P. Berrange]

Adding support for inactive domains was supposed to make everyone's life 
easier, but as luck would have it, its actually made one thing very much
harder. In the virt-inst/virt-manager tools provisioning works something
like this:

In paravirt case:

 - Create a guest using an explicit kernel/initrd from the images/xen
   directory on the install CD
 - Write a config file to /etc/xen setup to boot using pygrub

In fullvirt case:

 - Create a guest booting directly off a CDROM
 - Write a config file to /etc/xen setup to boot off the harddisk

So in both these cases, the libvirt XML config for the very first boot of
the guest is different, from the XML config for subsequent  boots. With 
the new inactive domain support in libvirt & xend, we can't write out 
config files directly, instead there is the virDomainDefine() API, which
calls to appropriate APIs in XenD. And this is where the problem arises:

 1. If we call virDomainDefine() to write the long term config, then 
    virDomainStart() will not be using the correct boot method for the
    initial install.
 2. If we call virDomainDefine() to write the initial install config,
    then virDomainStart() will kick off the install correctly, but on
    subsequent boots we'll end up booting the installer again instead
    of the just installed OS.
 3. We could just use virDomainCreate() to start installer, and try to
    use virDomainDefine() to write the long term config - the latter
    call will fail though because there will already be a running guest
    with that name.
 4. Conversely if we use virDomainDefine() to write the config, and then
    tried to create a one-off guest with virDomainCreate() the latter
    will fail due to duplicate names.

So, thus far the only way out of the trap I can think of is:

 1. Use virDomainCreate() to kick off the initial install
 2. Poll virDomainLookupByXXX() to watch for this initial guest shutting
    down
 3. Write out the persistent config using virDomainDefine()

The big problem with this, is that if the user were to exit virt-manager
sometime after the guest install starts, but before step 3, the config
for the guest will never be written, even though it has successfully
installed.

There are two further ideas I've had - both requiring additional APIs in
libvirt & probably XenD too.

 - Make it possible change the boot configuration of an existing guest.
   This would let us do:

      1. Use virDomainDefine() to define a config file suitable for installing
         the guest, ie using explicit kernel/initrd
      2. Use virDomainStart() to kick off the installer
      3. Uew new  API to change the guest config to remove explicit kernel
         & initrd config, and add a bootloader for pygrub. Or in HVM case,
         switch boot order to use harddisk instead of CDROM & detach the
         CDROM device.

 - Make it possible to start an existing inactive guest using an alternative
   one-off configuration. This would let us do:

       1. Use virDomainDefine() to define a config file suitable for running
          the guest during normal operation.
       2. Use virDomainStartConfig(xml) to start the guest with a special
          config suitable for installing the guest.


Ultimately I think we do need the means to change arbitrary parts of a guest
configuration, so I think the first option would be the preferred approach.
The trouble is that I think implementing this would require using the new 
XenAPI, or adding a number of new methods to the legacy SXPR API which is not
really very desirable.

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|