[Libvir] Tricky provisioning problem with inactive domains

[Karel Zak]

On Mon, Jan 15, 2007 at 09:46:37AM -0500, Daniel Veillard wrote:
> On Mon, Jan 15, 2007 at 02:14:36PM +0000, Daniel P. Berrange wrote:
> > Adding support for inactive domains was supposed to make everyone's life 
> > easier, but as luck would have it, its actually made one thing very much
> > harder. In the virt-inst/virt-manager tools provisioning works something
> > like this:
> > 
> > In paravirt case:
> > 
> >  - Create a guest using an explicit kernel/initrd from the images/xen
> >    directory on the install CD
> >  - Write a config file to /etc/xen setup to boot using pygrub
> > 
> > In fullvirt case:
> > 
> >  - Create a guest booting directly off a CDROM
> >  - Write a config file to /etc/xen setup to boot off the harddisk
> > 
> > So in both these cases, the libvirt XML config for the very first boot of
> > the guest is different, from the XML config for subsequent  boots. With 
> > the new inactive domain support in libvirt & xend, we can't write out 
> > config files directly, instead there is the virDomainDefine() API, which
> > calls to appropriate APIs in XenD. And this is where the problem arises:
> > 
> >  1. If we call virDomainDefine() to write the long term config, then 
> >     virDomainStart() will not be using the correct boot method for the
> >     initial install.
> >  2. If we call virDomainDefine() to write the initial install config,
> >     then virDomainStart() will kick off the install correctly, but on
> >     subsequent boots we'll end up booting the installer again instead
> >     of the just installed OS.
> >  3. We could just use virDomainCreate() to start installer, and try to
> >     use virDomainDefine() to write the long term config - the latter
> >     call will fail though because there will already be a running guest
> >     with that name.
> >  4. Conversely if we use virDomainDefine() to write the config, and then
> >     tried to create a one-off guest with virDomainCreate() the latter
> >     will fail due to duplicate names.
> > 
> > So, thus far the only way out of the trap I can think of is:
> > 
> >  1. Use virDomainCreate() to kick off the initial install
> >  2. Poll virDomainLookupByXXX() to watch for this initial guest shutting
> >     down
> >  3. Write out the persistent config using virDomainDefine()
> > 
> > The big problem with this, is that if the user were to exit virt-manager
> > sometime after the guest install starts, but before step 3, the config
> > for the guest will never be written, even though it has successfully
> > installed.
> > 
> > There are two further ideas I've had - both requiring additional APIs in
> > libvirt & probably XenD too.
> > 
> >  - Make it possible change the boot configuration of an existing guest.
> >    This would let us do:
> > 
> >       1. Use virDomainDefine() to define a config file suitable for installing
> >          the guest, ie using explicit kernel/initrd
> >       2. Use virDomainStart() to kick off the installer
> >       3. Uew new  API to change the guest config to remove explicit kernel
> >          & initrd config, and add a bootloader for pygrub. Or in HVM case,
> >          switch boot order to use harddisk instead of CDROM & detach the
> >          CDROM device.
> > 
> >  - Make it possible to start an existing inactive guest using an alternative
> >    one-off configuration. This would let us do:
> > 
> >        1. Use virDomainDefine() to define a config file suitable for running
> >           the guest during normal operation.
> >        2. Use virDomainStartConfig(xml) to start the guest with a special
> >           config suitable for installing the guest.
> > 
> > 
> > Ultimately I think we do need the means to change arbitrary parts of a guest
> > configuration, so I think the first option would be the preferred approach.
> > The trouble is that I think implementing this would require using the new 
> > XenAPI, or adding a number of new methods to the legacy SXPR API which is not
> > really very desirable.
> 
>   Well the core of the problem is that there is 2 set of information, distinct
> in their value and meaning and we have only one placeholder for both. It seems
> to me that basically we are overloading the <os> block while we would need
> a separate <install> block containing the informations needed to install the
> domain.
>   Why not plit them out, and keep both in the configuration ? It seems it would
> open the door to:
>     - the user option of restarting the installation of a domain, which
>       with virtualization is certainly a more common operation than 
>       reinstalling a bare system
>     - allow to keep track of how and where a given domain was installed
>       and with what data
>     - ease provisioning of a similar domain based on the informations
>       provided in that <install> block

 Yes. The idea with <install> and <os> is ***very good***.

>   Basically we could create the domain configuration with both the <os>
> and <install> block, the install one for example missing an installation
> date, suggesting the user to run the installation process instead of
> the normal boot of the instance.
>   I'm not even sure libvirt would have to be modified for this, it will by
> default ignore the <install> block, and the application on top of it could

 I think libvirt would have to be modified for this -- at least a
 separate API which is able to loads data from the <install> block and
 returns it to libvirt based application. I'd like to use

    virt-install --config /etc/xen/myvm.xml

 when I need to re-install the domain.

> just modify the XML before calling Create() . Making libvirt install aware
> would force an API extenstion though, I'm not 100% sure that's really needed.

 It's cheap add there this API extenstion for <install> and it's
 without an impact to the old libvirt API. Our life will be more funny
 with this API ;-)

>   Did I understood correctly the problem ? I think adding extra informations
> about the install data is in any case a good idea, especialy when one think
> in terms of provisionning.

 Yes.

    Karel

-- 
 Karel Zak  <kzak at redhat.com>