[et-mgmt-tools] Supporting SuSE paravirt installs with virt-inst/virt-manager

[Daniel P. Berrange]

The current virt-install code for provisioning paravirtualized guests
was originally written to work with Fedora / RHEL derived distros.
Basically, given an NFS/HTTP/FTP install tree URL it will:

  1. Fetch $INSTALL_URL/images/xen/vmlinux, save it to /var/lib/xen/images
  2. Fetch $INSTALL_URL/images/xen/initrd, save it to /var/lib/xen/images
  3. Boot guest pointing to these explicit kernel & ramdisk  files

This takes the user straight into anaconda.  All very cool for Fedora / RHEL
or CentOS users. Not so handy if you're trying to install paravirtualized
SuSE or Debian images. Debian appears to be focused on a chroot based
bootstrap approach for their guest installs - eg the install of the guest
FS is done entirely from Dom0, only booting a guest once install is complete.
This is not a model I want to go for in virt-install/virt-manager[1]. The
good news though, is that distros in the SuSE family do support booting the 
regular YAST installer in a DomU, so I'm very interested in getting this 
working for virt-intsall.

I found the following page describing the manual steps required to kick
off a guest install

   http://www.suse.de/~kraxel/xen/suse-guest.html

And have been trying to do a proof-of-concept automation of them in
python. I've attached my code to this mail, but basically is goes along
like this:

  1. Fetch $INSTALL_URL/ls-lR.gz, save it to a temp file
  2. Unzip it & scan the files looking for the latest version of the
     'kernel-xen' and  'install-initrd' RPMs with matching architecture
  3. Fetch $INSTALL_URL/$KERNEL_RPM saving it to a temp file
  4. Fetch $INSTALL_URL/$INSTALL_INITRD_RPM saving it to a temp file
  5. Use rpm2cpio & cpio to extract the /boot/vmlinuz-XXX file from
     the kernel RPM
  6. Use rpm2cpio & cpio to extract the entire of $INSTALL_INITRD_RPM
  7. Run the /usr/sbin/mkinstallinitrd script just extracted and 
     point it to the $KERNEL_RPM file saved earlier
  8. Boot guest pointing to the initrd from step 7 and kernel from
     step 5.

So it is definitely possible to kick off paravirtualized SuSE installs,
although the procss is rather long & convoluted - and the downloads
take a non-trivial amount of time (20 MB for kernel RPM & 6MB for the
install_initrd RPM). My main concern is that we're executing shell/perl
scripts from the install_initrd RPM which we've downloaded from an
arbitrary source :-(  One option would be to not run the script in step
7 and install write equivalent code to build an initrd directly in the
virtinst library. That would avoid the worry of running arbitrary scripts
but may not be reliable long term if SuSE change the requirements for
what goes in their initrds.

It'd be really nice if SuSE just provided pre-built vmlinux & initrd files
for installation as they already do with baremetal - but I've not found
any such files on the various download sites for SuSE I've looked at. Any
one have any ideas for making this process better ?

Regards,
Dan.

[1] It requires root on Dom0 to do the chroot stuff - so guest install can
    in theory compromise the entire host & all guests within. Similarly
    you can't control resource usage & thus install will impact performance
    of all other guests; Finally, it has requirements for the bootstrap
    tools of each distro to be installed in the Dom0
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 
-------------- next part --------------
#!/usr/bin/python


import urlgrabber.grabber as grabber
import urlgrabber.progress as progress
import gzip
import tempfile
import os
import re

baseurl = "http://ftp.opensuse.org/pub/opensuse/distribution/10.2/repo/oss/"
filelist="ls-lR.gz"

def _copy_temp(fileobj, prefix, scratchdir):
    (fd, fn) = tempfile.mkstemp(prefix="virtinst-" + prefix, dir=scratchdir)
    block_size = 16384
    try:
        while 1:
            buff = fileobj.read(block_size)
            if not buff:
                break
            os.write(fd, buff)
    finally:
        os.close(fd)
    return fn

def fetch(url, prefix):
    data = grabber.urlopen(url,
                           progress_obj = progress.TextMeter())

    return _copy_temp(data, prefix, "/var/tmp")


install_initrd = None
kernel = None
kernelname = "kernel-xen"

print "Fetching " + baseurl + filelist
tmpfn = fetch(baseurl + filelist, "filelist")
print
try:
    filelistData = gzip.GzipFile(tmpfn, mode = "r")

    arch = os.uname()[4]
    arches = [arch]
    if arch == "i686":
        arches.append("i586")
        arches.append("i386")
        kernelname = "kernel-xenpae"

    dir = None
    while 1:
        data = filelistData.readline()
        if not data:
            break
        if dir is None:
            for arch in arches:
                wantdir = "/suse/" + arch
                if data == "." + wantdir + ":\n":
                    dir = wantdir
                    break
        else:
            if data == "\n":
                dir = None
            else:
                if data[:5] != "total":
                    filename = re.split("\s+", data)[8]

                    if filename[:14] == "install-initrd":
                        install_initrd = dir + "/" + filename
                    elif filename[:len(kernelname)] == kernelname:
                        kernel = dir + "/" + filename
finally:
    os.unlink(tmpfn)

kernelfn = None
mkinitrdfn = None

try:
    print "Fetching " + baseurl + kernel
    kernelfn = fetch(baseurl + kernel, "kernel")
    print

    print "Fetching " + baseurl + install_initrd
    install_initrdfn = fetch(baseurl + install_initrd, "install-initrd")
    print

    scratchfn = tempfile.mkdtemp(prefix="virtinst-scratch", dir="/var/tmp")
    (initrdfd, initrdfn) = tempfile.mkstemp(prefix="virtinst-initrd", dir="/var/tmp")
    os.close(initrdfd)

    os.system("cd " + scratchfn + " && " + \
              "(rpm2cpio " + install_initrdfn + " | " + \
              " cpio --quiet --extract --unconditional --make-directories)")
    os.system("cd " + scratchfn + " && " + \
              "(rpm2cpio " + kernelfn + " | " + \
              " cpio --quiet --extract --unconditional --make-directories '*vmlinuz*')")
    os.system("/usr/bin/perl " + scratchfn + "/usr/sbin/mkinstallinitrd " + \
              "--libdir=" + scratchfn + "/usr/lib/install-initrd " + \
              "--kernel-rpm=" + kernelfn + " " + \
              initrdfn)

    
finally:
    pass
    #if kernelfn is not None:
    #    os.unlink(kernelfn)
    #if initrdfn is not None:
    #    os.unlink(initrdfn)