[Libguestfs] Re: libguestfs performance

[Pierre Riteau]

On 7 déc. 2009, at 10:57, Richard W.M. Jones wrote:

> On Tue, Dec 01, 2009 at 04:08:38PM +0100, Pierre Riteau wrote:
>> For a project of mine I would like to create VMs from tarballs of
>> root filesystems. I would like to use libguestfs since this is
>> exactly what the tar2vm.sh guestfish recipe does.
>> 
>> However, I need to extract tarballs that are in the hundreds of
>> megabytes, but my tests showed that performance is very low, both
>> with tgz-in and with guestmount (sorry I don't have the numbers any
>> more, but it was something like more than one hour to extract a
>> tarball containing a Debian Lenny installation).
>> 
>> Is there any way to drastically improve libguestfs' performance?
> 
> First of all, read:
> 
> http://libguestfs.org/FAQ.html#slow
> 
> Using tar-in/tgz-in to copy the tarball into the appliance is always
> going to be slow.  I would think you should get better performance if
> you write the tarball into an ISO first, and attach that.  Something
> like:
> 
> mkisofs -JR -o cd.iso the_tarball.tar.gz
> 
> guestfish -a guest.img -a cd.iso
> [...]
>> <fs> mkmountpoint /guest
>> <fs> mkmountpoint /cd
> [...]
>> <fs> mount /dev/sdb /cd
> [...]
>> <fs> debug sh "tar xcf /cd/the_tarball.tar.gz /guest"

I got it to work using the following sequence:

$ guestfish -a guest.img -a cd.iso
run
sfdiskM /dev/sda ","
mkfs ext3 /dev/sda1
mkmountpoint /guest
mkmountpoint /cd
mount /dev/sda1 /guest
mount /dev/sdb /cd
debug sh "tar -C /sysroot/guest -xzf /sysroot/cd/the_tarball.tar.gz"

This is much better (on my test system it writes at about 15 MB/s).
However, it is still far from what the hardware can do (more than 60 MB/s).

Now the last problem is that I need to install a bootloader. Can I easily install grub in the appliance, or do I need to recreate a new one?

> However we found experimentally that the fastest method of all for
> creating distributions was to prebuild an ext2/3/4 image, and 'dd' it
> over to the target partition, then resize it to the final (larger)
> size.  This method is extremely quick, enabling us to build Fedora
> images in just a minute or two:
> 
> http://rwmj.wordpress.com/2009/11/23/prebuilt-distributions-part-2/#content

I've read your blog, this is interesting. I will see how I can leverage that.

-- 
Pierre Riteau -- http://perso.univ-rennes1.fr/pierre.riteau/