[Libguestfs] Improving supermin appliance startup time (lkvm/qboot)

[Török Edwin]

Hi,

I remembered reading about Intel Clear Containers [1], and Hyper/Qboot [2]
that support startup times measured in the hundreds of miliseconds range.

On an AMD FX(tm)-8350 'guestfish -a /dev/null run' takes ~4s when run the 2nd/3rd time:
real	0m4.152s
user	0m2.120s
sys	0m0.564s

Are there any plans on applying these improvements to the supermin appliance?

I did some quick tests on what would be possible, see below:

If I try to use virtio 9P instead it becomes a little faster (instead of root fs on block device):
real	0m3.299s
user	0m2.084s
sys	0m1.828s

To do this we need to load just the virtio and 9p modules:
let kmods = [
"virtio*.ko*";
"9p*.ko*";
]
Modify init.c to mount the 9p root: https://gist.github.com/anonymous/cb32986fd9b85c315ae4
And wrap qemu to create the 9p device (the mount is just there for a quick test, in reality supermin itself should probably leave the unpacker dir around):
https://gist.github.com/anonymous/5cdbc18974f88d6ea9e0

Next step: kvmtool [3]. Avoids some delays introduced by legacy BIOS boot, and device probing, however
lkvm currently lacks support for additional virtio-serial channels, so - although the supermin appliance would  boot - guestfsd wouldn't be able to communicate with the host:
https://gist.github.com/anonymous/e6b0a12e1da811f24e5b

I tried QBoot [4], which requires that you first strip down your kernel+initrd to fit into 8128k,
and uses an unmodified Qemu otherwise.
But then I haven't been able to get it to actually launch guestfsd, it hangs somewhere earlier:
https://gist.github.com/anonymous/52fc2890a0884230e4f8
Maybe I removed too many modules here?

Another possibility would be to run guestfsd with rkt+lkvm stage1, or hyper.sh; but I don't see a way
to forward a unix socket or a virtio-serial channel through them, would guestfsd be able to work over a TCP socket
or the main virtio console? (its probably preferable to avoid having any networking in the guestfsd appliance)

[1] https://lwn.net/Articles/644675/
[2] https://hyper.sh/faq.html
[3] https://git.kernel.org/cgit/linux/kernel/git/will/kvmtool.git/tree/README
[4] https://github.com/bonzini/qboot

Best regards,
-- 
Edwin Török | Co-founder and Lead Developer

Skylable open-source object storage: reliable, fast, secure
http://www.skylable.com