[libvirt] Supporting vhost-net and macvtap in libvirt for QEMU

[Anthony Liguori]

Disclaimer: I am neither an SR-IOV nor a vhost-net expert, but I've CC'd 
people that are who can throw tomatoes at me for getting bits wrong :-)

I wanted to start a discussion about supporting vhost-net in libvirt.  
vhost-net has not yet been merged into qemu but I expect it will be soon 
so it's a good time to start this discussion.

There are two modes worth supporting for vhost-net in libvirt.  The 
first mode is where vhost-net backs to a tun/tap device.  This is 
behaves in very much the same way that -net tap behaves in qemu today.  
Basically, the difference is that the virtio backend is in the kernel 
instead of in qemu so there should be some performance improvement.

Current, libvirt invokes qemu with -net tap,fd=X where X is an already 
open fd to a tun/tap device.  I suspect that after we merge vhost-net, 
libvirt could support vhost-net in this mode by just doing -net 
vhost,fd=X.  I think the only real question for libvirt is whether to 
provide a user visible switch to use vhost or to just always use vhost 
when it's available and it makes sense.  Personally, I think the later 
makes sense.

The more interesting invocation of vhost-net though is one where the 
vhost-net device backs directly to a physical network card.  In this 
mode, vhost should get considerably better performance than the current 
implementation.  I don't know the syntax yet, but I think it's 
reasonable to assume that it will look something like -net 
tap,dev=eth0.   The effect will be that eth0 is dedicated to the guest.

On most modern systems, there is a small number of network devices so 
this model is not all that useful except when dealing with SR-IOV 
adapters.  In that case, each physical device can be exposed as many 
virtual devices (VFs).  There are a few restrictions here though.  The 
biggest is that currently, you can only change the number of VFs by 
reloading a kernel module so it's really a parameter that must be set at 
startup time.

I think there are a few ways libvirt could support vhost-net in this 
second mode.  The simplest would be to introduce a new tag similar to 
<source network='br0'>.  In fact, if you probed the device type for the 
network parameter, you could probably do something like <source 
network='eth0'> and have it Just Work.

Another model would be to have libvirt see an SR-IOV adapter as a 
network pool whereas it handled all of the VF management.  Considering 
how inflexible SR-IOV is today, I'm not sure whether this is the best model.

Has anyone put any more thought into this problem or how this should be 
modeled in libvirt?  Michael, could you share your current thinking for 
-net syntax?

-- 
Regards,

Anthony Liguori