consuming pre-created tap - with multiqueue

[Daniel P. Berrangé]

On Tue, Sep 22, 2020 at 01:48:08PM +0200, Miguel Duarte de Mora Barroso wrote:
> Hello,
> 
> On KubeVirt, we are trying to pre-create a tap device, then instruct
> libvirt to consume it (via the type=ethernet , managed='no'
> attributes).
> 
> It works as expected, **unless** when we create a multi-queue tap device.
> 
> The difference when creating the tap device is that we set the
> multi-queue flag; libvirt throws the following error when consuming
> it:
> 
> ```
> LibvirtError(Code=38, Domain=0, Message='Unable to create tap device
> tap0: Invalid argument')
> ```
> 
> After digging a bit on the libvirt code (we're using libvirt 6.0.0), I
> see this on the logs (immediately before the error):
> {"component":"virt-launcher","level":"info","msg":"Enabling
> IFF_VNET_HDR","pos":"virNetDevProbeVnetHdr:190","subcomponent":"libvirt","thread":"33"
> ,"timestamp":"2020-09-22T10:34:29.335000Z"}
> 
> I do not understand how it can try to set the VNET_HDR flag, since I
> have not set it when I created it, which, as per [0] should only
> happen when requested. Here's the tap device I'm creating: (output of
> `ip tuntap show`)
>   - tap0: tap persist0x100 user 107 group 107

IIUC the kernel code correctly, the VNET_HDR flag is not required
to be set when you first create the tap device - it appears to be
permitted any time you open a file descriptor for it.

AFAIK, there is no problem with VNET_HDR, as it is a standard flag
we've set on all tap devices on Linux for 10 years.

> I'm confused by this, since on [1] it says that only one flag can be
> used (persist *or* vnet_hdr). Also, from my limited understanding, in
> order to open a pre-created tap device, it must be set to persistent.

That comment you're pointing to is saying that the method only
has one named flag defined in the enum. It should have been updated
when we added the second PERSIST flag. They are not mutually exclusive
from either libvirt or kernel POV AFAICT.

> Am I right (in the sense that these flags cannot co-exist) ? If so,
> why would libvirt be setting the VNET_HDR flag on the tap device ?
> 
> [0] - https://github.com/libvirt/libvirt/blob/v6.0.0/src/util/virnetdevtap.c#L261
> [1] - https://github.com/libvirt/libvirt/blob/v6.0.0/src/util/virnetdevtap.c#L209


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|