[RFC] Seeking advice on support for generic vDPA device

[Jason Wang]

On Sat, Mar 25, 2023 at 9:14 PM jiangdongxu <jiangdongxu1 at huawei.com> wrote:
>
> vDPA devices allow high-performance devices in a virtual machine by
> providing a wire-speed data path. These devices require a vendor-specific host
> driver but the data path follows the virtio specification.
>
> The support for generic-vdpa device was recently added to qemu in v8.0.0-rc version.
> https://gitlab.com/qemu-project/qemu/-/commit/b430a2bd2303b9940cc80ec746887f463a25fc5c
>
> This allows libvirt to support these deviecs regardless of device type.
> Currently, libvirt supports vDPA network devices, We need a new solution
> to support generic-vdpa devices.
>
> At present, we have considered several options.
>
> Solution A:
>     Add type vdpa under the hostdev device，the device xml looks like
>     <devices>
>         <hostdev mode='subsystem' type='vdpa'>
>             <source dev='/dev/vhost-vdpa-0'/>
>         </hostdev>
>     </devices>
>     This solution is recommended. This configuration requires only that the
>     device supports the vDPA framework, easy for future expansion and same
>     as the qemu solution design idea.

I prefer this one, it's easier to implement and works for any type of
virtio devices and vDPA parents

>
> Solution B:
>     The current libvirt mode is used to add vDPA devices under different devices.
>     device xml looks like:
>     network:
>     <devices>
>       <interface type='generic-vdpa'>
>         <source dev='/dev/vhost-vdpa-0'/>
>       </interface>
>     </devices>
>
>     storage:
>     <devices>
>         <disk type='generic-vdpa’>
>           <source dev='/dev/vhost-vdpa-0'/>
>         </disk>
>     </devices>
>
>     The network interface has the vdpa type. Therefore, a new type is required to
>     distinguish the "generic-vdpa" type. However, this method differentiates
>     device types, which is not convenient for the expansion of other types
>     of vDPA devices. Not recommended.

This should be the way when we have dedicated block layer support for
vhost-vDPA. Which seems not necessary for general vDPA devices.

>
> Solution C:
>     Referring to PCI passthrough, Add the driver type (appropriate vendor-specific driver) to the driver of the PCI device.
>     The unbind/bind of the vDPA device driver and the creation of VDPA device are implemented by libvirt.
>     device xml looks like:
>     <devices>
>         <hostdev mode='subsystem' type='pci' managed='yes'>
>             <driver name='vp-vdpa'/>
>             <source>
>                 <address domain='0x0000' bus='0x04' slot='0x01' function='0x01'/>
>             </source>
>         </hostdev>
>     </devices>
>     This solution is related to the vendor-specific driver and supports only the vdpa created by the PCI device.
>     vdpa devices created in vduse mode are not supported. This solution is not universal and is not recommended.

I agree, this requires PCI specific knowledge which defeats the goal
of vDPA to be device agnostic.

Thanks

>
>
> What's your proposal?