[libvirt] support for configuring all ports of a multiport SRIOV VF when assigning to guest

[Laine Stump]

On 08/05/2017 01:14 PM, Laine Stump wrote:
> On 08/03/2017 02:33 AM, Moshe Levi wrote:
>> Hi Laine,
>>
>> I have a few question before I can give my opinion.
>> I the Mellanox Card Dual Port that support one PCI with 2 PF is  ConnectX-3 and ConnectX-3 Pro. (maybe others cards  I will check this)
>> The ConnectX-4 Dual Port and above is implemented with 2 PCI devices per 2 PF.
> So is the "multiple netdevs for a single PCI device" hardware model
> completely deprecated, and will never show up again in new products?
>
> If that's the case, maybe I shouldn't burden libvirt's config with all
> this new config that will only be used for legacy hardware. Perhaps a
> better approach would be to stick with the current config, and make it
> work properly for VFIO device assignment when a ConnectX-3 card is
> configured in single port mode - even that doesn't work correctly now[*]
> but it's a more easily solved problem, and can be done with no config
> changes.
>
> Opinions about this? If it's a deadend and the existing legacy hardware
> can be used in a reasonable manner by setting it to single port mode, I
> don't want to add externally-visible knobs to libvirt.
>
>
> [*]If a VF is configured to be "port 2 only", libvirt would still try to
> get/set the MAC and vlan tag with a netlink message to the *port 1*
> netdev of the PF, so it would be saving/setting the wrong (nonexistent?)
> VF netdev.

I just tried this and find I was partially mistaken. For example, lets
say that I have a PF with netdevs p2p1 (port 1) and p2p2 (port 2), and I
set it up to have the first 4 VFs on port 1 and the 2nd 4 on port 2. It
turns out that I can set the MAC/vlan tag using either p2p1 *or* p2p2
and it will be set in the correct place. However, if I try to *get* the
current MAC/vlan tag for a VF that uses port 2 via the PF netdev for
port 1, then I will just get back 0.

This might seem confusing/misleading, but it actually makes the job of
fixing device assignment for single port mode a bit simpler!


>
> I did send patches yesterday (any reviews/testing appreciated!) that
> make everything work properly when saving/setting/restoring the VF
> netdev MAC/vlan tags on dual port cards used for macvtap passthrough:
>
>   https://www.redhat.com/archives/libvir-list/2017-August/msg00170.html
>
> Those don't fix the situation when doing VFIO device assignment, but
> they do at least make macvtap passthrough work correctly (for all VF
> netdevs, even when the netdevs are dual port!), and are the first step
> in getting it right for VFIO.
>