Could you please help with questions about the net failover feature
Ken Cox
jkc at redhat.com
Wed Jul 8 14:02:55 UTC 2020
On 7/8/20 1:30 AM, Stefan Assmann wrote:
> On 2020-07-06 10:01, Laine Stump wrote:
>> On 7/6/20 5:10 AM, Yalan Zhang wrote:
>>> Hi Laine,
>>>
>>> For the feature testing before, I only test the linux bridge setting as
>>> in 2), it works.
>>> Now I tried 1), to use macvtap bridge mode connected to the PF, it can
>>> not work as the hostdev interface can not get dhcp ip address on the
>>> guest.
>>> Check on host, the /var/log/messages and dmesg both says:
>>>
>>> "Jul 6 04:54:45 dell-per730-xx kernel: ixgbe 0000:82:00.1 enp130s0f1: 1
>>> Spoofed packets detected
>>> ......
>>> Jul 6 04:56:17 dell-per730-xx kernel: ixgbe 0000:82:00.1 enp130s0f1: 1
>>> Spoofed packets detected
>>> Jul 6 04:56:54 dell-per730-xx kernel: ixgbe 0000:82:00.1 enp130s0f1: 1
>>> Spoofed packets detected
>>> "
>>> (enp130s0f1 is the PF's interface name, and 0000:82:00.1 is the PF's pci
>>> address)
>>> # rpm -q kernel
>>> kernel-4.18.0-193.4.1.el8_2.x86_64
>>>
>>> Could you please help to confirm if this is a kernel bug? Thank you
>>> very much!
>> Interesting. I'm not sure if this is expected behavior, or if it's improper
>> behavior and it just hasn't been tested before (obviously based on my
>> earlier recommendation, I think it *should* be able to work like this, and I
>> *thought* I had tried it, but maybe I just imagined it :-/).
>>
>> I'm Cc'ing Stefan Assmann to see if he has an opinion on whether or not this
>> should work. For his convenience, here is a summary of the config: The setup
>> is that there is a bridge-mode macvtap interface on the PF, and one of the
>> VF's has been given the same MAC address as the macvtap. the macvtap
>> interface is connected to an emulated NIC in the guest, and the VF is
>> assigned to the guest with VFIO.
IIUC, the problem is using the same mac address for macvtap and for a
VF. This is what's causing the spoofed packets.
Ken
> Hi Laine,
>
> I'm on PTO till the 20th. Ccing Ken as ixgbe maintainer.
>
> Stefan
>
>> I'll try this today on my setup, which uses I350 cards (igb driver).
>>
>>
>>>
>>>
>>> You have two choices for the backup virtio interface:
>>>
>>> 1) it can be a macvtap device connected to the PF of the same SRIOV device.
>>>
>>> 2) it can be a standard tap device connected to a Linux host bridge
>>> (created outside libvirt in the host system network config) that is
>>> attached to the PF (or alternately one of the VFs that isn't being used
>>> for VMs, or to another physical ethernet adapter on the host that is
>>> connected to the same network.
>>>
>>>
>>>
>>>
>>> -------
>>> Best Regards,
>>> Yalan Zhang
>>> IRC: yalzhang
>>>
>>>
>>> On Sun, Mar 22, 2020 at 6:50 AM Laine Stump <laine at redhat.com
>>> <mailto:laine at redhat.com>> wrote:
>>>
>>> On 3/21/20 1:08 AM, Yalan Zhang wrote:
>>>
>>> > In my understanding, the standby and primary hostdev interface
>>> may be in
>>> > different subnet.
>>>
>>> There is only one hostdev device in the team pair (that will be the one
>>> with <teaming type='transient'.../> since it needs to be unplugged
>>> during migration). The other device must be a virtio device (the one
>>> with <teaming type='persistent'/>). And no, they cannot be on different
>>> subnets. They must both connect into the same ethernet "collision
>>> domain", such that the guest could assign the same IP address to either
>>> of them and be able to communicate on the network.
>>>
>>> There is some explanation of the use case for this option. and some
>>> example config, here:
>>>
>>> https://www.libvirt.org/formatdomain.html#elementsTeaming
>>>
>>> > I'm not sure whether it is correct. Could you please help to
>>> explain?
>>> > Thank you in advance.
>>> >
>>> > For example, primary hostdev is connected to vf-pool with
>>> <pf='eth0'/>,
>>> > while the standby is connected to NAT network with " forward
>>> dev='eth0'".
>>> > The standby interface will get ip as 192.168.122.x, but after
>>> NAT, it
>>> > will be in the same subnet of the vf.
>>> >
>>> > So after the VF is unplugged, the packet will still broadcast in the
>>> > same subnet, and the vm will get the packet as the standby share the
>>> > same mac. Right?
>>>
>>> No, not right :-)
>>>
>>> The VF of an SRIOV network adapter is connected directly to the
>>> physical
>>> network, and will have an IP address that is on that network. Tap
>>> devices plugged into the default network (or any other libvirt network
>>> based on a bridge device that is created/managed by libvirt) have no
>>> direct connection to the physical network, and are on a different
>>> subnet. The fact that traffic from the guest *seems* to be coming from
>>> an IP on the physical subnet is meaningless. The *guest* needs to be
>>> able to use both NICs using the same IP address, and anything plugged
>>> into the default network will need to have an IP address on a different
>>> subnet from the perspective of the guest.
>>>
>>> You have two choices for the backup virtio interface:
>>>
>>> 1) it can be a macvtap device connected to the PF of the same SRIOV
>>> device.
>>>
>>> 2) it can be a standard tap device connected to a Linux host bridge
>>> (created outside libvirt in the host system network config) that is
>>> attached to the PF (or alternately one of the VFs that isn't being used
>>> for VMs, or to another physical ethernet adapter on the host that is
>>> connected to the same network.
>>>
>>>
>>> It is simplest to have the same name refer to the connection on the
>>> source and destination hosts of a migration. That can be handled by
>>> creating a libvirt network to refer to the bridge device created
>>> outside
>>> libvirt (or to the PF directly if you're going to use macvtap.
>>>
>>> For example, if you're going to use macvtap, and the PF's name on the
>>> host is ens4f0, you'd just create this network:
>>>
>>> <network>
>>> <name>persistent-net</name>
>>> <forward mode='bridge'>
>>> <interface dev='ens4f0'/>
>>> </forward>
>>> <network>
>>>
>>> any guest interface with this:
>>>
>>> <interface type='network'>
>>> <source network='persistent-net'/>
>>> <mac address='00:11:22:33:44:55'/>
>>> <model type='virtio'/>
>>> <teaming type='persistent'/>
>>> <alias name='ua-backup0'/>
>>> </interface>
>>>
>>> will get a macvtap device that's connected to ens4f0 in bridge mode.
>>>
>>> Or, if your host has a bridge device called br0 that is directly
>>> attached to the physical network (in whatever manner, it doesn't
>>> matter), you can define the network this way:
>>>
>>> <network>
>>> <name>persistent-net</name>
>>> <bridge name='br0'/>
>>> <forward mode='bridge'/>
>>> <network>
>>>
>>> The XML for the guest interface would be the same.
>>>
>>> Then for the vfio (transient) interface, you could also define a
>>> network:
>>>
>>> <network>
>>> <name>transient-net</name>
>>> <forward mode='hostdev'>
>>> <pf dev='ens4f0'/>
>>> </forward>
>>> </network>
>>>
>>> and instead of using <interface type='hostdev'> in the guest config,
>>> you
>>> would use this:
>>>
>>>
>>> <interface type='network'>
>>> <source network='transient-net'/>
>>> <model type='virtio'/> [1]
>>> <mac address='00:11:22:33:44:55'/>
>>> <teaming type='transient' persistent='ua-backup0'/>
>>> </interface>
>>>
>>> Even if the device names change on the other host (the destination of
>>> the migration), as long as the other host has networks named
>>> "persistent-net" and "transient-net" that are of similar types (macvtap
>>> or bridged for persistent-net, and hostdev for transient-net) then
>>> libvirt will be able to migrate the guest from one host to the other
>>> with no mangling of the XML required.
>>>
>>>
More information about the libvirt-users
mailing list