<div dir="ltr"><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>Hi Laine, </div><div><br></div><div>For the feature testing before, I only test the linux bridge setting as in 2), it works.</div><div>Now I tried 1), to use macvtap bridge mode connected to the PF, it can not work as the hostdev interface can not get dhcp ip address on the guest.</div><div>Check on host, the /var/log/messages and dmesg both says:</div><div><br></div><div>"Jul 6 04:54:45 dell-per730-xx kernel: ixgbe 0000:82:00.1 enp130s0f1: 1 Spoofed packets detected</div>......<br>Jul 6 04:56:17 dell-per730-xx kernel: ixgbe 0000:82:00.1 enp130s0f1: 1 Spoofed packets detected<br>Jul 6 04:56:54 dell-per730-xx kernel: ixgbe 0000:82:00.1 enp130s0f1: 1 Spoofed packets detected<br><div>"</div><div>(enp130s0f1 is the PF's interface name, and 0000:82:00.1 is the PF's pci address)</div><div># rpm -q kernel<br>kernel-4.18.0-193.4.1.el8_2.x86_64<br></div><div><br></div><div>Could you please help to confirm if this is a kernel bug? Thank you very much!</div><div><br></div><div><br></div><div><br></div><div>You have two choices for the backup virtio interface:<br><br>1) it can be a macvtap device connected to the PF of the same SRIOV device.<br><br>2) it can be a standard tap device connected to a Linux host bridge <br>(created outside libvirt in the host system network config) that is <br>attached to the PF (or alternately one of the VFs that isn't being used <br>for VMs, or to another physical ethernet adapter on the host that is <br>connected to the same network.</div><div><br></div><div><br></div><div><br></div><div><br>-------<br>Best Regards,<br>Yalan Zhang<br>IRC: yalzhang<br></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Mar 22, 2020 at 6:50 AM Laine Stump <<a href="mailto:laine@redhat.com">laine@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 3/21/20 1:08 AM, Yalan Zhang wrote:<br>
<br>
> In my understanding, the standby and primary hostdev interface may be in <br>
> different subnet.<br>
<br>
There is only one hostdev device in the team pair (that will be the one <br>
with <teaming type='transient'.../> since it needs to be unplugged <br>
during migration). The other device must be a virtio device (the one <br>
with <teaming type='persistent'/>). And no, they cannot be on different <br>
subnets. They must both connect into the same ethernet "collision <br>
domain", such that the guest could assign the same IP address to either <br>
of them and be able to communicate on the network.<br>
<br>
There is some explanation of the use case for this option. and some <br>
example config, here:<br>
<br>
<a href="https://www.libvirt.org/formatdomain.html#elementsTeaming" rel="noreferrer" target="_blank">https://www.libvirt.org/formatdomain.html#elementsTeaming</a><br>
<br>
> I'm not sure whether it is correct. Could you please help to explain? <br>
> Thank you in advance.<br>
> <br>
> For example, primary hostdev is connected to vf-pool with <pf='eth0'/>, <br>
> while the standby is connected to NAT network with " forward dev='eth0'".<br>
> The standby interface will get ip as 192.168.122.x, but after NAT, it <br>
> will be in the same subnet of the vf.<br>
><br>
> So after the VF is unplugged, the packet will still broadcast in the <br>
> same subnet, and the vm will get the packet as the standby share the <br>
> same mac. Right?<br>
<br>
No, not right :-)<br>
<br>
The VF of an SRIOV network adapter is connected directly to the physical <br>
network, and will have an IP address that is on that network. Tap <br>
devices plugged into the default network (or any other libvirt network <br>
based on a bridge device that is created/managed by libvirt) have no <br>
direct connection to the physical network, and are on a different <br>
subnet. The fact that traffic from the guest *seems* to be coming from <br>
an IP on the physical subnet is meaningless. The *guest* needs to be <br>
able to use both NICs using the same IP address, and anything plugged <br>
into the default network will need to have an IP address on a different <br>
subnet from the perspective of the guest.<br>
<br>
You have two choices for the backup virtio interface:<br>
<br>
1) it can be a macvtap device connected to the PF of the same SRIOV device.<br>
<br>
2) it can be a standard tap device connected to a Linux host bridge <br>
(created outside libvirt in the host system network config) that is <br>
attached to the PF (or alternately one of the VFs that isn't being used <br>
for VMs, or to another physical ethernet adapter on the host that is <br>
connected to the same network.<br>
<br>
<br>
It is simplest to have the same name refer to the connection on the <br>
source and destination hosts of a migration. That can be handled by <br>
creating a libvirt network to refer to the bridge device created outside <br>
libvirt (or to the PF directly if you're going to use macvtap.<br>
<br>
For example, if you're going to use macvtap, and the PF's name on the <br>
host is ens4f0, you'd just create this network:<br>
<br>
<network><br>
<name>persistent-net</name><br>
<forward mode='bridge'><br>
<interface dev='ens4f0'/><br>
</forward><br>
<network><br>
<br>
any guest interface with this:<br>
<br>
<interface type='network'><br>
<source network='persistent-net'/><br>
<mac address='00:11:22:33:44:55'/><br>
<model type='virtio'/><br>
<teaming type='persistent'/><br>
<alias name='ua-backup0'/><br>
</interface><br>
<br>
will get a macvtap device that's connected to ens4f0 in bridge mode.<br>
<br>
Or, if your host has a bridge device called br0 that is directly <br>
attached to the physical network (in whatever manner, it doesn't <br>
matter), you can define the network this way:<br>
<br>
<network><br>
<name>persistent-net</name><br>
<bridge name='br0'/><br>
<forward mode='bridge'/><br>
<network><br>
<br>
The XML for the guest interface would be the same.<br>
<br>
Then for the vfio (transient) interface, you could also define a network:<br>
<br>
<network><br>
<name>transient-net</name><br>
<forward mode='hostdev'><br>
<pf dev='ens4f0'/><br>
</forward><br>
</network><br>
<br>
and instead of using <interface type='hostdev'> in the guest config, you <br>
would use this:<br>
<br>
<br>
<interface type='network'><br>
<source network='transient-net'/><br>
<model type='virtio'/> [1]<br>
<mac address='00:11:22:33:44:55'/><br>
<teaming type='transient' persistent='ua-backup0'/><br>
</interface><br>
<br>
Even if the device names change on the other host (the destination of <br>
the migration), as long as the other host has networks named <br>
"persistent-net" and "transient-net" that are of similar types (macvtap <br>
or bridged for persistent-net, and hostdev for transient-net) then <br>
libvirt will be able to migrate the guest from one host to the other <br>
with no mangling of the XML required.<br>
<br>
<br>
</blockquote></div>