network config not working on newer libvirt

[Laine Stump]

On 9/4/20 12:38 AM, daggs wrote:
> Greetings,
> 
> up until a year ago, I was running a server with Debian 10 (stable) on it with the latest versions of libvirt, qemu and kernel 4.19.x Debian 10 had to offer (both libvirt and qemu versions were really old).
> 
> the network config was simple, one of the vm acted as a router and provided the ip for both the host and the vm.
> I've recently switched distro and now I'm running latest stable libvirt, qemu and kernel 5.4.x

You haven't said which distro, nor what is the libvirt exact libvirt 
version (probably won't matter in this case, but in general "libvirt 
x.y.z" is more useful than "latest stable libvirt").

If you have full connectivity once you've manually assigned IP 
addresses, then you don't have any routing problems, so that can be 
counted out. (Anyway, DHCP packets never go beyond the local network).

In that case, you've most likely either got a firewall problem on host 
or guest, or a problem with your dhcp server.


> I've tried to reinstate the network config on the new distro but I cannot get ip via dhcp for the second vm.
> if I assign manual ip and gateway, I have access to the outside world.

 From where? The host? or the guests?

> 
> here are the relevant dumps:
> network on the router vm:
>      <interface type='network'>
>        <mac address='52:54:00:53:1c:6b'/>
>        <source network='default'/>
>        <target dev='virtsw0-vm1'/>
>        <model type='virtio'/>
>        <address type='pci' domain='0x0000' bus='0x21' slot='0x01' function='0x0'/>
>      </interface>
> 
> the other vm
>      <interface type='network'>
>        <mac address='52:54:00:5a:4c:8c'/>
>        <source network='default'/>
>        <target dev='virtsw0-vm2'/>
>        <model type='virtio'/>
>        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
>      </interface>
> 
> and finally:
> <network connections='2'>
>    <name>default</name>
>    <uuid>61bc1a72-bd02-408a-b88e-dec696742c20</uuid>
>    <bridge name='virtsw0' stp='on' delay='0'/>
>    <mac address='52:54:00:6b:1b:92'/>
> </network>

Your config is for a bridge that's created by libvirt, but with no 
iptables rules, no dnsmasq instance, and no IP address on the host. So 
any DHCP server config is outside libvirt's realm, as are any iptables 
or nftables rules, so in this case there is nothing to look at in the 
libvirt config for either of these issues.

I would start troubleshooting by making sure that the dhcp server is 
running, and that you can communicate between the machine with DHCP 
server and the guest once a manual IP is assigned. Then use tcpdump or 
wireshark at different places on the path between those two to see how 
far the DHCP request is getting out, whether a response is being sent by 
the server, and if so how far the response is getting back (i.e. on the 
host, run tcpdump on the guest's tap device; if you see the DHCP request 
there, then run tcpdump on the bridge, if you see it there, run it on 
the tap device for the guest, if you see it there, then run tcpdump 
inside the guest; then check the dhcp server logs to see if it's 
receiving requests. While you're doing all of this, you can also be 
noticing whether or not a DHCP response is arriving at each step (and if 
you see the response, you can skip looking further ahead in the packet 
path, since you know by inference that it made it all the way to the 
DHCP server). Once you find the point that the packet is blocked, you'll 
be better able to determine why.


> 
> as it is possible I'm missing a kernel config, here is the output of lsmod:
> vfio_pci               49152  6
> vfio_virqfd            16384  1 vfio_pci
> vfio_iommu_type1       32768  2
> vfio                   28672  16 vfio_iommu_type1,vfio_pci
> ip6table_nat           16384  1
> iptable_nat            16384  1
> ebtables               24576  0
> bridge                143360  0
> stp                    16384  1 bridge
> llc                    16384  2 bridge,stp
> cfg80211              647168  0
> x86_pkg_temp_thermal    20480  0
> kvm_intel             237568  8
> vhost_net              24576  3
> vhost                  36864  1 vhost_net
> tap                    24576  1 vhost_net
> kvm                   663552  1 kvm_intel
> tun                    53248  8 vhost_net
> r8152                  73728  0
> nct6775                57344  0
> mei_me                 32768  0
> hwmon_vid              20480  1 nct6775
> irqbypass              16384  22 vfio_pci,kvm
> mii                    16384  1 r8152
> mei                    77824  1 mei_me
> coretemp               16384  0
> efivarfs               16384  1
> 
> and the .config at https://dpaste.com/9ZUCBDE9R
> 
> any ideas how to fix it?
> 
> Thanks,
> 
> Dagg.
>