[libvirt-users] What's the best way to make use of VLAN interfaces with VMs?

[Richard Achmatowicz]

Hi Laine

What you have suggested sounds eminently reasonable. Thanks for your 
advice. I'm going to give it a shot and report back.

Richard

On 11/27/19 1:38 PM, Laine Stump wrote:
> On 11/26/19 11:07 PM, Richard Achmatowicz wrote:
>> Hello
>>
>> I have a problem with attaching VMs to a VLAN interface.
>>
>> Here is my setup: I have several physical hosts connected by a 
>> physical switch.  Each host has two NICs leading to the switch, which 
>> have been combined into a team, team0. Each host a has a bridge br1, 
>> which has team0 as a slave. So communication between hosts is based 
>> on the IP address of bridge br1 on each host.
>>
>> Up until recently, using libvirt and KVM, I was creating VMs which 
>> had one interface attached the default virtual network and one 
>> interface attached to the bridge:
>>
>> virt-install ... --network network=default --network bridge=br1 ...
>>
>> I would then statically assign an IP address to the bridge interface 
>> on the guest when installing the OS.
>>
>> A few days ago, a VLAN was introduced to split up the network. I 
>> created a new VLAN interface br1.600 on each of the hosts. My initial 
>> attempt was to do try this:
>>
>> virt-install ... --network network=default --network bridge=br1.600 ...
>>
>> which did not work. It then dawned on me that a VLAN interface and a 
>> bridge aren't treated the same. So I started to look for ways to 
>> allow my VMs to bind to this new interface.
>>
>> This would seem to be a common situation. What is the best way to 
>> work around this?
>>
>> Both the host bridge and the host VLAN interface already have their 
>> assigned IP addresses and appear like this in libvirt:
>>
>> [root at clusterdev01 ]# ifconfig
>> br1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>>          inet 192.168.0.110  netmask 255.255.255.0  broadcast 
>> 192.168.0.255
>>          inet6 fe80::1e98:ecff:fe1b:276d  prefixlen 64  scopeid 
>> 0x20<link>
>>          ether 1c:98:ec:1b:27:6d  txqueuelen 1000  (Ethernet)
>>          RX packets 833772  bytes 2976958254 (2.7 GiB)
>>          RX errors 0  dropped 0  overruns 0  frame 0
>>          TX packets 331237  bytes 23335124 (22.2 MiB)
>>          TX errors 0  dropped 0 overruns 0  carrier 0 collisions 0
>>
>> br1.600: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>>          inet 192.168.1.110  netmask 255.255.255.0  broadcast 
>> 192.168.1.255
>>          inet6 fe80::1e98:ecff:fe1b:276d  prefixlen 64  scopeid 
>> 0x20<link>
>>          ether 1c:98:ec:1b:27:6d  txqueuelen 1000  (Ethernet)
>>          RX packets 189315  bytes 9465744 (9.0 MiB)
>>          RX errors 0  dropped 0  overruns 0  frame 0
>>          TX packets 302  bytes 30522 (29.8 KiB)
>>          TX errors 0  dropped 0 overruns 0  carrier 0 collisions 0
>>
>> [root at clusterdev01]# virsh iface-list --all
>>   Name                 State      MAC Address
>> ---------------------------------------------------
>>   br1                  active     1c:98:ec:1b:27:6d
>>   br1.600           active     1c:98:ec:1b:27:6d
>>
>> [root at clusterdev01 sysadmin]# virsh iface-dumpxml br1.600
>> <interface type='vlan' name='br1.600'>
>>    <protocol family='ipv4'>
>>      <ip address='192.168.1.110' prefix='24'/>
>>    </protocol>
>>    <protocol family='ipv6'>
>>      <ip address='fe80::1e98:ecff:fe1b:276d' prefix='64'/>
>>    </protocol>
>>    <link state='up'/>
>>    <vlan tag='600'>
>>      <interface name='br1'/>
>>    </vlan>
>> </interface>
>>
>> I tried following some suggestions which wrapped the vlan interface 
>> in a bridge interface, but in ended up trashing the br1.600 interface 
>> which was originally defined on the host.
>>
>> Is there a failsafe way to deal with such a situation? Am I doing 
>> something completely wrong here? In would like br1.600 to behave like 
>> br1 .....
>>
>> Any suggestions or advice greatly appreciated.
>
>
> I guess what you need is for all the traffic from your guests to go 
> out on the physical network tagged with vlan id 600, and you want that 
> to be transparent to the guests, right?
>
> The simplest way to handle this is to create a vlan interface off of 
> the ethernet that you have attached to br1 (not br1 itself), so it 
> would be named something like "eth0.600", and then create a new bridge 
> (call it, say "br600") and attach eth0.600 to br600. Then your guests 
> would be created with "--network bridge=br600"
>
> (Note that Linux host bridges do now support vlan tagging (and maybe 
> even trunking) at the port level, but libvirt hasn't added support for 
> it. (in other words, "Patches Welcome!" :-))
>