[Fedora-xen] VLANs on DomU domains

[Bertho Stultiens]

> I am setting up a box with FC6 + xen with FC6 for the DomU guests.
> I have a server with 4 gigabit ports and I want to use 802.1Q vlans on
> one of the gig ports.
> I have the VLANs configured in Dom0 without any addresses attached to
> them.
> How do I make the VLANs available to the guests? I want to make 4
> VLANs available to each guest. I have thought of a few alternatives
> but I cannot get any to work.

The easy way; I have installed FC6 with 802.1Q and 4 guests. It works
very nicely. For each guest do:
1 - You need to enable vlans /etc/sysconfig/network.
	NETWORKING=yes
	VLAN=yes
	NETWORKING_IPV6=no
	HOSTNAME=xenbox2.example.com
	GATEWAY=192.168.42.1

2 - create a subinterface by file editting
/etc/sysconfig/network-scripts/ifcfg-eth0.42:
	DEVICE=eth0.42
	BOOTPROTO=static
	ONBOOT=yes
	IPADDR=192.168.42.102
	NETMASK=255.255.255.0
	TYPE=Ethernet

3 - depending on whether you want untagged packets on the same
interface, you need to edit the /etc/.../ifcfg-eth0 file too. (note: the
parent interface must be up for the tagged interface to work.) The lines
commented out will assign an IP address to the untagged interface:
	DEVICE=eth0
	BOOTPROTO=none
	ONBOOT=yes
	TYPE=Ethernet
	# BOOTPROTO=static
	# IPADDR=172.16.42.102
	# NETMASK=255.255.255.0

4 - make sure that your /etc/hosts setup is correct. If you have DNS
available 24/7, then only define localhost[.localdomain} as 127.0.0.1
and not the hostname itself.

When you restart the network in the guest, then your guest will default
to use the tagged interface vlan 42 (above example). If you want the
untagged interface as default, then you can change the gateway and
assign an IP address to it.

On the host, you need to do a similar thing. Create a subinterface
eth0.42 and assign an IP address to it. Once that is done, you can talk
to your guests via dot1Q from your host.

Beware of iptables! If you have a firewall setup, then you need to make
sure that it is done right. Doing it right, though, is a though job with
dot1Q in place. I have the firewall completely disabled on the host and
guests. My firewall is placed physically separated on another box.


> 1. Bridge the tagged 802.1Q stream from the Gig port through to the
> guests and configure eth0.x. on the guests

This would be above scenario.


> 2. Create a bridge on Dom0 for each VLAN so that they appear as
> eth0...eth3 on the guests.

The xenbrX interfaces on the host already transport all packets because
they are L2 bridges (see brctl). The bridge does not care whether the
packets are .1Q tagged or not. The only thing you need is to instruct
the kernel to look for/act on specifics by creating (sub-)interfaces.

Now, if you want the guests not to know that they are connected to a
VLAN, then you need a brigde-interface to do (un-)tagging for you. That
means that you need to create a dot1Q subinterface on the host's
physical peth[0-3] and add the subinterface to a new bridge.

Normal scenario:
peth0  -> xenbr0 -> vif0.0(host eth0)
                    vif1.0(guest1 eth0)

Dot1Q scenario:
peth0.42 -> xenbr42 -> vif1.0(guest1 eth0)

i.e.:
# brctl addbr xenbr42
# ip link set xenbr42 up
# vconfig add peth0 42
# ip link set peth0.42 up
# brctl addif xenbr42 peth0.42

In the VM config you have now something like:
vif = [ 'mac=00:16:3e:11:22:33, bridge=xenbr42', ]

The VM's eth0 interface is created as part of bridge that will fowrard
to a subinterface. The kernel will now do the tagging/untagging (note:
slow) and the VM cannot see that it is attached to a VLAN.

Theoretically it should work, but I was not able to make this scenario
work as expected though. I suspect that the hosts peth0 -> eth0 bridge
is interfering (which has a tagged interface too in my setup). Or maybe
my fingers were too fast at typing and I forgot a simple thing...



> The xen documentation is minimal for configuring VLANs. Is there a
> (FC6) supported way of configuring them?

Vlans is a no-issue for xen. It is layered in the network and xen only
passes the packets.


-- 
Greetings Bertho


Bertho Stultiens
Senior Systems Manager
Mobilethink A/S