[libvirt] IPv6: getting it to work

[Gene Czarcinski]

Before I start creating patches (since it is not only source code but 
also documentation, schemas, tests, etc), I thought I would run this by 
you folks for comments/suggestions.

With IPv4, it is relatively easy to set up working networks: just use 
nat/MASQUERADE and things pretty much just work.

With IPv6, it is a bit more difficult because IPv6 is route-only with 
(theoretically) unique addresses known across the Internet.  It took me 
a while but once I realized that I needed to define some static routes 
on my default router, it because much easier.  The default route(s) on 
the default router connect the IPv6 guest network used by a 
qemu-kvm/libvirt with the virtual host's IPv6 address.

My recommendation is to use a /48 or /56 IPv6 network and assign it to a 
specific virtual host.  This virtual host needs to have a fixed IPv6 
address either with manual configuration or using a client-id to pin a 
specific IPv6 address.  Then make each of the virtual networks on that 
host be a /64 network.  So far so good and really not too much of an 
administrative burden.  NetworkManager can be used to set this default 
route.

Now lets add some additional virtual network layers/segments to the 
mix.  For example:

   guest10 <-> net-a <-> guest20 <-> net-b <-> guest30 <-> virbr<n> 
host40 <-> router50 <-> host60

guest30 can talk to other systems such as host40, router50, and host60 
on the real network since it is covered by the static route on the 
default router.

guest10, guest20, and guest30 can talk to each other with some 
additional static routes (or just default routes).

The problem: guest10 cannot talk to other real hosts such as host40 or 
host60.  The problem is that NetworkManager will not set a static route 
for any network on a libvirt bridge device (or any bridge device which 
NM does not "own").

At first I thought this was a NM problem but I now believe that this 
should be fixed by libvirt.

I did some manual configuration to figure out what needed to be done so 
that guest10 could use IPv6 to talk to another host on the real network.

1. You have defined a static route on the default router for 
fd00:aa:bb::/48 to the virtual host.

2. You have an libvirt network defined with (for example) 
fd00:aa:bb:10::/64 and a guest on that network with the address of 
fd00:aa:bb:10::2/128.  Lets say it is on virbr4.

3.  Your secondary (isolated) virtual network is fd00:aa:bb:11::/64.

4.  You need to issue firewalld commands so it will pass the additional 
network on virbr4.  These are of the form:
     firewall-cmd --direct --passthrough ipv6 -I FORWARD -1 -d 
fd00:aa:bb:11::/64 -o virbr4 -j ACCEPT
     firewall-cmd --direct --passthrough ipv6 -I FORWARD -1 -s 
fd00:aa:bb:11::/64 -i virbr4 -j ACCEPT

5. Create the route:
   ip  -6  route  fd00:aa:bb:11::/64  via  fd00:aa:bb:10::2 dev virbr4  
proto static  metric  1

OK, that is what has to be done but I want libvirt to do all of this for 
me after some simple configuration.

I propose adding a new optional xml-element to the <ip> element: <via>

<via> would be an exclusive alternate to <dhcp> and both <via> and 
<dhcp> could not be used under a single <ip> definition.  As implied by 
the name, <via> would specify the gateway address which is to receive 
the packets on the designated network.

Right now, if you specify an additional IPv6 address to a network 
definition, you get the correct ip6tables rules but you also get a 
ip-addr for that additional definition and an ip-route for the related 
network.  With <via>, this last part would be replaced with NO 
additional ip-addr and a static route for the network to the gateway.

Anticipated code changes (besides the tests, schemas, and documentation) 
are:

network_conf.c to handle the new <via> element.

virnetdev.c to create and issue the ip-6-route command.

bridge_driver.c to detect when an IPv6 address is a "via" and do the 
ip-6-route instead of adding the address.

Although this is being done for IPv6, there is no reason not to make 
sure it also works with IPv4.

Comments, suggestions appreciated.

Gene