[libvirt] IPv6: getting it to work

[Gene Czarcinski]

On 03/12/2013 08:05 AM, Gene Czarcinski wrote:
> On 03/11/2013 03:32 PM, Gene Czarcinski wrote:
>> Before I start creating patches (since it is not only source code but 
>> also documentation, schemas, tests, etc), I thought I would run this 
>> by you folks for comments/suggestions.
>>
>> With IPv4, it is relatively easy to set up working networks: just use 
>> nat/MASQUERADE and things pretty much just work.
>>
>> With IPv6, it is a bit more difficult because IPv6 is route-only with 
>> (theoretically) unique addresses known across the Internet. It took 
>> me a while but once I realized that I needed to define some static 
>> routes on my default router, it because much easier. The default 
>> route(s) on the default router connect the IPv6 guest network used by 
>> a qemu-kvm/libvirt with the virtual host's IPv6 address.
>>
>> My recommendation is to use a /48 or /56 IPv6 network and assign it 
>> to a specific virtual host.  This virtual host needs to have a fixed 
>> IPv6 address either with manual configuration or using a client-id to 
>> pin a specific IPv6 address.  Then make each of the virtual networks 
>> on that host be a /64 network.  So far so good and really not too 
>> much of an administrative burden. NetworkManager can be used to set 
>> this default route.
>>
>> Now lets add some additional virtual network layers/segments to the 
>> mix.  For example:
>>
>>   guest10 <-> net-a <-> guest20 <-> net-b <-> guest30 <-> virbr<n> 
>> host40 <-> router50 <-> host60
>>
>> guest30 can talk to other systems such as host40, router50, and 
>> host60 on the real network since it is covered by the static route on 
>> the default router.
>>
>> guest10, guest20, and guest30 can talk to each other with some 
>> additional static routes (or just default routes).
>>
>> The problem: guest10 cannot talk to other real hosts such as host40 
>> or host60.  The problem is that NetworkManager will not set a static 
>> route for any network on a libvirt bridge device (or any bridge 
>> device which NM does not "own").
>>
>> At first I thought this was a NM problem but I now believe that this 
>> should be fixed by libvirt.
>>
>> I did some manual configuration to figure out what needed to be done 
>> so that guest10 could use IPv6 to talk to another host on the real 
>> network.
>>
>> 1. You have defined a static route on the default router for 
>> fd00:aa:bb::/48 to the virtual host.
>>
>> 2. You have an libvirt network defined with (for example) 
>> fd00:aa:bb:10::/64 and a guest on that network with the address of 
>> fd00:aa:bb:10::2/128.  Lets say it is on virbr4.
>>
>> 3.  Your secondary (isolated) virtual network is fd00:aa:bb:11::/64.
>>
>> 4.  You need to issue firewalld commands so it will pass the 
>> additional network on virbr4.  These are of the form:
>>     firewall-cmd --direct --passthrough ipv6 -I FORWARD -1 -d 
>> fd00:aa:bb:11::/64 -o virbr4 -j ACCEPT
>>     firewall-cmd --direct --passthrough ipv6 -I FORWARD -1 -s 
>> fd00:aa:bb:11::/64 -i virbr4 -j ACCEPT
>>
>> 5. Create the route:
>>   ip  -6  route  fd00:aa:bb:11::/64  via  fd00:aa:bb:10::2 dev 
>> virbr4  proto static  metric  1
>>
>> OK, that is what has to be done but I want libvirt to do all of this 
>> for me after some simple configuration.
>>
>> I propose adding a new optional xml-element to the <ip> element: <via>
>>
>> <via> would be an exclusive alternate to <dhcp> and both <via> and 
>> <dhcp> could not be used under a single <ip> definition.  As implied 
>> by the name, <via> would specify the gateway address which is to 
>> receive the packets on the designated network.
>
> An alternative is to have via specified as pat of the <ip> definition 
> itself.  That is, <ip> would have family=, address=, prefix=, and now 
> via=
Too late.  I have implemented the "alternative"

And it appears to work.  Now for the "fun stuff" of documentation, 
tests, and the schema.

>
>>
>> Right now, if you specify an additional IPv6 address to a network 
>> definition, you get the correct ip6tables rules but you also get a 
>> ip-addr for that additional definition and an ip-route for the 
>> related network.  With <via>, this last part would be replaced with 
>> NO additional ip-addr and a static route for the network to the gateway.
>>
>> Anticipated code changes (besides the tests, schemas, and 
>> documentation) are:
>>
>> network_conf.c to handle the new <via> element.
>>
>> virnetdev.c to create and issue the ip-6-route command.
>>
>> bridge_driver.c to detect when an IPv6 address is a "via" and do the 
>> ip-6-route instead of adding the address.
>>
>> Although this is being done for IPv6, there is no reason not to make 
>> sure it also works with IPv4.
>>
>> Comments, suggestions appreciated.
>>
>> Gene
>>
>