[libvirt] IPv6: getting it to work

[Gene Czarcinski]

On 03/11/2013 03:32 PM, Gene Czarcinski wrote:
> Before I start creating patches (since it is not only source code but 
> also documentation, schemas, tests, etc), I thought I would run this 
> by you folks for comments/suggestions.
>
> With IPv4, it is relatively easy to set up working networks: just use 
> nat/MASQUERADE and things pretty much just work.
>
> With IPv6, it is a bit more difficult because IPv6 is route-only with 
> (theoretically) unique addresses known across the Internet. It took me 
> a while but once I realized that I needed to define some static routes 
> on my default router, it because much easier. The default route(s) on 
> the default router connect the IPv6 guest network used by a 
> qemu-kvm/libvirt with the virtual host's IPv6 address.
>
> My recommendation is to use a /48 or /56 IPv6 network and assign it to 
> a specific virtual host.  This virtual host needs to have a fixed IPv6 
> address either with manual configuration or using a client-id to pin a 
> specific IPv6 address.  Then make each of the virtual networks on that 
> host be a /64 network.  So far so good and really not too much of an 
> administrative burden. NetworkManager can be used to set this default 
> route.
>
> Now lets add some additional virtual network layers/segments to the 
> mix.  For example:
>
>   guest10 <-> net-a <-> guest20 <-> net-b <-> guest30 <-> virbr<n> 
> host40 <-> router50 <-> host60
>
> guest30 can talk to other systems such as host40, router50, and host60 
> on the real network since it is covered by the static route on the 
> default router.
>
> guest10, guest20, and guest30 can talk to each other with some 
> additional static routes (or just default routes).
>
> The problem: guest10 cannot talk to other real hosts such as host40 or 
> host60.  The problem is that NetworkManager will not set a static 
> route for any network on a libvirt bridge device (or any bridge device 
> which NM does not "own").
>
> At first I thought this was a NM problem but I now believe that this 
> should be fixed by libvirt.
>
> I did some manual configuration to figure out what needed to be done 
> so that guest10 could use IPv6 to talk to another host on the real 
> network.
>
> 1. You have defined a static route on the default router for 
> fd00:aa:bb::/48 to the virtual host.
>
> 2. You have an libvirt network defined with (for example) 
> fd00:aa:bb:10::/64 and a guest on that network with the address of 
> fd00:aa:bb:10::2/128.  Lets say it is on virbr4.
>
> 3.  Your secondary (isolated) virtual network is fd00:aa:bb:11::/64.
>
> 4.  You need to issue firewalld commands so it will pass the 
> additional network on virbr4.  These are of the form:
>     firewall-cmd --direct --passthrough ipv6 -I FORWARD -1 -d 
> fd00:aa:bb:11::/64 -o virbr4 -j ACCEPT
>     firewall-cmd --direct --passthrough ipv6 -I FORWARD -1 -s 
> fd00:aa:bb:11::/64 -i virbr4 -j ACCEPT
>
> 5. Create the route:
>   ip  -6  route  fd00:aa:bb:11::/64  via  fd00:aa:bb:10::2 dev virbr4  
> proto static  metric  1
>
> OK, that is what has to be done but I want libvirt to do all of this 
> for me after some simple configuration.
>
> I propose adding a new optional xml-element to the <ip> element: <via>
>
> <via> would be an exclusive alternate to <dhcp> and both <via> and 
> <dhcp> could not be used under a single <ip> definition.  As implied 
> by the name, <via> would specify the gateway address which is to 
> receive the packets on the designated network.

An alternative is to have via specified as pat of the <ip> definition 
itself.  That is, <ip> would have family=, address=, prefix=, and now via=

>
> Right now, if you specify an additional IPv6 address to a network 
> definition, you get the correct ip6tables rules but you also get a 
> ip-addr for that additional definition and an ip-route for the related 
> network.  With <via>, this last part would be replaced with NO 
> additional ip-addr and a static route for the network to the gateway.
>
> Anticipated code changes (besides the tests, schemas, and 
> documentation) are:
>
> network_conf.c to handle the new <via> element.
>
> virnetdev.c to create and issue the ip-6-route command.
>
> bridge_driver.c to detect when an IPv6 address is a "via" and do the 
> ip-6-route instead of adding the address.
>
> Although this is being done for IPv6, there is no reason not to make 
> sure it also works with IPv4.
>
> Comments, suggestions appreciated.
>
> Gene
>