[libvirt] [PATCH] Fix dnsmasq/radvd on bridges not being able to be bound to IPv6 address on "recent" kernels

[Daniel P. Berrange]

On Tue, Jun 19, 2012 at 07:00:58PM +0200, Benjamin Cama wrote:
> Hi,
> 
> I hit this problem recently when trying to create a bridge with an IPv6
> address on a 3.2 kernel: dnsmasq (and, further, radvd) would not bind to
> the given address (resp. interface), waiting 20s and then giving up with
> -EADDRNOTAVAIL (resp. exiting immediately with "error parsing or
> activating the config file", without libvirt noticing it, BTW). This can
> be reproduced with (I think) any kernel >= 2.6.39 and the following XML
> (to be used with "virsh net-create"):
> 
>         <network>
>           <name>test-bridge</name>
>           <bridge name='testbr0' />
>           <ip family='ipv6' address='fd00::1' prefix='64'>
>           </ip>
>         </network>
> 
> (it happens even when you have an IPv4, too)
> 
> The problem is that since commit [1] (which, ironically, was made to
> “help IPv6 autoconfiguration”) the linux bridge code makes bridges
> behave like “real” devices regarding carrier detection. This makes the
> bridges created by libvirt, which are started without any up devices,
> stay with the NO-CARRIER flag set, and thus prevents DAD (Duplicate
> address detection) from happening, thus letting the IPv6 address flagged
> as “tentative”. Such addresses cannot be bound to (see RFC 2462), so
> dnsmasq fails binding to it (for radvd, it detects that "interface XXX
> is not RUNNING", thus that "interface XXX does not exist, ignoring the
> interface" (sic)). It seems that this behavior was enhanced somehow with
> commit [2] by avoiding setting NO-CARRIER on empty bridges, but I
> couldn't reproduce this behavior on my kernel. Anyway, with the “dummy
> tap to set MAC address” trick, this wouldn't work.
> 
> To fix this, the idea is to get the bridge's attached device to be up so
> that DAD can happen (deactivating DAD altogether is not a good idea, I
> think). Currently, libvirt creates a dummy TAP device to set the MAC
> address of the bridge, keeping it down. But even if we set this device
> up, it is not RUNNING as soon as the tap file descriptor attached to it
> is closed, thus still preventing DAD. So, we must modify the API a bit,
> so that we can get the fd, keep the tap device persistent, run the
> daemons, and close it after DAD has taken place. After that, the bridge
> will be flagged NO-CARRIER again, but the daemons will be running, even
> if not happy about the device's state (but we don't really care about
> the bridge's daemons doing anything when no up interface is connected to
> it).
> 
> Other solutions that I envisioned were:
>       * Keeping the *-nic interface up: this would waste an fd for each
>         bridge during all its life. May be acceptable, I don't really
>         know.

Is that because closing the TAP device FD makes it go into offline
state ?  In the great scheme of things, I don't think one extra
file descriptor per network is too onerous - there would only ever
be a handful of networks per host.

>       * Stop using the dummy tap trick, and set the MAC address directly
>         on the bridge: it is possible since quite some time it seems,
>         even if then there is the problem of the bridge not being
>         RUNNING when empty, contrary to what [2] says, so this will need
>         fixing (and this fix only happened in 3.1, so it wouldn't work
>         for 2.6.39)

While you can set the MAC address on a bridge, there are some
problems with doing so. The MAC address you set on the bridge
must match the MAC address of one of the enslaved interfaces
or no traffic will flow. We can't use the MAC addr of any of
the guest TAP devices, so we must have a dummy TAP device
present in the bridge.

>       * Using the --interface option of dnsmasq, but I saw somewhere
>         that it's not used by libvirt for backward compatibility. I am
>         not sure this would solve this problem, though, as I don't know
>         how dnsmasq binds itself to it with this option.

I don't know either.


> This is why this patch does what's described earlier. I see radvd
> yelling quite often in the logs when the interface is NO-CARRIER, but
> it's ok, it keeps running.
> 
> Still, this patch is not exactly correct, as radvd get daemonized “too
> soon” most of the time (i.e. when not debugging libvirtd…) and probes
> the bridge once it has been set down (even if started before setting it
> down), thus failing as before (and libvirt lets it be like that: this
> would need some more checking, maybe). One /may/ introduce some delay
> between networkStartRadvd() and setting the dummy tap down to solve it,
> but it seemed too hackish to me. But I couldn't come with a better
> solution. I would welcome suggestions here.

Introducing delays is indeed hackish & potentially unreliable if
starting a network on a highly loaded system. The only real way
to avoid that race would be to leave the dummy tap device in the
online state permanently AFAICT.


Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|