[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [libvirt] RFC: put domain's interfaces into distinct namespaces

On 07.11.2018 11:48, Nikolay Shirokovskiy wrote:
> Hi, all!
> There is performance issue with network filters and broadcast ethernet traffic.
> If L2 segment is large enough (several thousands of VMs) then there is a lot of
> broadcast ARP traffic (about frames 100/s). As aresult on host with several hundreds
> VMs (say 300) we have kernel thread eating 100% of CPUs just for checking this traffic
> against firewall rules. The problem is if there are rules in ebtables POSTROUTING chain
> (clean-traffic is example of such filter) then when every single broadcast frame turns into
> 300, one for every distinct bridge port and then each one of these 300 is checked against
> 300 / 2 rules average to find chain for that port. As a result we have 100 * 300 * 300 / 2
> = 4.5 * 10^6 rules checks per second. Kernel does not spread this workload onto
> different CPUs and anyway this is wasting CPUs!
> The simple solution is to put rules that ACCEPT ARP traffic into POSTROUTING
> itself before any port specific chains. But this will affect non-VM ports too 
> and host itself. So can we instead make a distinct network namespace for every
> VM and put tap there, next add the bridge into the namespace too so we can apply
> ebtables rules there and insert tap into the bridge. Finally connect the bridges
> in root namespace and VM namespace by veth pair. As result in the situation
> described above each cloned frame will be cheched only againt rules for this
> very VM. The regular TCP traffic will have same benefits. On the other hand we
> need a bridge and veth pair for every VM and some CPU power to process this extra
> traffic path.
> The proposed approach also fixes the problem of slow libvirtd restarting with
> network filters ([1], [2]) as it is rather difficult to mess network rules in
> different network namespace, at least restarting/reloading firewalld won't
> hurt such rules so we just don't need to reinstantiate rules at all.

Not true. I forgot sometimes libvirt updates can bring updated filters definitions
and/or daemon code that instantiates them so we still need reinstantiate on
restart generally. In this case we can use approach of [2] or [3] which one
is prefearable.

[3] [PATCH] nwfilter: intantiate filters on loading driver


> [1] [RFC] Faster libvirtd restart with nwfilter rules
> https://www.redhat.com/archives/libvir-list/2018-September/msg01206.html
>   which continues in
> https://www.redhat.com/archives/libvir-list/2018-October/msg00657.html
> [2] [PATCH v2 0/2] nwfilter: don't reinstantiate rules if there is no need to
> https://www.redhat.com/archives/libvir-list/2018-October/msg01317.html
> Nikolay
> --
> libvir-list mailing list
> libvir-list redhat com
> https://www.redhat.com/mailman/listinfo/libvir-list

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]