[libvirt] increase daemon task limit

[Daniel P. Berrange]

On Wed, Apr 19, 2017 at 01:11:41PM -0600, Jim Fehlig wrote:
> On 04/19/2017 09:16 AM, Daniel P. Berrange wrote:
> > On Wed, Apr 19, 2017 at 09:03:55AM -0600, Jim Fehlig wrote:
> > > I recently received a report of a libvirt+Xen installation reaching the pids
> > > cgroup controller TasksMax limit
> > > 
> > > kernel: [71282.213347] cgroup: fork rejected by pids controller in
> > > /system.slice/libvirtd.service
> > > 
> > > The default setting of TaskMax is 512 on this system
> > > 
> > > # cat /sys/fs/cgroup/pids/system.slice/libvirtd.service/pids.max
> > > 512
> > > 
> > > Depending on domain type and configuration, I've noticed between 5-7 tasks
> > > (IO threads, qemu process, etc.) are created when starting a domain. It
> > > doesn't take too many domains before the 512 limit is reached.
> > > 
> > > LimitNOFILE was recently changed by commit 27cd763500 to support 4096
> > > domains. Following similar logic, would it be ok to increase TasksMax to
> > > 32768? That would accommodate 4096 domains with 8 tasks each. TasksMax also
> > > supports the special value of "infinity", but that seems a bit aggressive to
> > > me.
> > 
> > KVM uses 1 thread per vCPU, so 8 pids is probably somewhat on the low
> > side if considering high vcpu counts. I think we could go for something
> > like 128k pids which allows for closer to 32 vcpus per guest
> 
> It looks like the vcpu threads are under
> /sys/fs/cgroup/pids/machine.slice/machine-qemu<name>kvm.scope/, where
> pids.max is already 16k. I don't think these would be constrained by
> attributes under /sys/fs/cgroup/pids/system.slice/libvirtd.service. In fact,
> I don't see any additional tasks under
> /sys/fs/cgroup/pids/system.slice/libvirtd.service/tasks when starting a KVM
> guest.
> 
> Since the libxl library runs in libvirtd process, any threads/processes it
> creates are accounted for in
> /sys/fs/cgroup/pids/system.slice/libvirtd.service. We are probably safe with
> the more conservative 32k.

Oh I see, I didn't realize it was a xen specific issue. I agree with your
suggestion


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|