[libvirt] [PATCH 0/9] Resolve libvirtd hang on termination with connected long running client

[Marc Hartmayer]

On Tue, Jan 23, 2018 at 05:41 PM +0100, Marc Hartmayer <mhartmay at linux.vnet.ibm.com> wrote:
> On Tue, Jan 23, 2018 at 04:01 PM +0100, John Ferlan <jferlan at redhat.com> wrote:
>> On 01/23/2018 04:21 AM, Marc Hartmayer wrote:
>>> On Fri, Jan 19, 2018 at 06:23 PM +0100, John Ferlan <jferlan at redhat.com> wrote:
>>>> RFC:
>>>> https://www.redhat.com/archives/libvir-list/2018-January/msg00318.html
>>>>
>>>> Adjustments since RFC...
>>>>
>>>> Patches 1&2: No change, were already R-B'd
>>>> Patch 3: Removed code as noted in code review, update commit message
>>>> Patch 4: From old series removed, see below for more details
>>>> Patch 9: no change
>>>> NB: Patches 5-8 and 10 from Nikolay Shirokovskiy <nshirokovskiy at virtuozzo.com>
>>>>     are removed as they seemed to not be necessary
>>>>
>>>> Replaced the former patch 4 with series of patches to (slowly) provide
>>>> support to disable new connections, handle removing waiting jobs, causing
>>>> the waiting workers to quit, and allow any running jobs to complete.
>>>>
>>>> As it turns out, waiting for running jobs to complete cannot be done
>>>> from the virNetServerClose callbacks because that means the event loop
>>>> processing done during virNetServerRun will not allow any currently
>>>> long running worker job thread a means to complete.
>>>>
>>>> So when virNetDaemonQuit is called as a result of the libvirtd signal
>>>> handlers for SIG{QUIT|INT|TERM}, instead of just causing virNetServerRun
>>>> to quit immediately, alter to using a quitRequested flag and then use
>>>> that quitRequested flag to check for long running worker threads before
>>>> causing the event loop to quit resulting in libvirtd being able to run
>>>> through the virNetDaemonClose processing.
>>>
>>> Gave a quick test:
>>>  + Didn't get a segmentation fault at the end of libvirtd (at least for
>>>    my quick test)
>>>  - a single SIGTERM doesn’t always lead to the termination of libvirtd
>>>    now (debugged it: main thread is waiting for poll()). This behavior
>>>    can be easily reproduced: Start libvirtd on the CLI, wait some
>>>    seconds for the first initialization -> CTRL + C -> libvirtd doesn’t
>>>    terminate, but also doesn’t accept new connections.
>>>
>>
>> Does this include the "update" in my response to patch 7?  It should be
>> extract-able and apply-able.
>
> No, I forgot this one :( I’ll try it tomorrow.

It works with the new patch 7.

[…snip]

--
Beste Grüße / Kind regards
   Marc Hartmayer

IBM Deutschland Research & Development GmbH
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294