[libvirt-users] python-libvirt for 0.9.3 leaking file descriptors

[Daniel P. Berrange]

On Tue, Jul 26, 2011 at 06:29:58PM +0200, Matthias Bolte wrote:
> 2011/7/26 Igor Serebryany <igor47 at moomers.org>:
> > i've reported this issue before, so i guess this is a regression. looks
> > like the python bindings for 0.9.3 are leaking file descriptors:
> >
> > root at cloud1:~# python
> > Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
> > [GCC 4.4.5] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> >>>> import libvirt
> >>>> libvirt.getVersion()
> > 9003
> >>>> def openclose():
> > ...     c = libvirt.open('qemu:///system')
> > ...     c.close()
> > ...
> >>>> import os
> >>>> os.system('lsof -p %d | wc -l' % os.getpid())
> > 37
> >>>> openclose()
> >>>> os.system('lsof -p %d | wc -l' % os.getpid())
> > 40
> >>>> openclose()
> >>>> os.system('lsof -p %d | wc -l' % os.getpid())
> > 43
> >
> > every time I open and close a connection to libvirt, i loose 3 file
> > descriptors. these are a socket and two fifo pipes. i'm using the -3
> > build on debian sid.
> >
> > can someone test this on a fedora machine? curious if this is something
> > inside libvirt or a problem with the build...
> 
> I can reproduce this with libvirt 0.9.3, but not with current git.
> Therefore, this problem seems to be fixed in git. I've you want to
> backport the fix to libvirt 0.9.3 you should be able to find it using
> git bisect.
> 
> That one of the FDs is a socket suggests that this is related to the
> remote driver.

Yes, it wasn't a python bug, its a libvirt RPC client bug


commit afe8839f011c8c54c429f33ca0e6515fceb4e0fd
Author: Daniel P. Berrange <berrange at redhat.com>
Date:   Fri Jul 8 12:41:06 2011 +0100

    Fix leak of remote driver if final 'CLOSE' RPC call fails
    
    When closing a remote connection we issue a (fairly pointless)
    'CLOSE' RPC call to the daemon. If this fails we skip all the
    cleanup of private data, but the virConnectPtr object still
    gets released as normal. This causes a memory leak. Since the
    CLOSE RPC call is pretty pointless, just carry on freeing the
    remote driver if it fails.
    
    * src/remote/remote_driver.c: Ignore failure to issue CLOSE
      RPC call


which was in turn exposed by a libvirtd bug


commit 3cfdc57b8553cae95b8849bbcb7a4b227085cec1
Author: Daniel P. Berrange <berrange at redhat.com>
Date:   Fri Jul 8 12:54:29 2011 +0100

    Fix sending of reply to final RPC message
    
    The dispatch for the CLOSE RPC call was invoking the method
    virNetServerClientClose(). This caused the client connection
    to be immediately terminated. This meant the reply to the
    final RPC message was never sent. Prior to the RPC rewrite
    we merely flagged the connection for closing, and actually
    closed it when the next RPC call dispatch had completed.

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|