[Freeipa-devel] Unit tests failing on F18

[Martin Kosek]

On 10/12/2012 06:16 PM, John Dennis wrote:
> On 10/12/2012 11:20 AM, Martin Kosek wrote:
>> Hello,
>>
>> I was investigating global unit test failure on Fedora 18 for most of today, I
>> would like to share results I found so far.
>>
>> Unit test and its related scripts on F18 now reports NSS BUSY exception, just
>> like this one:
>>
>> # ./make-testcert
>> Traceback (most recent call last):
>>    File "./make-testcert", line 134, in <module>
>>      sys.exit(makecert(reqdir))
>>    File "./make-testcert", line 111, in makecert
>>      add=True)
>>    File "./make-testcert", line 68, in run
>>      result = self.execute(method, *args, **options)
>>    File "/root/freeipa-master2/ipalib/backend.py", line 146, in execute
>>      raise error #pylint: disable=E0702
>> ipalib.errors.NetworkError: cannot connect to
>> 'http://vm-042.idm.lab.bos.redhat.com/ipa/session/xml': [Errno -8053]
>> (SEC_ERROR_BUSY) NSS could not shutdown. Objects are still in use.
>>
>> Something In F18 must have changed, this worked before... But leaked
>> NSSConnection objects without proper close() now ends with the exception above.
>>
>> In case of make-testcert script, the exception is raised because the script
>> does the following procedure:
>>
>> 1) connect, do one command
>> 2) disconnect
>> 3) connect, do second command
>>
>> However, during disconnect, NSSConnection is leaked which makes NSS very
>> uncomfortable during second connection atempt (and nss_shutdown()). I managed
>> to fix this issue with attached patch. ./make-testcert or "./make-test
>> tests/test_xmlrpc/test_group_plugin.py" works fine now.
>>
>> But global "./make-test" still fails, I think there is some remaining
>> NSSConnection leak, I suspect there is something wrong with how we use our
>> context (threading.local object). It looses a connection or some other thread
>> invoked in ldap2 module may be kicking in, here is my debug output:
>>
>> CONTEXT[xmlclient] = <ipalib.request.Connection object at 0x9a1f5ec>
>>
>> Test a simple LDAP bind using ldap2 ... SKIP: No directory manager password in
>> /root/.ipa/.dmpw
>> Test the `ipaserver.rpcserver.jsonserver.unmarshal` method. ... ok
>> tests.test_ipaserver.test_rpcserver.test_session.test_mount ... CONTEXT
>> 150714476: GET languages
>>
>> CONTEXT[xmlclient] = None
>>
>> The connection is in the context, but then something happens and it is gone.
>> Then, unit tests try to connect again and NSS fails.
>>
>> I would be really glad if somebody with a knowledge of NSS or how threads in
>> Python/IPA work could give me some advice...
> 
> O.K. I'll take a look at it. I seem to recall Rob looked into something similar
> a couple of days ago. Rob, do you have any additional information to share?

Great, with your NSS+Python knowledge this should be walk in the park :-)
 Any luck with investigation of this issue?

Thanks,
Martin