[Fedora-directory-users] FDS SSL performance tuning query

[Richard Megginson]

Jonathan Barber wrote:
> Hello all, currently we have a FDS instance running on RHEL4 with a
> small number of entries (6,000), we also have a linux compute cluster of
> 100 nodes which uses LDAP for user account data (via libnss_ldap).
>
> nss_ldap on the cluster is configured to use SSL, and everything is fine
> most of the time. However, occasionally, when a large job is started on
> the cluster, the number of connections increases from 100/minute to
> 1600/minute (26/sec).
>
> This causes the server to become generally unresponsive, and FDS
> especially so (as judged by the time required to retrieve the DSE via
> TLS). Which is a right pain as it causes our samba PDC to timeout and
> everything goes wrong very quickly.
>
> I can reproducably, impact on FDS performance by running:
> $ getent passwd | cut -d: -f 1 | while read i; do id $i; done
>
> across the cluster. When SSL is off, the command to run fine and doesn't
> impact on other searches.
>
> As a short term measure, we've disabled LDAPS on the cluster nodes,
> which is fine as users don't log into them, but we had planned to expand
> the use of LDAP to cover more hosts (Macs and Linux) that require a 
> confidential channal for authentication. So this experience is giving us
> some trepidation about moving forward with that plan.
>
> Our system is configured following the guidance of the wiki [0], with a
> maximum of 16834 available file descriptors and 50M of cache (more than
> enough to hold the DB) - and the ratio of cache hits/misses look good
> with little paging out. Running logconv.pl on the access logs doesn't
> show any unindexed searches, so that isn't an issue.
>
> Our server CPU is a 3Ghz Xeon with 1G of RAM, and looking at the
> performance of NSS 3.2 [1], I would expect the machine to be able to
> setup and tear down many more connections than we are currently seeing.
> Indeed, running the test described in [1] with the nss-3.11.4 binaries,
> I get over 1200 connections per second [2], so it certainly doesn't seem
> to be a problem with NSS.
>
> This suggests to me that the problem lies in FDS somewhere. So, does
> anyone have any suggestions as to how to improve the SSL/TLS performance
> of FDS, or point me at tuning docs for the SSL side of FDS?
>   
I don't know.  But opening and closing SSL connections is pretty 
expensive, with all of the TLS/SSL protocol operations.  Is it possible 
you could configure the client machines to use LDAP (not LDAPS) and use 
the LDAP startTLS operation to start up the TLS session on the 
non-secure port?  This might allow the server to process the connection 
+ TLS session creation more efficiently.
> Cheers.
>
> [0] http://directory.fedoraproject.org/wiki/Performance_Tuning
> [1] http://www.mozilla.org/projects/security/pki/nss/nss-3.2-performance-results
> [2] server$ ./selfserv -n "Server-Cert" -p 6000
>     client$ time ./strsclnt -p 6000 server -c 1000
>     strsclnt: -- SSL: Server Certificate Validated.
>     strsclnt: 0 cache hits; 1 cache misses, 0 cache not reusable
>     strsclnt: 999 cache hits; 1 cache misses, 0 cache not reusable
>
>     real    0m0.605s
>     user    0m0.795s
>     sys     0m0.226s
>   

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3245 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/fedora-directory-users/attachments/20070807/a4344804/attachment.bin>