[Fedora-directory-users] FDS SSL performance tuning query

[Jonathan Barber]

On Tue, Aug 07, 2007 at 10:26:46AM -0600, Richard Megginson wrote:
> Jonathan Barber wrote:
> >Hello all, currently we have a FDS instance running on RHEL4 with a
> >small number of entries (6,000), we also have a linux compute cluster of
> >100 nodes which uses LDAP for user account data (via libnss_ldap).
> >
> >nss_ldap on the cluster is configured to use SSL, and everything is fine
> >most of the time. However, occasionally, when a large job is started on
> >the cluster, the number of connections increases from 100/minute to
> >1600/minute (26/sec).
> >
> >This causes the server to become generally unresponsive, and FDS
> >especially so (as judged by the time required to retrieve the DSE via
> >TLS). Which is a right pain as it causes our samba PDC to timeout and
> >everything goes wrong very quickly.
> >
> >I can reproducably, impact on FDS performance by running:
> >$ getent passwd | cut -d: -f 1 | while read i; do id $i; done
> >
> >across the cluster. When SSL is off, the command to run fine and doesn't
> >impact on other searches.
> >
> >As a short term measure, we've disabled LDAPS on the cluster nodes,
> >which is fine as users don't log into them, but we had planned to expand
> >the use of LDAP to cover more hosts (Macs and Linux) that require a 
> >confidential channal for authentication. So this experience is giving us
> >some trepidation about moving forward with that plan.
> >
> >Our system is configured following the guidance of the wiki [0], with a
> >maximum of 16834 available file descriptors and 50M of cache (more than
> >enough to hold the DB) - and the ratio of cache hits/misses look good
> >with little paging out. Running logconv.pl on the access logs doesn't
> >show any unindexed searches, so that isn't an issue.
> >
> >Our server CPU is a 3Ghz Xeon with 1G of RAM, and looking at the
> >performance of NSS 3.2 [1], I would expect the machine to be able to
> >setup and tear down many more connections than we are currently seeing.
> >Indeed, running the test described in [1] with the nss-3.11.4 binaries,
> >I get over 1200 connections per second [2], so it certainly doesn't seem
> >to be a problem with NSS.
> >
> >This suggests to me that the problem lies in FDS somewhere. So, does
> >anyone have any suggestions as to how to improve the SSL/TLS performance
> >of FDS, or point me at tuning docs for the SSL side of FDS?
> >  
>
> I don't know.  But opening and closing SSL connections is pretty 
> expensive, with all of the TLS/SSL protocol operations.  Is it possible 
> you could configure the client machines to use LDAP (not LDAPS) and use 
> the LDAP startTLS operation to start up the TLS session on the 
> non-secure port?  This might allow the server to process the connection 
> + TLS session creation more efficiently.

I'll give it a go and see how it works. I had assumed SSL would be less
expensive than a start TLS.

Do you have any benchmarks (even rough numbers) available as to how many
connections FDS can copes with TLS/SSL vs. plain LDAP? I've read Howard
Chu's presentation (http://highlandsun.com/hyc/SambaXP.pdf) but it
doesn't compare against SSL, and I didn't do any SSL benchmarks with FDS
when I evaluted LDAP servers. I don't have any real feeling as to how
many TLS/SSL connection you get compared to plain TCP/IP.

Ta.

> >Cheers.
> >
> >[0] http://directory.fedoraproject.org/wiki/Performance_Tuning
> >[1] 
> >http://www.mozilla.org/projects/security/pki/nss/nss-3.2-performance-results
> >[2] server$ ./selfserv -n "Server-Cert" -p 6000
> >    client$ time ./strsclnt -p 6000 server -c 1000
> >    strsclnt: -- SSL: Server Certificate Validated.
> >    strsclnt: 0 cache hits; 1 cache misses, 0 cache not reusable
> >    strsclnt: 999 cache hits; 1 cache misses, 0 cache not reusable
> >
> >    real    0m0.605s
> >    user    0m0.795s
> >    sys     0m0.226s
> >  
> 



> --
> Fedora-directory-users mailing list
> Fedora-directory-users at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users


-- 
Jonathan Barber
High Performance Computing Analyst
Tel. +44 (0) 1382 386389