[Fedora-directory-users] FDS SSL performance tuning query

[Richard Megginson]

Jonathan Barber wrote:
> On Tue, Aug 07, 2007 at 10:26:46AM -0600, Richard Megginson wrote:
>   
>> Jonathan Barber wrote:
>>     
>>> Hello all, currently we have a FDS instance running on RHEL4 with a
>>> small number of entries (6,000), we also have a linux compute cluster of
>>> 100 nodes which uses LDAP for user account data (via libnss_ldap).
>>>
>>> nss_ldap on the cluster is configured to use SSL, and everything is fine
>>> most of the time. However, occasionally, when a large job is started on
>>> the cluster, the number of connections increases from 100/minute to
>>> 1600/minute (26/sec).
>>>
>>> This causes the server to become generally unresponsive, and FDS
>>> especially so (as judged by the time required to retrieve the DSE via
>>> TLS). Which is a right pain as it causes our samba PDC to timeout and
>>> everything goes wrong very quickly.
>>>
>>> I can reproducably, impact on FDS performance by running:
>>> $ getent passwd | cut -d: -f 1 | while read i; do id $i; done
>>>
>>> across the cluster. When SSL is off, the command to run fine and doesn't
>>> impact on other searches.
>>>
>>> As a short term measure, we've disabled LDAPS on the cluster nodes,
>>> which is fine as users don't log into them, but we had planned to expand
>>> the use of LDAP to cover more hosts (Macs and Linux) that require a 
>>> confidential channal for authentication. So this experience is giving us
>>> some trepidation about moving forward with that plan.
>>>
>>> Our system is configured following the guidance of the wiki [0], with a
>>> maximum of 16834 available file descriptors and 50M of cache (more than
>>> enough to hold the DB) - and the ratio of cache hits/misses look good
>>> with little paging out. Running logconv.pl on the access logs doesn't
>>> show any unindexed searches, so that isn't an issue.
>>>
>>> Our server CPU is a 3Ghz Xeon with 1G of RAM, and looking at the
>>> performance of NSS 3.2 [1], I would expect the machine to be able to
>>> setup and tear down many more connections than we are currently seeing.
>>> Indeed, running the test described in [1] with the nss-3.11.4 binaries,
>>> I get over 1200 connections per second [2], so it certainly doesn't seem
>>> to be a problem with NSS.
>>>
>>> This suggests to me that the problem lies in FDS somewhere. So, does
>>> anyone have any suggestions as to how to improve the SSL/TLS performance
>>> of FDS, or point me at tuning docs for the SSL side of FDS?
>>>  
>>>       
>> I don't know.  But opening and closing SSL connections is pretty 
>> expensive, with all of the TLS/SSL protocol operations.  Is it possible 
>> you could configure the client machines to use LDAP (not LDAPS) and use 
>> the LDAP startTLS operation to start up the TLS session on the 
>> non-secure port?  This might allow the server to process the connection 
>> + TLS session creation more efficiently.
>>     
>
> I'll give it a go and see how it works. I had assumed SSL would be less
> expensive than a start TLS.
>   
It might be in toto, but if you use startTLS you at least spread out the 
expense, create TCP connection and resources first, then TLS handshake 
and session resources.
> Do you have any benchmarks (even rough numbers) available as to how many
> connections FDS can copes with TLS/SSL vs. plain LDAP? I've read Howard
> Chu's presentation (http://highlandsun.com/hyc/SambaXP.pdf) but it
> doesn't compare against SSL, and I didn't do any SSL benchmarks with FDS
> when I evaluted LDAP servers. I don't have any real feeling as to how
> many TLS/SSL connection you get compared to plain TCP/IP.
>   
We don't have anything like that.
> Ta.
>
>   
>>> Cheers.
>>>
>>> [0] http://directory.fedoraproject.org/wiki/Performance_Tuning
>>> [1] 
>>> http://www.mozilla.org/projects/security/pki/nss/nss-3.2-performance-results
>>> [2] server$ ./selfserv -n "Server-Cert" -p 6000
>>>    client$ time ./strsclnt -p 6000 server -c 1000
>>>    strsclnt: -- SSL: Server Certificate Validated.
>>>    strsclnt: 0 cache hits; 1 cache misses, 0 cache not reusable
>>>    strsclnt: 999 cache hits; 1 cache misses, 0 cache not reusable
>>>
>>>    real    0m0.605s
>>>    user    0m0.795s
>>>    sys     0m0.226s
>>>  
>>>       
>
>
>
>   
>> --
>> Fedora-directory-users mailing list
>> Fedora-directory-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>>     
>
>
>   

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3245 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/fedora-directory-users/attachments/20070807/ab6a8564/attachment.bin>