<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40" xmlns:ns0="urn:schemas-microsoft-com:office:smarttags"> <head> <meta http-equiv=Content-Type content="text/html; charset=us-ascii"> <meta name=Generator content="Microsoft Word 11 (filtered medium)"> <o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="State"/> <o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PlaceName"/> <o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="place"/> <o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="City"/>  <style>  </style> </head> <body lang=EN-GB link=blue vlink=blue> <div class=Section1> Operating system?<o:p></o:p> <o:p> </o:p> CPU and Ram look good….<o:p></o:p> <o:p> </o:p> Iostat?<o:p></o:p> <o:p> </o:p> Hi,<o:p></o:p> <o:p> </o:p> Where is your basic fault finding data? <o:p></o:p> <o:p> </o:p> We run a database backend for spam control and find that cleaning up / indexing the database has dramatic effects….like after 6 months our Dell 6850 is screwed….clean it up and its disk i/o is 2% again….but this shows up under iostat….using mrtg you can see the graph climbing steadily over the months…..<o:p></o:p> <o:p> </o:p> A pair of SATA disks (I assume raid 1) is not very fast (on board raid?...shudder…)….also these are SATA, SATA sucks for random i/o and guess what you have a database doing random i/o……..<o:p></o:p> <o:p> </o:p> In terms of code, yes it is often the code at fault. Somehow developers who have written sucky code expect sys admins to spend serious time and money on hardware compensating for their bad code….it does not work.<o:p></o:p> <o:p> </o:p> This could be the hard thing to prove….ie….is your disk i/o inadequate or is their code so bad its causing the i/o!<o:p></o:p> <o:p> </o:p> If its disk i/o……<o:p></o:p> <o:p> </o:p> Generally, you are going to be spending most of your time reading from disk….so you need to be optimising for reads….raid5 is ideal but testing will prove this….LDAP can be distributed over disk sets…so would 4 disks in two raid1s out perform a R5 3+1? From my experience a r5 3+1 for databases is 20% faster than raid 1s….<o:p></o:p> <o:p> </o:p> I suspect you are very budget conscious and have no-name white boxes….so articles like this,<o:p></o:p> <o:p> </o:p> <a href="http://www.tomshardware.com/2007/11/30/more_serial_raid_controllers_from_amcc/">http://www.tomshardware.com/2007/11/30/more_serial_raid_controllers_from_amcc/</a><o:p></o:p> <o:p> </o:p> Can give you good pointers as what to get….generally avoid SATA, look at SAS….for databases the LSIMegaraid SAS 8888ELP in a raid 5 looks worth buying….maybe a very small disk strip is in order so small raid5 sets….<o:p></o:p> <o:p> </o:p> You are ahead of me at present I’m still piloting FDS….<o:p></o:p> <o:p> </o:p> Regards<o:p></o:p> <o:p> </o:p> <div> Steven Jones Senior Linux/Unix/San/Vmware System Administrator APG -Technology Integration Team <ns0:PlaceName w:insAuthor="Full Name" w:insDate="2007-12-14T09:06:00Z" w:endInsAuthor="Full Name" w:endInsDate="2007-12-14T09:06:00Z"><st1:PlaceName w:st="on">Victoria</st1:PlaceName></ns0:PlaceName> <ns0:PlaceName w:insAuthor="Full Name" w:insDate="2007-12-14T09:06:00Z" w:endInsAuthor="Full Name" w:endInsDate="2007-12-14T09:06:00Z"><st1:PlaceName w:st="on">University</st1:PlaceName></ns0:PlaceName> of <ns0:place w:insAuthor="Full Name" w:insDate="2007-12-14T09:06:00Z" w:endInsAuthor="Full Name" w:endInsDate="2007-12-14T09:06:00Z"><ns0:City w:insAuthor="Full Name" w:insDate="2007-12-14T09:06:00Z" w:endInsAuthor="Full Name" w:endInsDate="2007-12-14T09:06:00Z"><st1:place w:st="on"><st1:City w:st="on">Wellington</st1:City></st1:place></ns0:City></ns0:place> Phone: +64 4 463 6272<o:p></o:p> </div> <div> <div class=MsoNormal align=center style='text-align:center'> <hr size=2 width="100%" align=center tabindex=-1> </div> From: fedora-directory-users-bounces@redhat.com [mailto:fedora-directory-users-bounces@redhat.com] On Behalf Of Jared B. Griffith Sent: Friday, 14 December 2007 9:05 a.m. To: Chris G. Sellers Cc: General discussion list for the Fedora Directory server project. Subject: Re: [Fedora-directory-users] LDAP Accounts for large website<o:p></o:p> </div> <o:p> </o:p> Here are the specs on the server: 2 x 2.0Ghz Intel Dual Core Xeon 4 x 1Gb Registered ECC RAM 2 x 74Gb Western Digital Raptors Given that, the hardware issue should be more than sufficient. Network connections are Gigabit full duplex throughout our cage network. How would we go about indexing the attributes? Me and another sys admin have a distinct feeling that it is an issue with the query, but they are going to point blame at us, so we want to make sure that we are golden before saying it's the code. ----- Original Message ----- From: "Chris G. Sellers" <chris.sellers@nitle.org> To: "Jared B. Griffith" <jared.griffith@farheap.com>, "General discussion list for the Fedora Directory server project." <fedora-directory-users@redhat.com> Sent: Thursday, December 13, 2007 11:47:35 AM (GMT-0800) America/Los_Angeles Subject: Re: [Fedora-directory-users] LDAP Accounts for large website<o:p></o:p> <div> Performance is often impacted greatly by<o:p></o:p> <div> <o:p> </o:p> </div> <div> 1) Memory on the LDAP server. Make sure you can store as much of your directory data store in RAM for fast access<o:p></o:p> </div> <div> 2) Indexing. Make sure attributes that you search on freqently are indexed. Also, limit what fields you search on to avoid having a heavy indexing tax.<o:p></o:p> </div> <div> 3) Make sure your network connections are stable, and your not connecting on a 100MB half duplex connection while your network equiptment is expecting a full duplex connection.<o:p></o:p> </div> <div> <o:p> </o:p> </div> <div> Once you have auditing those situations, please check your performance again.<o:p></o:p> </div> <div> <o:p> </o:p> </div> <div> Sellers<o:p></o:p> </div> <div> <o:p> </o:p> </div> <div> 50k accounts is not that much, and a 2GHz Pentium Class or 1.5GHz Core 2 system with 1GB-2GB of RAM should perform okay. <o:p></o:p> </div> <div> <o:p> </o:p> </div> <div> <o:p> </o:p> </div> <div> <div> <div> On Dec 13, 2007, at 2:36 PM, Jared B. Griffith wrote:<o:p></o:p> </div> <blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'> <o:p> </o:p> <div> I was wondering if anyone here has ever used LDAP for a website, that will potentially have millions of LDAP accounts. If so, are you experiencing slow query responses or other issues? If you were experiencing slow query responses, and were able to rectify the issue, how did you do this? We are currently using FDS for our main website for customer accounts. We currently have over 52,000 accounts in LDAP and have only been using this for 3 months. We are now experiencing extreme slow down in query response when getting customer data into and out of the LDAP servers. Any help would be greatly appreciated. -- - Thank you, - Jared B. Griffith - Farheap Solutions, Inc. - Lead Systems Administrator - <st1:State w:st="on"><st1:place w:st="on">California</st1:place></st1:State> IT Department - Email - <a href="mailto:jared.griffith@farheap.com" target="_blank">jared.griffith@farheap.com</a> - Phone - 949.417.1500 ext. 266 - Cell Phone - 949.910.6542 -- Fedora-directory-users mailing list <a href="mailto:Fedora-directory-users@redhat.com" target="_blank">Fedora-directory-users@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/fedora-directory-users" target="_blank">https://www.redhat.com/mailman/listinfo/fedora-directory-users</a><o:p></o:p> </div> </blockquote> </div> <o:p> </o:p> <div> <div> <div> ______________________________________________ Chris G. Sellers | NITLE Technology<o:p></o:p> </div> <div> 734.661.2318 | <a href="mailto:chris.sellers@nitle.org" target="_blank">chris.sellers@nitle.org</a><o:p></o:p> </div> <div> AIM: imthewherd | GTalk: <a href="mailto:cgseller@gmail.com" target="_blank">cgseller@gmail.com</a><o:p></o:p> </div> </div> </div> <o:p> </o:p> </div> </div> -- - Thank you, - Jared B. Griffith - Farheap Solutions, Inc. - Lead Systems Administrator - <st1:State w:st="on"><st1:place w:st="on">California</st1:place></st1:State> IT Department - Email - jared.griffith@farheap.com - Phone - 949.417.1500 ext. 266 - Cell Phone - 949.910.6542<o:p></o:p> </div> </body> </html>