From data_mine at hotmail.com Tue Sep 20 03:40:01 2005 From: data_mine at hotmail.com (Data Mine) Date: Tue, 20 Sep 2005 03:40:01 +0000 Subject: Splunk Search Message-ID: Posted for a friend... Splunk server can be downloaded for free and allows you to search linux/solaris log files... http://www.splunk.com/?ac=secret From maheshs at cdac.in Thu Sep 22 12:09:59 2005 From: maheshs at cdac.in (Mahesh Shinde) Date: Thu, 22 Sep 2005 17:39:59 +0530 (India Standard Time) Subject: Performance problem (high Load average) in Mail Server Message-ID: We have a Mail Server with following Hardware/Software configuration. IBM X-345 Intel-based Server DUAL Intel Xeon 2.6 GHz 2.5 GB RAM 3 Nos. 72 GB HDD OS: Red Hat Enterprise Linux 3 AS Update 5 kernel: 2.4.21-32.0.1.ELsmp We are regularly updating the recommended patches from Red Hat Network on Mail Server and it was working fine. But from last 15 days we are facing severe load problems due to which server gets busy and user experencing low response/time-outs while accessing mails from their email clients. We observed that the busiest process is usually kscand, and overall kscand and kswapd have the highest CPU utilization times. As per recent test the server perform well after initial boot, but after some time only 20 MB of free memory space left out of total 2.5 GB RAM and around 1 GB swap space get used out of 2 GB of swap memory. After that kscand gets much busier and CPU load seems to increased wildly. Please provide us solution to this critical problem at the earliest. Regards, Mahesh Shinde C-DAC, Pune From jrumpelein at gmail.com Thu Sep 22 12:19:55 2005 From: jrumpelein at gmail.com (John Rumpelein) Date: Thu, 22 Sep 2005 05:19:55 -0700 Subject: Performance problem (high Load average) in Mail Server In-Reply-To: References: Message-ID: What's eating all the RAM? There's a whitepaper on converting to dovecot from uw-imapd (is RHEL3 uw-imapd based?) which describes performance issues like this. On 9/22/05, Mahesh Shinde wrote: > > We have a Mail Server with following Hardware/Software configuration. > > IBM X-345 Intel-based Server > DUAL Intel Xeon 2.6 GHz > 2.5 GB RAM > 3 Nos. 72 GB HDD > > OS: Red Hat Enterprise Linux 3 AS Update 5 > kernel: 2.4.21-32.0.1.ELsmp > > We are regularly updating the recommended patches from Red Hat Network on > Mail Server and it was working fine. But from last 15 days we are facing > severe load problems due to which server gets busy and user experencing > low response/time-outs while accessing mails from their email clients. > > We observed that the busiest process is usually kscand, and overall kscand > and kswapd have the highest CPU utilization times. > > As per recent test the server perform well after initial boot, but after > some time only 20 MB of free memory space left out of total 2.5 GB RAM and > around 1 GB swap space get used out of 2 GB of swap memory. After that > kscand gets much busier and CPU load seems to increased wildly. > > Please provide us solution to this critical problem at the earliest. > > Regards, > Mahesh Shinde > C-DAC, Pune > > -- > redhat-sysadmin-list mailing list > redhat-sysadmin-list at redhat.com > https://www.redhat.com/mailman/listinfo/redhat-sysadmin-list > From maheshs at cdac.in Thu Sep 22 13:03:36 2005 From: maheshs at cdac.in (Mahesh Shinde) Date: Thu, 22 Sep 2005 18:33:36 +0530 (India Standard Time) Subject: Performance problem (high Load average) in Mail Server In-Reply-To: References: Message-ID: Dear John, Yes RHEL3 is uw-imapd based and users also uses WebMail heavily and httpd process takes 25% of CPU resouces. Can you tell me how to check which user or program/application is eating all the RAM. The output of "free" command is as follows: -------------------------------------------------------------------------- # free total used free shared buffers cached Mem: 2565840 2547952 17888 0 37200 2269280 -/+ buffers/cache: 241472 2324368 Swap: 2048248 423556 1624692 ------------------------------------------------------------------------- Regards, Mahesh On Thu, 22 Sep 2005, John Rumpelein wrote: > What's eating all the RAM? > > There's a whitepaper on converting to dovecot from uw-imapd (is RHEL3 > uw-imapd based?) which describes performance issues like this. > > > On 9/22/05, Mahesh Shinde wrote: >> >> We have a Mail Server with following Hardware/Software configuration. >> >> IBM X-345 Intel-based Server >> DUAL Intel Xeon 2.6 GHz >> 2.5 GB RAM >> 3 Nos. 72 GB HDD >> >> OS: Red Hat Enterprise Linux 3 AS Update 5 >> kernel: 2.4.21-32.0.1.ELsmp >> >> We are regularly updating the recommended patches from Red Hat Network on >> Mail Server and it was working fine. But from last 15 days we are facing >> severe load problems due to which server gets busy and user experencing >> low response/time-outs while accessing mails from their email clients. >> >> We observed that the busiest process is usually kscand, and overall kscand >> and kswapd have the highest CPU utilization times. >> >> As per recent test the server perform well after initial boot, but after >> some time only 20 MB of free memory space left out of total 2.5 GB RAM and >> around 1 GB swap space get used out of 2 GB of swap memory. After that >> kscand gets much busier and CPU load seems to increased wildly. >> >> Please provide us solution to this critical problem at the earliest. >> >> Regards, >> Mahesh Shinde >> C-DAC, Pune >> >> -- >> redhat-sysadmin-list mailing list >> redhat-sysadmin-list at redhat.com >> https://www.redhat.com/mailman/listinfo/redhat-sysadmin-list >> > > -- > redhat-sysadmin-list mailing list > redhat-sysadmin-list at redhat.com > https://www.redhat.com/mailman/listinfo/redhat-sysadmin-list > From sysadmin at bssnews.org Thu Sep 22 13:27:19 2005 From: sysadmin at bssnews.org (System Administrator) Date: Thu, 22 Sep 2005 19:27:19 +0600 Subject: Performance problem (high Load average) in Mail Server References: Message-ID: <007401c5bf79$5d588ec0$2100a8c0@Sultan> One way to check is run the command 'top'. Then you will see all the programs running the amount of memory and cpu they are using. Type 'shift+m' to sort by memory usage on the top screen. However this is for Fedora. It may work for RHEL or it may not, there may be something similar on RHEL. Regards Sultanuzzaman Khan BSS, Dhaka. ----- Original Message ----- From: "Mahesh Shinde" To: "John Rumpelein" ; Sent: Thursday, September 22, 2005 7:03 PM Subject: Re: Performance problem (high Load average) in Mail Server > > Dear John, > > Yes RHEL3 is uw-imapd based and users also uses WebMail heavily and httpd > process takes 25% of CPU resouces. > > Can you tell me how to check which user or program/application is eating > all the RAM. > > The output of "free" command is as follows: > -------------------------------------------------------------------------- > # free > total used free shared buffers cached > Mem: 2565840 2547952 17888 0 37200 2269280 > -/+ buffers/cache: 241472 2324368 > Swap: 2048248 423556 1624692 > ------------------------------------------------------------------------- > > Regards, > Mahesh > > On Thu, 22 Sep 2005, John Rumpelein wrote: > >> What's eating all the RAM? >> >> There's a whitepaper on converting to dovecot from uw-imapd (is RHEL3 >> uw-imapd based?) which describes performance issues like this. >> >> >> On 9/22/05, Mahesh Shinde wrote: >>> >>> We have a Mail Server with following Hardware/Software configuration. >>> >>> IBM X-345 Intel-based Server >>> DUAL Intel Xeon 2.6 GHz >>> 2.5 GB RAM >>> 3 Nos. 72 GB HDD >>> >>> OS: Red Hat Enterprise Linux 3 AS Update 5 >>> kernel: 2.4.21-32.0.1.ELsmp >>> >>> We are regularly updating the recommended patches from Red Hat Network >>> on >>> Mail Server and it was working fine. But from last 15 days we are facing >>> severe load problems due to which server gets busy and user experencing >>> low response/time-outs while accessing mails from their email clients. >>> >>> We observed that the busiest process is usually kscand, and overall >>> kscand >>> and kswapd have the highest CPU utilization times. >>> >>> As per recent test the server perform well after initial boot, but after >>> some time only 20 MB of free memory space left out of total 2.5 GB RAM >>> and >>> around 1 GB swap space get used out of 2 GB of swap memory. After that >>> kscand gets much busier and CPU load seems to increased wildly. >>> >>> Please provide us solution to this critical problem at the earliest. >>> >>> Regards, >>> Mahesh Shinde >>> C-DAC, Pune >>> >>> -- >>> redhat-sysadmin-list mailing list >>> redhat-sysadmin-list at redhat.com >>> https://www.redhat.com/mailman/listinfo/redhat-sysadmin-list >>> >> >> -- >> redhat-sysadmin-list mailing list >> redhat-sysadmin-list at redhat.com >> https://www.redhat.com/mailman/listinfo/redhat-sysadmin-list >> > > -- > redhat-sysadmin-list mailing list > redhat-sysadmin-list at redhat.com > https://www.redhat.com/mailman/listinfo/redhat-sysadmin-list > From mgalgoci at redhat.com Thu Sep 22 14:16:01 2005 From: mgalgoci at redhat.com (Matthew Galgoci) Date: Thu, 22 Sep 2005 10:16:01 -0400 Subject: Performance problem (high Load average) in Mail Server In-Reply-To: References: Message-ID: > IBM X-345 Intel-based Server > DUAL Intel Xeon 2.6 GHz > 2.5 GB RAM > 3 Nos. 72 GB HDD > > OS: Red Hat Enterprise Linux 3 AS Update 5 > kernel: 2.4.21-32.0.1.ELsmp > > We are regularly updating the recommended patches from Red Hat Network on > Mail Server and it was working fine. But from last 15 days we are facing > severe load problems due to which server gets busy and user experencing > low response/time-outs while accessing mails from their email clients. > > We observed that the busiest process is usually kscand, and overall kscand > and kswapd have the highest CPU utilization times. > > As per recent test the server perform well after initial boot, but after > some time only 20 MB of free memory space left out of total 2.5 GB RAM and > around 1 GB swap space get used out of 2 GB of swap memory. After that > kscand gets much busier and CPU load seems to increased wildly. kscand is looking for pages to evict so it can page in stuff that is already on swap that needs to run. Basically, you don't have enough ram for what you are trying to do, or something is running amuck and consuming all of your ram - and it's not kscand. What does a ps -fwax show when kscand is busy? -- Matthew Galgoci GIS Production Operations Red Hat, Inc 919.754.3700 x44155 From d1 at ms.unimelb.edu.au Fri Sep 23 00:07:28 2005 From: d1 at ms.unimelb.edu.au (Alan) Date: Fri, 23 Sep 2005 10:07:28 +1000 Subject: Performance problem (high Load average) in Mail Server In-Reply-To: References: Message-ID: <43334740.7080305@ms.unimelb.edu.au> Mahesh Shinde wrote: > > Dear John, > > Yes RHEL3 is uw-imapd based and users also uses WebMail heavily and > httpd process takes 25% of CPU resouces. > > Can you tell me how to check which user or program/application is eating > all the RAM. "ps aux" or "top" will tell you VSZ and RSS. Cheers, Alan > > The output of "free" command is as follows: > -------------------------------------------------------------------------- > # free > total used free shared buffers cached > Mem: 2565840 2547952 17888 0 37200 2269280 > -/+ buffers/cache: 241472 2324368 > Swap: 2048248 423556 1624692 > ------------------------------------------------------------------------- > > Regards, > Mahesh > > On Thu, 22 Sep 2005, John Rumpelein wrote: > >> What's eating all the RAM? >> >> There's a whitepaper on converting to dovecot from uw-imapd (is RHEL3 >> uw-imapd based?) which describes performance issues like this. >> >> >> On 9/22/05, Mahesh Shinde wrote: >> >>> >>> We have a Mail Server with following Hardware/Software configuration. >>> >>> IBM X-345 Intel-based Server >>> DUAL Intel Xeon 2.6 GHz >>> 2.5 GB RAM >>> 3 Nos. 72 GB HDD >>> >>> OS: Red Hat Enterprise Linux 3 AS Update 5 >>> kernel: 2.4.21-32.0.1.ELsmp >>> >>> We are regularly updating the recommended patches from Red Hat >>> Network on >>> Mail Server and it was working fine. But from last 15 days we are facing >>> severe load problems due to which server gets busy and user experencing >>> low response/time-outs while accessing mails from their email clients. >>> >>> We observed that the busiest process is usually kscand, and overall >>> kscand >>> and kswapd have the highest CPU utilization times. >>> >>> As per recent test the server perform well after initial boot, but after >>> some time only 20 MB of free memory space left out of total 2.5 GB >>> RAM and >>> around 1 GB swap space get used out of 2 GB of swap memory. After that >>> kscand gets much busier and CPU load seems to increased wildly. >>> >>> Please provide us solution to this critical problem at the earliest. >>> >>> Regards, >>> Mahesh Shinde >>> C-DAC, Pune >>> From maheshs at cdac.in Thu Sep 29 14:32:35 2005 From: maheshs at cdac.in (Mahesh Shinde) Date: Thu, 29 Sep 2005 20:02:35 +0530 (India Standard Time) Subject: Performance problem (high Load average) in Mail Server (fwd) Message-ID: Dear Matthew, We are using Trend micro's IMSS antivirus software and its processes consumes normally around 1 GB of RAM. But this software is running from last 6 months and we are facing problem of high load average from last one month. What I observed recently in "top" is 2 or 3 httpd processes are using more that 25% of CPU resources and then load average goes on increasing in the range of 10 to 20. After restarting "httpd" service load average comes to normal. What is causing "httpd" to suddenly consuming high CPU resources and how to find out where is going wrong? Please reply, Regards, Mahesh Shinde C-DAC, Pune --------------------------------------------------- On Thu, 22 Sep 2005, Matthew Galgoci wrote: >> IBM X-345 Intel-based Server >> DUAL Intel Xeon 2.6 GHz >> 2.5 GB RAM >> 3 Nos. 72 GB HDD >> >> OS: Red Hat Enterprise Linux 3 AS Update 5 >> kernel: 2.4.21-32.0.1.ELsmp >> >> We are regularly updating the recommended patches from Red Hat Network on >> Mail Server and it was working fine. But from last 15 days we are facing >> severe load problems due to which server gets busy and user experencing >> low response/time-outs while accessing mails from their email clients. >> >> We observed that the busiest process is usually kscand, and overall kscand >> and kswapd have the highest CPU utilization times. >> >> As per recent test the server perform well after initial boot, but after >> some time only 20 MB of free memory space left out of total 2.5 GB RAM and >> around 1 GB swap space get used out of 2 GB of swap memory. After that >> kscand gets much busier and CPU load seems to increased wildly. > > kscand is looking for pages to evict so it can page in stuff that is > already on swap that needs to run. Basically, you don't have enough > ram for what you are trying to do, or something is running amuck and > consuming all of your ram - and it's not kscand. > > What does a ps -fwax show when kscand is busy? > > From sbathe at gmail.com Fri Sep 30 06:48:16 2005 From: sbathe at gmail.com (Saurabh Bathe) Date: Fri, 30 Sep 2005 12:18:16 +0530 Subject: Performance problem (high Load average) in Mail Server (fwd) In-Reply-To: References: Message-ID: <433CDFB0.5070806@gmail.com> Mahesh Shinde wrote: > > Dear Matthew, > > We are using Trend micro's IMSS antivirus software and its processes > consumes normally around 1 GB of RAM. But this software is running from > last 6 months and we are facing problem of high load average from last > one month. > > What I observed recently in "top" is 2 or 3 httpd processes are using > more that 25% of CPU resources and then load average goes on increasing > in the range of 10 to 20. After restarting "httpd" service load average > comes to normal. > > What is causing "httpd" to suddenly consuming high CPU resources and how > to find out where is going wrong? > This could be due to very large IMAP mail boxes. Check that when the load goes very high, which mailboxes are being accessed through web. mailboxes > 600-700 M and web clients like squirrelmail, could be a problem. To check if this is problem, try accessing the same same mailbox with a mail client like Mozilla Mail/Thunderbird and see if the server load increases again. This way we can see if this is a web clinet's problem or that of the mail server. //Saurabh