Machine not locked-up but can't log on either

[Ryan Golhar]

/var is up to 89% full.  Looking at /var/log, I see:

audbin[30934]: saving binary audit log /var/log/audit.d/bin.0
audbin[30934]: threshold 20.00 exceeded for filesystem
/var/log/audit.d/. - free blocks down to 15.59%
auditd[1716]: Notify command /usr/sbin/audbin -S
/var/log/audit.d/save.%u -C -T 20% exited with status 1
auditd[1716]: output error
auditd[1716]: output error
auditd[1716]: output error; suspending execution

audit is on for runlevel 2,3,5.  I've turned it off (for now) and
deleted the log files.  I still need to clear some space, but for now,
we'll see if this has anything to do with it.  Thanks,

Ryan


-----Original Message-----
From: Wayne Betts [mailto:wbetts at bnl.gov] 
Sent: Thursday, May 04, 2006 1:50 PM
To: golharam at umdnj.edu; General Red Hat Linux discussion list
Subject: Re: Machine not locked-up but can't log on either


Ryan Golhar wrote:

>I have a machine configured just like 20 other machines (kickstart). 
>Every once in a while the machine just hangs.  The X terminal is dead, 
>but you can toggle to the other console terminals.  After you type in 
>your name and password, it hangs.
>
>I'm able to ssh to the machine, and type a name/password but it hangs. 
>I can't even log in as root from the console - root is set up as a 
>local account in case the network goes out we can still get access to
it.
>
>The only thing I can do is hit the power button but I'd like to find 
>out why this is happening.  Any suggestions on what I can try?
>  
>
I've seen and read of similar behaviour with logins if the auditd 
service is running and the partition containing the /var/ directory 
fills beyond the limit specified in the configuration file (typically 80

percent).  Since rebooting fixes your problem, however, I'm less 
inclined to think this might be what you are encountering, since 
rebooting probably doesn't free up disk space.  But since it is so easy 
to check (df /var/ (>80%?) and "chkconfig --list audit" to see if audit 
is running by default), why not try stopping and disabling auditd and 
see if your problem goes away.  Also, I'm pretty sure there is evidence 
in the log files if this is the problem, but I can't recall any of the 
details from my encounters with this issue.

- Wayne