Severe Server Problems (FC4)

[Karl Pearson]

On Mon, 14 Aug 2006, Rick Stevens wrote:

> On Mon, 2006-08-14 at 14:21 -0600, Karl Pearson wrote:
>> On Mon, 14 Aug 2006, Rick Stevens wrote:
>>
>>> On Mon, 2006-08-14 at 00:40 -0600, karlp at ourldsfamily.com wrote:
>>>> My RH8.0 email/web server crashed. I've been struggling over the weekend
>>>> to get things back up and still some things are hammered. You can see
>>>> them by doing mailq -qL and re-process them by doing for i in "ls Q*"; do
>>>> sendmail -v -qI$i -d11; done from in /var/spool/mqueue, which has drwx
>>>> --- --- permissions, which is correct. I'm suspect of sm-client, too,
>>>> because it starts, then dies. I had to change the location of
>>>> /var/run/sm-client.pid because it failed to start at all because of
>>>> permissions.
>>>>
>>>> Forms on my server don't submit the emails to me they once did for usage
>>>> tracking, etc. In maillog:
>>>>    1. Aug 14 00:25:58 moroni sendmail[25780]: NOQUEUE: SYSERR(apache):
>>>> can not chdir(/var/spool/mqueue/): Permission denied:
>>>
>>> You need to check the sendmail.cf and submit.cf files to see what user the
>>> programs are trying to run as.  Generally, sendmail tries to run as root
>>> when it's doing things such as creating mail queue entries and as user
>>> 8:12 (mail:mail) when doing other things such as delivery and such.
>>> smclient usually runs as user "smmsp" at all times.
>>>
>>> As far as /var/run/sm-client.pid, the trick is that it is created by
>>> root's performing a "touch" of it first, then does a "chown smmsp:smmsp"
>>> of the file BEFORE sm-client is fired up.  Check /etc/rc.d/init.d/sendmail
>>> for details.
>>
>> How do I verify that they run as those users? I see in sendmail.mc 8:12,
>> which looks right.
>
> Yes 8:12 is right (check /etc/passwd for user mail).

I did that. It's 8:12

>
>>  Also, sm-client.pid is chmoded to smmsp:smmsp and it
>> still doesn't work. Doing a service sendmail reload succeeds for sendmail
>> and fails for sm-client.
>
> Uh, hmmm.  Well, submit.cf should have "O RunAsUser=smmsp" in it, which
> should also make it run as user smmsp.  The error message you get when
> you reload sendmail refers to an inability to open
> /var/run/sm-client.pid?  That's just plain odd.  For giggles, can you
> "service sendmail stop", then delete /var/run/sm-client.pid, then
> "service sendmail start" and see what that does?

Tried that and solved it by being stupid... kind of. I added the following 
line to /etc/rc.d/init.d/sendmail after the `touch /var/run/sm-client.pid` 
line:
chmod 666 /var/run/sm-client.pid

It no longer complains about THAT issue...

>
>>
>> Another interesting thing is that local email fails, but if I use pine from
>> my PC, it works.
>>
>>>
>>>> It appears these errors result in the next error, which is generated by
>>>> the following CGI command: cat $FNM|tr 'A-Z' 'a-z' |mail -s "Access to
>>>> $ACCESS" karlp
>>>>
>>>>    2. mail: invalid option -- r
>>>> Usage: mail [-iInv] [-s subject] [-c cc-addr] [-b bcc-addr] to-addr ...
>>>>             [-- sendmail-options ...]
>>>>        mail [-iInNv] -f [name]
>>>>        mail [-iInNv] [-u user]
>>>> 554 5.3.0 unknown mailer error 1
>>>
>>> That's rather odd.  Your command line doesn't specify a "-r" anywhere.
>>> This leads one to believe that one of the programs (possibly "tr") got
>>> whapped and is spewing out garbage.  You really should try redirecting the
>>> output of it to a file to test it:
>>>
>>> 	cat $FNM|tr 'A-Z' 'a-z' >/tmp/testoutput
>>>
>>> and you should look at the expansion of "$ACCESS" to see if it has an
>>> embedded quote or something in it.
>>
>> I see nothing in it at all and the email still doesn't come... Same error,
>> too. Interesting that the owner is apache:smmsp in mqueue and both files are
>> there, but one starts with Q rather than q... as before. I was going to
>> `watch` a cat of the file to see if a visit from another host messes up.
>
> Queue files that start with a "Q" indicate that the delivery attempts
> have completely failed (all retries exhausted).  That usually indicates
> a bogus "To" address, the destination server isn't listening to SMTP,
> or it's rejecting the mail repeatedly.  Check the content of the Q file
> and its associated "d" file for clues.

I now see an error of: mydomain.com: host not recognized.

I suspect I've messed up sendmail.mc with MASQUERADE_DOMAIN(...)dnl
settings. I put sub hosts that send email, IE, My pc and the server's 
hostname. I suspect I should only put localhost and localhost.localdomain in 
there, right?


>
>>>
>>>
>>>> I apologize for being a bit scattered. I've been relatively brain
>>>> hampered for 3 days now... 13 hours of sleep in the past 50+ hours.
>>>
>>> Lightweight!  I've been running on 3 hours of sleep or less per night for
>>> 3 weeks.  If you're looking for sympathy, you've come to the WRONG place!
>>> :-p
>>
>> I don't use caffeine. Any sympathy now... (as in 'can you hear me [whine]
>> now?)
>
> Heheheheheh!  And in answer to your missive...no, I still don't feel
> your pain.  As a matter of fact, I'm rather numb all over.  I don't know
> if I feel my own toes at this point.

Better check for some. I can still feel mine.

>
>> PS. I'd love to give you (Rick) ssh access and have you poke around and see
>> how bad an upgrade from RH8.0 to FC4 can go... Just picture named, for
>> example, without a chroot option working getting kludged back to the old
>> way.
>
> Oh, my GAWD!  8.0 to FC4?  Yikes!  Just 8.0 alone is frightening.  That
> was possibly one of the worst RH releases in known times!

I had it running solid on 2 boxes for 3 years.

karl

>