Blackhole

[Reuben D. Budiardja]

On Monday 11 April 2005 10:22, Chris Kenward wrote:
> Hi Mike
>
> > Perhaps this will help to identify the file:
<snip>
> Many thanks. The web server has more than 200 websites on it, which is
> going to make it exceedingly difficult to track which of those allowed the
> attack. The server has only recently been rebuilt, at the cost of lots of
> stress while our customers whinged about their sites not being there, and
> I'm pretty loathe to go through that all again.

If you have proper backup procedure and have proper notes for the 
installation / configuration, it should be less stressful to rebuild the 
server. All you need then is copy the necessary configuration files and data 
and all should be set. Yes, it's not fun, and it's a real pain in the neck 
(been there..), but it is the only way to make sure that your machine is 
clean if your it has been compromised.


> There is mention in the link above regarding directories called:
> /tmp/.blackhole.c
>
> There isn't a directory called .blackhole.c on the server - just the one
> executable binary in the /tmp folder.

This does not prove anything.

> I can't find anything else on the 
> server which looks as though someone has had root access to the machine but
> there again I'm no Linux expert so it could be staring me in the face.

I would try to do the following:

1. Verify all your RPM installed file using 'rpm -Va'. Make sure that you can 
account for any files that are reported to have been changed (size, md5, 
etc). Make sure that _none_ of the binary installed by RPM has been changed, 
unless you know exactly who changed it and for what purpose. Important 
binaries are , eg. : ps, ls, top.

2. Run several rootkit detection. Rookit hunter (http://www.rootkit.nl/) and 
chkrootkit (http://www.chkrootkit.org/) are two commons program for rootkit 
detection.

3. Check for all incoming, outgoing connections and make sure that you can 
account for those. For example you can netstat to do this (read 'man 
netstat).

4. Check for the third party program (especially like the web-based program) 
installed in your machine for the versions, and check it againts reported 
recent vulnerabilities. For example, one of your users may have installed 
vulnerable version of awstats, gallery, etc, and did not update it. You as 
the system administrator have to check for all those

5. Check logs, firewall / iptables logs, httpd logs, secure logs, etc. If you 
find anything suspicious, make sure you can account and rule out those, and 
if you find attack, you have to determine if it's successful or not. 

6. Are you sure that none of your users would do something 'stupid' and try to 
attack / exploit vulnerabilites of other machines using your machine ? If 
necessary, check what users have done. Be careful here not to violate any 
privacy policies that you / your company have.

I regularly also run password crackers software (ie. John the Ripper) on the 
machine that I manage to make sure that none of my users use weak password. 

> Is there an "easy" way to track how this person got into the server? 

No, not really. Even if you do all of the above, it does not guarantee that 
your machine is completely secure. Security is a process, and most of the 
times you have to make judgement whether or not you are reasonable confident 
that your machine is secure.


> I 
> notice that the latest update for PHP from the RHN is 4.3.2 and I
> understand from searches I've done on the 'net that 4.3.10 or even the
> latest 4.3.11 is urgently advised due to "holes" in earlier versions. Not
> sure, however, whether this is how the person managed to drop that on the
> server.

Red Hat often backported security fixes to an earlier version of their 
software. You have to check again RHN errate to see if the 'holes' you 
mentioned has been plugged by Redhat's backport security fixes. Versions do 
not always tell the whole story.

RDB

-- 
Reuben D. Budiardja
Dept. Physics and Astronomy
University of Tennessee, Knoxville, TN

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GIT/M/MU/P/S d-(++) s: a-- C++(+++) UL++++ P-- L+++>++++ E- W+++ 
N+ o? K- w--- !O M- V? !PS !PE Y PGP- t+ 5 X R- tv+ 
b++>+++ DI D(+) G e++>++++ h+(*) r++ y->++++
------END GEEK CODE BLOCK------