[K12OSN] SSH

[Jon Spriggs]

Hi there Tim,

I don't know how many systems we're talking about here, but it might
be worth doing a couple of things.

If you use a Linux machine as your Internet router, install Wireshark
(which is the new name for Ethereal) on there. If you don't use Linux
on your router, put a hub in between all your switches and your
internet router - likely to be no more than about 3 or 4 at a guess,
then attach a desktop machine to each of the hubs, on which should be
installed something like Fedora or another desktop distro, and then
install Wireshark on these machines. You should be monitoring for (as
previously mentioned) port 22 for all outbound traffic, for example,
if your internal network is 10.0.0.0 and your netmask is 255.255.255.0
then your filter is:

tcp.port == 22 && ip.dst != 10.0.0.0/24

This will then show you the originating server for all your outbound
SSH requests.

Next, you isolate the machine. If it's not a machine you manage,
you'll have to figure out which box on the network it is. If you can
resolve the ARP address, then you can look on your switches (assuming
they are managed switches) to see which port the machine is plugged
into, or you'll just have to do a massive amount of pinging against
that IP address, and look at the traffic lights on the switches to see
which port is receiving a lot of traffic. Unplug the box and wait to
see who moans that their computer has stopped working, then pull the
owner to one side and give them a dressing down about letting someone
use their machine (because obviously it *won't* be them LOL).

If it's a machine that you manage, then you should remove any tools
like NMAP from it, and then take two things, either a DD image or a
tar of the machine's hard disks for evidence, and then an MD5SUM of
all the files on the server. This should be fairly easy to do by using
the find command and the md5sum command... but I'll admit, offhand my
bash-scripting knowledge is a bit lacking. You should also use the
find command to make a list of all files on the drive. Next, send an
e-mail to all users of that box saying that you've had your upstream
connection blocked due to an issue which you have located as
originating on this box, and then wait a day or so. Take another
snapshot of the drive, and compare the MD5SUMs of the files on the
server against those you took the day before, and whatever has changed
see if the original snapshot of those files looks like NMAP or
similar.

Check your login logs (if you make them) and see who logged in in that
24 hours. See which user owns the file that has changed (and pray it's
not "root:root") and then confront the user. If that doesn't work,
then monitor your wireshark logs and see at what time of day the
traffic is being generated. If it's outside your normal operating
hours, check the crontabs for all your users, see what ps aux throws
up - especially for high processor usage or long-running tasks. You
can also try using ntop to see what processes are running "hot" as far
as network traffic goes.

And after all that lot, phone your ISP and tell them that you've
sorted the problem, and that if they have any further issues, rather
than cutting you off at the ISP level, maybe they should contact a
local administrator, such as yourself, and provide your phone number.

Or, just drop outbound port 22 at your firewall :)

Regards,

Jon

On 2/5/07, Tim Hart <hartt at glenburn.net> wrote:
> I am getting my Linux servers shut down by by ISP for outbound ssh
> scanning. I can turn it off but would like to know what the issue could be
> so I can still use ssh. Ideas?
>
> Tim
>
> _______________________________________________
> K12OSN mailing list
> K12OSN at redhat.com
> https://www.redhat.com/mailman/listinfo/k12osn
> For more info see <http://www.k12os.org>
>