How to detect a hidden process in linux?

We have a box we suspect has been rooted at work. The question is how do we find it? I am no system administrator, but I was brought onto the team to resolve the situation and I am curious where good places to look for such as problem might be?

The reason we suspect this is that we have noticed a higher than normal network utilization on the machine from high (what appear to be random) ports.

What can do we to locate the problem child? What can we do to protect from this in the future? Is there monitoring we can run to make us aware of this in the future? (Aside from network monitoring which we are already working on keeping a closer eye on.)

Thanks in advance and I can provide more details if needed. Appreciate your time.

You cant trust any system tools you have on the machine. Rootkits will replace ps, netstat, ls, and more to hide their presence. You should take the machine offline, take out its hard drive, make a forensic copy (think dd) then work from that on a seoncdary machine to scan for the rootkit.

If you insist on working on the live machine (which is usually futile) then you can try downloading a rescue distribution on CD (its very important the copy be read-only) and use its copies of ps,lsmod, etc.

Even this might fail as a rootkit can install kernel modules to hide entries in /proc where tools like ps normally operate on.

Good luck!

The problem with well constructed rootkit is that they modify your system's command; like ps and top to not show the processes of the rootkit and ls to not show files of the rootkit.

So what you'll need to do is to get these command possibly from source or in binairies form. (Be sure to be well signed). But the trick of a root kit (I've seen it) is that they maybe corrupted your compiler too. So when the compiler knows he is compiling ls or ps or any command he infectet them as well.

When I saw this problem I said fine lets recompile gcc, but no what I need to have to compile gcc...the infecte gcc....so when he knows he is compiling itself he infect it so it can infect the command.

You will say that this come big and difficult to detect, yes but rare are the root kit that are so bulletproof I just gave youthe worse case.

Seriously, if you are sure that there is a root kit in your server, reinstall it!

The best way to know whether your server has been "rooted" is to be running a host-based intrusion detection system (HIDS). Unfortunately, if you're not running a HIDS now, then it's too late to install one. The proper time to install a HIDS is when the server is first installed, and before it is put onto a network.

Briefly, most HIDS work by computing cryptographic hashes of all system binaries, and storing those hashes (along with numerous other file statistics) into a database, called the baseline database. Then, periodically, the HIDS rescans your system, comparing all files in its baseline database to the actual system files.

Yes, of course, it is possible for a rootkit to modify your baseline database, which is why you need to take a copy of that database and store it separately of the server before you put the server online. Then, if you suspect you are "rooted" (and you suspect your baseline database was also tampered with), you can boot your system from the install media, restore the known-good database from your backup, and then run a scan against the known-good. It is much more likely, however, that a rootkit will not anticipate having to defeat your particular HIDS, and so you will receive a notification from the HIDS that system files have changed, indicating a probable system intrusion.

Since you were not running a HIDS, you have no quick way to determine for certain whether you have been rooted, or what system files have been modified. You could spend a whole lot of time comparing your system files to known-good files pulled from known-good installation media, but that time is most likely better spent reinstalling your system from that media. If you want to investigate how you were rooted after the fact, the best course is to take an image of your system before you wipe it and reinstall.