Pain removing a perl rootkit

So, we host a geoservice webserver thing at the office.

Someone apparently broke into this box (probably via ftp or ssh), and put some kind of irc-managed rootkit thing.

Now I'm trying to clean the whole thing up, I found the process pid who tries to connect via irc, but i can't figure out who's the invoking process (already looked with ps, pstree, lsof) The process is a perl script owned by www user, but ps aux |grep displays a fake file path on the last column.

Is there another way to trace that pid and catch the invoker?

Forgot to mention: the kernel is 2.6.23, which is exploitable to become root, but I can't touch this machine too much, so I can't upgrade the kernel

EDIT: lsof might help:

lsof -p 9481
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAMEss
perl 9481 www cwd DIR 8,2 608 2 /ss
perl 9481 www rtd DIR 8,2 608 2 /ss
perl 9481 www txt REG 8,2 1168928 38385 /usr/bin/perl5.8.8ss
perl 9481 www mem REG 8,2 135348 23286 /lib64/ld-2.5.soss
perl 9481 www mem REG 8,2 103711 23295 /lib64/libnsl-2.5.soss
perl 9481 www mem REG 8,2 19112 23292 /lib64/libdl-2.5.soss
perl 9481 www mem REG 8,2 586243 23293 /lib64/libm-2.5.soss
perl 9481 www mem REG 8,2 27041 23291 /lib64/libcrypt-2.5.soss
perl 9481 www mem REG 8,2 14262 23307 /lib64/libutil-2.5.soss
perl 9481 www mem REG 8,2 128642 23303 /lib64/libpthread-2.5.soss
perl 9481 www mem REG 8,2 1602809 23289 /lib64/libc-2.5.soss
perl 9481 www mem REG 8,2 19256 38662 /usr/lib64/perl5/5.8.8/x86_64-linux-threa d-multi/auto/IO/IO.soss
perl 9481 www mem REG 8,2 21328 38877 /usr/lib64/perl5/5.8.8/x86_64-linux-threa d-multi/auto/Socket/Socket.soss
perl 9481 www mem REG 8,2 52512 23298 /lib64/libnss_files-2.5.soss
perl 9481 www 0r FIFO 0,5 1068892 pipess
perl 9481 www 1w FIFO 0,5 1071920 pipess
perl 9481 www 2w FIFO 0,5 1068894 pipess
perl 9481 www 3u IPv4 130646198 TCP 192.168.90.7:60321->www.****.net:ircd (SYN_SENT)

If I can give you any advice, it is to stop wasting your time cleaning up. Make an image of the OS for forensic stuff for later, and just reinstall the server.

Sorry, but its the only secure way to resolving yourself from being rootkitted.

Later you can check the image, for certain reasons, why it happened.

From my own personal experience, I did this, and later found an internal user which had a SSH key containing the flaw of openssl in 2008.

I hope, it clears up the things.

Note:
If you are going to image/backup the server before reinstalling, be very careful, how you do this. As @dfranke said, boot from a trusted medium to backup.

You shouldn't connect to other machines from a rooted server, as great rootkits are known to be able to spread through trusted sessions such as SSH.

The commandline can be changed if the process alters argv[0]. Try ls -l /proc/[pid]/exe

From man 5 proc

this file is a symbolic link containing the actual pathname of the executed command. This symbolic link can be dereferenced normally; attempting to open it will open the executable. You can even type /proc/[number]/exe to run another copy of the same executable as is being run by process [number]. In a multithreaded process, the contents of this symbolic link are not available if the main thread has already terminated

ps auxwf | less gives you the "forest view" of processes which can show you what process launched this process (unless the rootkit is hiding it, or the app's parent has exited and it's been reparented to init).

This would be mostly academic and probably just a timewaster, but strings -n 10 /proc/[pid]/mem might be fun to watch scroll past. You could also echo 0x7 > /proc/[pid]/coredump_filter and use gdb gcore to force a coredump with everything possible in it, but then the process dies, which could block further analysis.

But definitely take Arenstar's advice. Back up data only, restore everything executable from backups, and start over. You should probably restore the website from backups as well, there could be malicious javascript added to every html or php file. If you're considering legal action, you'll want to just set the machine aside, unplug it from the net, and stop whatever it is you're doing until forensic experts have done their job.