Hacked CentOS 5 server - possible rootkit installed? [duplicate]

centos rootkit

Possible Duplicate:
How do I know if my Linux server has been hacked?
My server's been hacked EMERGENCY

I am running CentOS 5.3 and here is the result of "chkrootkit": 

Possible t0rn v8 \(or variation\) rootkit installed

Warning: Possible Showtee Rootkit installed
 /usr/include/file.h /usr/include/proc.h
Warning: `//root/.mysql_history' file size is zero
INFECTED (PORTS:  465)
You have    61 process hidden for readdir command
You have    62 process hidden for ps command
chkproc: Warning: Possible LKM Trojan installed
 The tty of the following user process(es) were not found
 in /var/run/utmp !
! RUID          PID TTY    CMD
! root         3040 tty2   /sbin/mingetty tty2
! root         3041 tty3   /sbin/mingetty tty3
! root         3042 tty4   /sbin/mingetty tty4
! root         3043 tty5   /sbin/mingetty tty5
! root         3046 tty6   /sbin/mingetty tty6

I don't understand what the warnings mean.

Is the server infected or in danger?

Edit:

Let me add that I first got strange message on the command line:

Unknown HZ value! (##) Assume 100

Then I followed this great instructions and replaced my hacked files with new ones. I replaced: 

/sbin/ifconfig
/bin/netstat
/usr/bin/pstree
/usr/bin/top

They were all repored as infected by the "chkrootkit".

Now I re-ran "chkrootkit" and got the above output. How to proceed to get rid off all the warnings?

Edit 2:

After checking rpm integrity with: rpm -vVa | grep 'S\.5\.\.\.\.\T' > rpmverify.txt this is what I got:

S.5....T  c /etc/mail/spamassassin/local.cf
S.5....T  c /etc/pam.d/system-auth
S.5....T  c /etc/sudoers
S.5....T  c /etc/samba/smb.conf
S.5....T    /opt/drweb/lib/drweb32.dll
S.5....T    /var/drweb/bases/drw50000.vdb
S.5....T    /var/drweb/bases/drw50001.vdb
S.5....T    /var/drweb/bases/drw50002.vdb
S.5....T    /var/drweb/bases/drw50003.vdb
S.5....T    /var/drweb/bases/drw50004.vdb
S.5....T    /var/drweb/bases/drw50005.vdb
S.5....T    /var/drweb/bases/drw50006.vdb
S.5....T    /var/drweb/bases/drw50007.vdb
S.5....T    /var/drweb/bases/drw50008.vdb
S.5....T    /var/drweb/bases/drw50009.vdb
S.5....T    /var/drweb/bases/drw50010.vdb
S.5....T    /var/drweb/bases/drw50011.vdb
S.5....T    /var/drweb/bases/drw50012.vdb
S.5....T    /var/drweb/bases/drw50013.vdb
S.5....T    /var/drweb/bases/drw50014.vdb
S.5....T    /var/drweb/bases/drw50015.vdb
S.5....T    /var/drweb/bases/drw50016.vdb
S.5....T    /var/drweb/bases/drw50017.vdb
S.5....T    /var/drweb/bases/drw50018.vdb
S.5....T    /var/drweb/bases/drw50019.vdb
S.5....T    /var/drweb/bases/drw50020.vdb
S.5....T    /var/drweb/bases/drw50021.vdb
S.5....T    /var/drweb/bases/drw50022.vdb
S.5....T    /var/drweb/bases/drw50023.vdb
S.5....T    /var/drweb/bases/drw50024.vdb
S.5....T    /var/drweb/bases/drw50025.vdb
S.5....T    /var/drweb/bases/drw50026.vdb
S.5....T    /var/drweb/bases/drw50027.vdb
S.5....T    /var/drweb/bases/drw50028.vdb
S.5....T    /var/drweb/bases/drw50029.vdb
S.5....T    /var/drweb/bases/drwebase.vdb
S.5....T    /var/drweb/bases/drwnasty.vdb
S.5....T    /var/drweb/bases/drwrisky.vdb
S.5....T    /var/drweb/bases/drwtoday.vdb
S.5....T    /var/drweb/bases/dwn50001.vdb
S.5....T    /var/drweb/bases/dwn50002.vdb
S.5....T    /var/drweb/bases/dwntoday.vdb
S.5....T    /var/drweb/bases/dwr50001.vdb
S.5....T    /var/drweb/bases/dwrtoday.vdb
S.5....T    /bin/basename
S.5....T    /bin/cat
S.5....T    /bin/chgrp
S.5....T    /bin/chmod
S.5....T    /bin/chown
S.5....T    /bin/cp
S.5....T    /bin/cut
S.5....T    /bin/dd
S.5....T    /bin/df
S.5....T    /bin/env
S.5....T    /bin/false
S.5....T    /bin/link
S.5....T    /bin/ln
S.5....T  c /etc/proftpd.conf
S.5....T  c /root/.bash_profile
S.5....T  c /etc/httpd/conf.d/mailman.conf
S.5....T    /usr/lib/mailman/Mailman/mm_cfg.pyc
S.5....T  c /etc/drweb/drweb32.ini
S.5....T    /opt/drweb/ldwrap.sh
S.5....T  c /etc/drweb/users.conf
S.5....T    /usr/share/psa-horde/imp/compose.php
S.5....T    /usr/share/psa-horde/imp/contacts.php
S.5....T    /usr/local/psa/admin/plib/api-common/cuMail.php
S.5....T    /usr/local/psa/admin/sbin/autoinstaller
S.5....T    /usr/local/psa/admin/htdocs/modules/watchdog/stats-graph.php
S.5....T    /usr/local/psa/etc/modules/watchdog/monitrc
S.5....T    /usr/local/psa/etc/modules/watchdog/wdcollect.inc.php
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter      /db/backdoorports.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/mirrors.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/programs_bad.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/suspscan.dat
S.5....T  c /etc/courier-imap/imapd.cnf
S.5....T  c /etc/php.ini
S.5....T  c /etc/ssh/sshd_config
S.5....T  c /etc/syslog.conf
S.5....T  c /etc/sysconfig/named
S.5....T  c /etc/httpd/conf.d/ssl.conf
S.5....T  c /etc/smartd.conf
S.5....T  c /etc/vsftpd/vsftpd.conf
S.5....T    /usr/share/psa-horde/util/icon_browser.php
S.5....T  c /etc/init.d/psa
S.5....T    /usr/lib/plesk-9.0/key-handler
S.5....T    /usr/local/psa/admin/htdocs/domains/databases/phpMyAdmin/librari/config.default.php
S.5....T    /usr/local/psa/admin/plib/class.ComponentsChecker.php
S.5....T    /usr/local/psa/admin/plib/class.ComponentsShow.php
S.5....T    /usr/local/psa/admin/plib/class.RestartServForm.php
S.5....T    /usr/local/psa/admin/plib/class.ServiceControl.php
S.5....T    /usr/local/psa/admin/sbin/packagemng
S.5....T    /usr/local/psa/admin/plib/backup/BackupCreateBackupNowForm.php
S.5....T  c /etc/samba/smbusers
S.5....T  c /etc/pam.d/ekshell
S.5....T  c /etc/pam.d/kshell
S.5....T  c /etc/printcap
S.5....T  c /etc/my.cnf
S.5....T    /usr/bin/spf_example_static
S.5....T    /usr/bin/spfd_static
S.5....T    /usr/bin/spfquery_static
S.5....T    /usr/bin/spftest_static
S.5....T    /usr/lib/libspf2.so.2.1.0
S.5....T  c /etc/awstats/awstats.model.conf
S.5....T    /usr/local/sso/base/Cookie.php
S.5....T  c /etc/httpd/conf/httpd.conf
S.5....T    /usr/sbin/suexec

Does that help?

Edit 3:

Here is rpm check result after core utils have been reinstalled:

S.5....T  c /etc/mail/spamassassin/local.cf
S.5....T  c /etc/pam.d/system-auth
S.5....T  c /etc/sudoers
S.5....T  c /etc/samba/smb.conf
S.5....T    /opt/drweb/lib/drweb32.dll
S.5....T    /var/drweb/bases/drw50000.vdb
S.5....T    /var/drweb/bases/drw50001.vdb
S.5....T    /var/drweb/bases/drw50002.vdb
S.5....T    /var/drweb/bases/drw50003.vdb
S.5....T    /var/drweb/bases/drw50004.vdb
S.5....T    /var/drweb/bases/drw50005.vdb
S.5....T    /var/drweb/bases/drw50006.vdb
S.5....T    /var/drweb/bases/drw50007.vdb
S.5....T    /var/drweb/bases/drw50008.vdb
S.5....T    /var/drweb/bases/drw50009.vdb
S.5....T    /var/drweb/bases/drw50010.vdb
S.5....T    /var/drweb/bases/drw50011.vdb
S.5....T    /var/drweb/bases/drw50012.vdb
S.5....T    /var/drweb/bases/drw50013.vdb
S.5....T    /var/drweb/bases/drw50014.vdb
S.5....T    /var/drweb/bases/drw50015.vdb
S.5....T    /var/drweb/bases/drw50016.vdb
S.5....T    /var/drweb/bases/drw50017.vdb
S.5....T    /var/drweb/bases/drw50018.vdb
S.5....T    /var/drweb/bases/drw50019.vdb
S.5....T    /var/drweb/bases/drw50020.vdb
S.5....T    /var/drweb/bases/drw50021.vdb
S.5....T    /var/drweb/bases/drw50022.vdb
S.5....T    /var/drweb/bases/drw50023.vdb
S.5....T    /var/drweb/bases/drw50024.vdb
S.5....T    /var/drweb/bases/drw50025.vdb
S.5....T    /var/drweb/bases/drw50026.vdb
S.5....T    /var/drweb/bases/drw50027.vdb
S.5....T    /var/drweb/bases/drw50028.vdb
S.5....T    /var/drweb/bases/drw50029.vdb
S.5....T    /var/drweb/bases/drwebase.vdb
S.5....T    /var/drweb/bases/drwnasty.vdb
S.5....T    /var/drweb/bases/drwrisky.vdb
S.5....T    /var/drweb/bases/drwtoday.vdb
S.5....T    /var/drweb/bases/dwn50001.vdb
S.5....T    /var/drweb/bases/dwn50002.vdb
S.5....T    /var/drweb/bases/dwntoday.vdb
S.5....T    /var/drweb/bases/dwr50001.vdb
S.5....T    /var/drweb/bases/dwrtoday.vdb
S.5....T  c /etc/proftpd.conf
S.5....T  c /etc/profile.d/colorls.csh
S.5....T  c /etc/profile.d/colorls.sh
S.5....T  c /root/.bash_profile
S.5....T  c /etc/httpd/conf.d/mailman.conf
S.5....T    /usr/lib/mailman/Mailman/mm_cfg.pyc
S.5....T  c /etc/drweb/drweb32.ini
S.5....T    /opt/drweb/ldwrap.sh
S.5....T  c /etc/drweb/users.conf
S.5....T    /usr/share/psa-horde/imp/compose.php
S.5....T    /usr/share/psa-horde/imp/contacts.php
S.5....T    /usr/local/psa/admin/plib/api-common/cuMail.php
S.5....T    /usr/local/psa/admin/sbin/autoinstaller
S.5....T    /usr/local/psa/admin/htdocs/modules/watchdog/stats-graph.php
S.5....T    /usr/local/psa/etc/modules/watchdog/monitrc
S.5....T    /usr/local/psa/etc/modules/watchdog/wdcollect.inc.php
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter /db/backdoorports.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/mirrors.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/programs_bad.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/suspscan.dat
S.5....T  c /etc/courier-imap/imapd.cnf
S.5....T  c /etc/php.ini
S.5....T  c /etc/ssh/sshd_config
S.5....T  c /etc/syslog.conf
S.5....T  c /etc/sysconfig/named
S.5....T  c /etc/httpd/conf.d/ssl.conf
S.5....T  c /etc/smartd.conf
S.5....T  c /etc/vsftpd/vsftpd.conf
S.5....T    /usr/share/psa-horde/util/icon_browser.php
S.5....T  c /etc/init.d/psa
S.5....T    /usr/lib/plesk-9.0/key-handler
S.5....T    /usr/local/psa/admin/htdocs/domains/databases/phpMyAdmin/libraries/config.default.php
S.5....T    /usr/local/psa/admin/plib/class.ComponentsChecker.php
S.5....T    /usr/local/psa/admin/plib/class.ComponentsShow.php
S.5....T    /usr/local/psa/admin/plib/class.RestartServForm.php
S.5....T    /usr/local/psa/admin/plib/class.ServiceControl.php
S.5....T    /usr/local/psa/admin/sbin/packagemng
S.5....T    /usr/local/psa/admin/plib/backup/BackupCreateBackupNowForm.php
S.5....T  c /etc/samba/smbusers
S.5....T  c /etc/pam.d/ekshell
S.5....T  c /etc/pam.d/kshell
S.5....T  c /etc/printcap
S.5....T  c /etc/my.cnf
S.5....T    /usr/bin/spf_example_static
S.5....T    /usr/bin/spfd_static
S.5....T    /usr/bin/spfquery_static
S.5....T    /usr/bin/spftest_static
S.5....T    /usr/lib/libspf2.so.2.1.0
S.5....T  c /etc/awstats/awstats.model.conf
S.5....T    /usr/local/sso/base/Cookie.php
S.5....T  c /etc/httpd/conf/httpd.conf
S.5....T    /usr/sbin/suexec

Solution 1:

It's a CentOS system. I typically repair these rootkits, but the chances you'll detect/get everything, having not done this before, is slim...

You could start with an RPM verification...

Run rpm -vVa | grep 'S\.5\.\.\.\.\T' > rpmverify.txt

Then examine the output in rpmverify.txt. That would allow you to check to see which binaries and config files to not match the checksums from the RPM database. It's the first place I start on fixing these systems (after making sure no unauthorized network daemons/services are running).

Edit:

I see the output of your RPM verify command. If your yum still works, run yum install yum-utils in order to gain access to the yumdownloader command.

Based on your output, your coreutils and possibly the httpd package has been compromised (cat, df, dd, chown, cp, etc.). Run yumdownloader coreutils to obtain the rpm. It'll download into your current directory. I'd force the reinstallation of the RPM (rpm -ivh --force coreutils*) and re-run the verify I suggested above.

Update:

Hackers/rootkits will often replace binaries with Trojaned versions and set the immutable flag on the file to prevent them from being removed.

Please take a look at the attributes on the /bin/ls binary by running lsattr /bin/ls.

You will possibly see an "a", "u", "i" and "s" in the output. Running chattr -uisa on the same file should remove the immutable flag and allow you to run the rpm installation.

The attributes should look like:

[root@kitteh ~]# lsattr /bin/ls
------------- /bin/ls

Repeat for any other files that fail in the RPM installation. You may need to also change/remove those attributes on the enclosing directory...

Related

pptp vpn - Kill User Connection
is it possible to direct connect a client and server with 10GBASE CX4 copper cables?
Something is blocking port 25 - how to debug?
Determining type of Linux machine
How to make domain point to another web directory?
vsftpd: **ONLY** Ftp access for local user?
How do you run Nginx on port 8080 and Apache on port 80 on the same box?
Can the startup order for windows services be configured? If so, where?
vSwitch configuration with 12 uplinks
Bash piping answer in Ubuntu
Running two services on port 80
Automate Amazon EBS Snapshot creation?