Site Hacked Using ?cmd=ls

I strongly disagree with Chris S's claim that all files/directories should be owned by root. There is a reason for the Unix permissions system.

There are two basic ways to run Apache/PHP. One is to run it as the www-data user, and have the files owned by a non-root username. Apache runs as a low-privilege account and must be granted access to particular directories/files in order to write to them. This is why Joomla has the ftp layer, to compensate for this. However, in a shared server environment, the fact that all files need to be world readable makes config files for other sites on that machine easily read.

The other way is to run Apache with PHP running suPHP, which is what CPanel prefers. In this case, Apache runs as a low privilege user, but, all PHP requests are handed to a script that changes the ownership to the username that owns the files. While you can now use Unix permissions to prevent other rogue scripts on the machine to browse your directories, any compromised PHP script is able to be run as your username and as a consequence, modify/deface any of the files owned by your username.

Since you're not well versed in server security, finding well hidden rootkits, etc on the machine would not be a fun task. First, you'd have to know whether the kernel was exploitable (unless you're running a very recent kernel, the answer here is yes), and whether anything had been affected. This particular hack usually occurs through a compromised FTP account at which point they are able to execute scripts. Since you found that code, it also suggests that the 'hacker' using it wasn't very sophisticated. There are many ways that he could have hidden that code and prevented your logs from seeing what he was doing.

mojah is correct. Once they get in, they try to run a script from /dev/shm/.something or /tmp that connects to their IRC network, or, acts as a takeover bot on undernet or another competing network. You'll likely find one or two scripts running, perhaps a cron entry to restart it, and, other remote shells hidden throughout your Joomla installation. Look for files in the /uploads or /images directory named similar to existing files. i.e. img_1034.jpg.php. They will usually hide their irc bot in multiple places that aren't web accessible so that you won't stumble across them when you log in, but, will have stashed their remote shells in places so that they can get back in and rerun their script and have it reconnect to their network.

In any case, the task you're faced with is somewhat tricky. You've got a site that you need to stay online, you lack some of the experience with these situations, and, you just want your site to work.

Take a dump of your database through Joomla's export function, make sure it is a complete dump. Create a second site and import the dump to verify it. Once you are sure you have a good, importable dump, make a backup of the site. Delete all files, reinstall Joomla, basic installation, use the existing MySQL connection information - it might believe you are upgrading, in which case allow it to upgrade. If you are on a VPS somewhere, perhaps have them hand you a fresh image and reinstall.

Your server is wildly unsecured, probably as a result of the hacking. It needs to be taken offline as soon as possible.

The best course of action at this point is to wipe it clean, restore from backup, and make sure it is secured. You'll have almost no way of being sure you got rid of the hacker/virus/etc unless you wipe it.

I agree with Chris S. You're too exploited. You need to wipe and restore from backup. And this time, before you go live, you need to be extremely careful with your write and execute permissions.

Once an attacker has obtained system-level access, you can't trust your code anymore.

Directory permissions are HUGE. This cannot be stressed enough. They uploaded code to your site via an exploit, but that can only be done if your code can write to the local directories. If it can't, or if the local directories that it CAN write to can't be interpreted or used to host executable code, then the damage that can be done is extremely limited.

I recommend removing write permissions everywhere you can, ALL OF THEM, even the ones belonging to root. The only things that should be writable anyway are upload directories and whatever directory stores your session files. If you don't allow uploads, then only the session file directory, and that one should be as locked down as you can make it.

You should also run regular file integrity scans. Unfortunately, that's not as easy if you don't have complete access to the server. Still you can download the site and compare it to your backup on a regular basis. Ideally, you should be able to overwrite the entire site from backup at any time, and have no one notice the difference.