When I have to administrate an existing Linux server what's the best way to check if it is secure?
Download Nessus and perform a network check on it. It will tell you about remotely exploitable vulnerabilities.
Also, install Ossec; although it is not its primary purpose, it will find some common misconfigurations (improperly configured accounts for example). And its primary function -- host based intrusion detection -- will help find if someone is trying to exploit vulnerabilities.
I would start with the Center for Internet Security's "benchmark" checklists. These are consensus based checklists compiled by security professionals for a variety of platforms and software packages. Some tools mentioned by the checklists, or otherwise commonly recommended that will aid in your hunt for security issues:
- Nessus/OpenVAS (vulnerability scanner)
- Nmap (port scanner)
- TCPdump/Wireshark (libpcap packet capture)
- SNORT (intrusion detection system)
(tcpdump is installed on many linux systems by default, or may be easily installed from a package repository, and has a comprehensive man page)
If this is for the company you work for, make sure that security analysis is authorized by management, and that the scans aren't going to cause any outage or application silliness. Yes a simple portscan can cause problems - portscan older HP Laserjet printers and they'll spit out heaps of paper.
As a very quick first check:
Run
netstat -ltnp
as root. That will show you all services listening on the network:
This might show you stuff you want to shut down right away. Then you can continue with the solutions in the other answers.
For services that need to run, but not be accessible from the outside (such as a local DB server), consider changing the config so it only listens on localhost/127.0.0.1. That way it can only be accessed by local users.
I would check out Bastille-Linux at http://www.bastille-unix.org/, its a set of scripts that you can run and it will check the system settings, file permissions, user setup etc. I've used it once or twice on my own boxes, and if finds issues on default installs (mostly r_x on rsh/rsync utils). It outputs as html/java+curses/flat text.