How to get into server when it's overloaded and ssh takes ages to connect
My debian server sometimes gets overloaded (due to lots of apache processes and a badly-written website). The software developers are taking their time fixing the problem, but until they fix it, I have to cope with a server that keeps overloading.
When it overloads, I struggle to get an SSH connection to the box because it is under so much load. If and when I manage to get an SSH connection, I am always able to restart apache which rescues the server from the crazy website code.
But sometimes it can take me an hour or so to get a connection as my SSH connection keeps timing out.
Can anyone suggest a way I can guarantee that I can always SSH in, even if the server gets under a big load?
Renice sshd
process to raise its priority. Should help when resources are tight
The ideal solution to this problem is to enable some sort of out-of-band access method. The simplest is probably a serial console. You can get serial consoles that allow you to ssh in and get access to the serial console of your server. Many decent hosting companies can supply this sort of access on dedicated and colo hosted servers. Alternatively, you might have something like a IP enabled KVM or lights out management like HP's iLO or Dell's DRAC.