What's the best way to clean up after a fork bomb?
Prevent the fork bomb from exhausting the process limit with a reasonable per user process limit using ulimit.
That way, a single user will exhaust their process quota long before the system limit is reached.
The first thing to try would be to get users that are logged in to logout. It's possible their shell may be the parent process of the process doing all the forking and that might end the problem.
If that doesn't work, you could try running kill -STOP -2
as root to freeze all processes running as any user other than root. If that works, you can then use kill -CONT <pid>
to unfreeze some known processes that are unrelated to the fork bomb and kill them to eliminate the full process table issue and give you some breathing room to track down and kill the original source of the problem. Sendmail would be a good example of a system process to kill as it would be easy to identify by using the .pid file to identify the pid. For example, kill -CONT $(< /var/run/sendmail.pid); kill $(< /var/run/sendmail.pid)
.
Not sure how you could even send a STOP signal, since spawning kill
would require an available process handle. Besides, in my experience systems become overloaded and unusable long before running out of processes.
Have you considered simply enforcing per-user process limits with ulimit
? That would prevent your users from launching fork bombs (accidentally or not).