Fork bomb protection not working : Amount of processes not limited

I just came to realize that my system is not limiting the amount of processes per user properly thus not preventing a user from doing a fork-bomb and crashing the entire system:

user@thebe:~$ cat /etc/security/limits.conf | grep user
user        hard    nproc   512
user@thebe:~$ ulimit -u
1024
user@thebe:~$ :(){ :|:& };:
[1] 2559
user@thebe:~$ ht-bash: fork: Cannot allocate memory
-bash: fork: Cannot allocate memory
-bash: fork: Cannot allocate memory
-bash: fork: Cannot allocate memory
-bash: fork: Cannot allocate memory
-bash: fork: Cannot allocate memory
-bash: fork: Cannot allocate memory
-bash: fork: Cannot allocate memory
...
Connection to thebe closed by remote host.

Is this a bug or why is it ignoring the limit in limits.conf and why is not applying the limit that ulimit -n claims it to be?

PS: I really don't think the memory limit is hit before the process limit. This machine has 8GB ram and it was using only 4% of it at the time when I dropped the fork bomb.

EDIT:

I managed to reproduce this on a live CD. So I guess this must be a bug. It basically ends up killing all processes, including system critical things like X11, SSHD etc.

Any user can crash the system.

Solution 1:

Turns out that /etc/security/limits.conf does work, but needs reboot before it gets interpreted. A log-out is not sufficient.

I recommend to anybody to a limit to the config file like

user hard nproc 512

Replace user with any username that you would want to limit.

Or, better:

@group hard nproc 512

Replace group with any user-group that you want to limit.