how to limit the number of simultaneous processes per user in the system?
A kernel security patch grsecurity has a feature called Enforce RLMIT_NPROC on execs. When enabled, it enforces per-user limits to be system-wide instead of per-login (or worse, as you have observed).