ssh immediately disconnects "Write failed: broken pipe"

Solution 1:

Another solution is detailed here: https://access.redhat.com/solutions/30316

I started with the same problem and during the investigation I had another user try to su as my user. He got the error that lead me to the link above (cannot set user id: Resource temporarily unavailable). I was indeed above the allowed thread limit for my user and increasing that limit allowed me login again.

Basically, run this against your user (this is the current number of threads active for that user):

ps -u testuser -L | wc -l

If the result is bigger than whatever limit is set in /etc/security/limits.d/90-nproc.conf (default is 1024) for that user, than you need to increase it. You will need to do that with another user that has not yet exceeded his limit (or somehow kill some of the threads to be able to login).

To increase it, just add a line similar to the existing one, but specific for your user:

testuser   soft    nproc     2048

I know it's an old question, but this might help people that have this issue.

Solution 2:

So, in my specific case, I had left a screen open running gdb attached to a process trying to reproduce an issue a customer was having related to our product spawning too many threads. Well apparently the issue had been reproduced overnight and my user had reached the system's user thread limit.

So sshd could not spawn a new process under my user due to the process and thread limit. Ultimately what we had to do was to log in with a different user and kill my running screen, gdb, and the process being debugged. This freed up process and thread resources allowing me to finally reconnect.