Rsync and "ssh_exchange_identification: Connection closed by remote host"

I have a script which rsync's a local directory to about 10 remote servers. It basically has a list of the servers and for loops through each of them doing the rsync. To speed up the process I have it fork for each of the rsyncs, so that all 10 happen in parallel. The problem is the last few of the servers come back with a "ssh_exchange_identification: Connection closed by remote host". This doesn't happen when doing the rsyncs individually or in serial, and putting a half second delay in between each fixes the problem (usually, sometimes the last server in the list still returns the error).

Does anyone know why this would be happening? I assume it's an issue with rysnc and not my script, as the script just does a system call to rysnc anyway.

I have seen this issue before. You need to crank up the MaxStartups value on the SSH servers you are connecting to. Note the default value is 10. You can do this by editing /etc/ssh/sshd_config and reloading the SSH daemon (hopefully you control the remote servers).

From man sshd_config:

 MaxStartups
         Specifies the maximum number of concurrent unauthenticated connections to the SSH daemon.  Additional connections will be dropped until authentication succeeds or the
         LoginGraceTime expires for a connection.  The default is 10.

         Alternatively, random early drop can be enabled by specifying the three colon separated values “start:rate:full” (e.g. "10:30:60").  sshd(8) will refuse connection attempts
         with a probability of “rate/100” (30%) if there are currently “start” (10) unauthenticated connections.  The probability increases linearly and all connection attempts are
         refused if the number of unauthenticated connections reaches “full” (60).