What causes SSH error: kex_exchange_identification: Connection closed by remote host?
I setup a SSH server online that is publicly accessible by anyone. Therefore, I get a lot of connections from IPs all over the world. Weirdly, none actually try to authenticate to open a session. I can myself connect and authenticate without any problem.
From time to time, I get the error: kex_exchange_identification: Connection closed by remote host in the server logs. What causes that?
Here is 30 minutes of SSH logs (public IPs have been redacted):
# journalctl SYSLOG_IDENTIFIER=sshd -S "03:30:00" -U "04:00:00"
-- Logs begin at Fri 2020-01-31 09:26:25 UTC, end at Mon 2020-04-20 08:01:15 UTC. --
Apr 20 03:39:48 myhostname sshd[18438]: Connection from x.x.x.207 port 39332 on 10.0.0.11 port 22 rdomain ""
Apr 20 03:39:48 myhostname sshd[18439]: Connection from x.x.x.207 port 39334 on 10.0.0.11 port 22 rdomain ""
Apr 20 03:39:48 myhostname sshd[18438]: Connection closed by x.x.x.207 port 39332 [preauth]
Apr 20 03:39:48 myhostname sshd[18439]: Connection closed by x.x.x.207 port 39334 [preauth]
Apr 20 03:59:36 myhostname sshd[22186]: Connection from x.x.x.83 port 34876 on 10.0.0.11 port 22 rdomain ""
Apr 20 03:59:36 myhostname sshd[22186]: error: kex_exchange_identification: Connection closed by remote host
And here is my SSH configuration:
# ssh -V
OpenSSH_8.2p1, OpenSSL 1.1.1d 10 Sep 2019
# cat /etc/ssh/sshd_config
UsePAM yes
AddressFamily any
Port 22
X11Forwarding no
PermitRootLogin prohibit-password
GatewayPorts no
PasswordAuthentication no
ChallengeResponseAuthentication no
PrintMotd no # handled by pam_motd
AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2 /etc/ssh/authorized_keys.d/%u
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
KexAlgorithms [email protected],diffie-hellman-group-exchange-sha256
Ciphers [email protected],[email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr
MACs [email protected],[email protected],[email protected],hmac-sha2-512,hmac-sha2-256,[email protected]
LogLevel VERBOSE
UseDNS no
AllowUsers root
AuthenticationMethods publickey
MaxStartups 3:100:60
After searching the web, I have seen references to MaxStartups indicating that it could be the reason for this error but after changing the default value as shown in my sshd_config and attempting more than 3 connections, the server unambiguously indicates the probem
Apr 20 07:26:59 myhostname sshd[31468]: drop connection #3 from [x.x.x.226]:54986 on [10.0.0.11]:22 past MaxStartups
So, what causes error: kex_exchange_identification: Connection closed by remote host?
Weirdly, none actually try to authenticate to open a session.
Some spiders and services like Shodan scans public ipv4 addresses for open services, e.g. salt masters, ftp servers, RDPs, and also SSH services. These spiders usually only connect to the services without doing any valid authentication steps.
I get the error:
kex_exchange_identification: Connection closed by remote host in the server logs. What causes that?
I haven't found conclusive answers about that, so... time to browse the source then.
In OpenSSH source code, kex_exchange_identification is a function to exchange server and client identification (duh), and the specified error happened if the socket connection between OpenSSH server and client is interrupted (see EPIPE), i.e. client already closed its connection.
I've just had this exact issue, and the cause was that I had a port translation happening internally to the load balancer, meaning that my ssh connections were reaching the host on port 80 instead of port 22.
The host was they rightly terminating the connections, and the error message returned to my terminal was as follows;
~/Documents/Projects$ ssh -vvvvA [email protected]
OpenSSH_8.1p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/dave/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 47: Applying options for *
debug2: resolve_canonicalize: hostname xx.xx.xx.250 is address
debug2: ssh_connect_direct
debug1: Connecting to xx.xx.xx.250 [xx.xx.xx.250] port 22.
debug1: Connection established.
debug1: identity file /Users/dave/.ssh/id_rsa type 0
debug1: identity file /Users/dave/.ssh/id_rsa-cert type -1
debug1: identity file /Users/dave/.ssh/id_dsa type -1
debug1: identity file /Users/dave/.ssh/id_dsa-cert type -1
debug1: identity file /Users/dave/.ssh/id_ecdsa type -1
debug1: identity file /Users/dave/.ssh/id_ecdsa-cert type -1
debug1: identity file /Users/dave/.ssh/id_ed25519 type -1
debug1: identity file /Users/dave/.ssh/id_ed25519-cert type -1
debug1: identity file /Users/dave/.ssh/id_xmss type -1
debug1: identity file /Users/dave/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.1
debug1: kex_exchange_identification: banner line 0: HTTP/1.1 400 Bad Request
debug1: kex_exchange_identification: banner line 1: Server: nginx/1.14.0 (Ubuntu)
debug1: kex_exchange_identification: banner line 2: Date: Fri, 20 Nov 2020 09:30:23 GMT
debug1: kex_exchange_identification: banner line 3: Content-Type: text/html
debug1: kex_exchange_identification: banner line 4: Content-Length: 182
debug1: kex_exchange_identification: banner line 5: Connection: close
debug1: kex_exchange_identification: banner line 6:
debug1: kex_exchange_identification: banner line 7: <html>
debug1: kex_exchange_identification: banner line 8: <head><title>400 Bad Request</title></head>
debug1: kex_exchange_identification: banner line 9: <body bgcolor="white">
debug1: kex_exchange_identification: banner line 10: <center><h1>400 Bad Request</h1></center>
debug1: kex_exchange_identification: banner line 11: <hr><center>nginx/1.14.0 (Ubuntu)</center>
debug1: kex_exchange_identification: banner line 12: </body>
debug1: kex_exchange_identification: banner line 13: </html>
kex_exchange_identification: Connection closed by remote host
Fixed the internal port translation, and now the problem has gone away.
I resolved my issue with 'kex_exchange_identification: Connection closed by remote host' when I noticed I was trying to connect using the Server IP when I should have been using the Private IP.
My set up may be worlds apart from all of you, just thought to pass on my own discovery.
EDIT:
With some hosting providers you will have two IPs, one is public, one is private, the private is the one you should use in this instance.
You either know or don't, I appreciate this will not apply to everyone, which is why I say it may be a different set up.
No other answers worked for me, until I used the private key.