SSH apparently not reading keys inside ~/.ssh
This is output from a Ubuntu 16.04 client:
OpenSSH_7.2p2 Ubuntu-4, OpenSSL 1.0.2g-fips 1 Mar 2016
debug1: Reading configuration data /home/manuth/.ssh/config
debug1: /home/manuth/.ssh/config line 1: Applying options for r2d2.manuth.life
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to r2d2.manuth.life [103.12.163.90] port 900.
debug1: Connection established.
debug1: identity file /home/manuth/.ssh/dqar-rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/manuth/.ssh/dqar-rsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4
ssh_exchange_identification: Connection closed by remote host
The path and permission for /home/manuth/.ssh/dqar-rsa are also correct:
$ ls -l /home/manuth/.ssh/dqar-rsa*
-rw------- 1 manuth manuth 3243 Nov 7 11:27 /home/manuth/.ssh/dqar-rsa
-rw-r--r-- 1 manuth manuth 740 Nov 7 11:27 /home/manuth/.ssh/dqar-rsa.pub
The entry for this host in ~/.ssh/config is:
host r2d2.manuth.life
IdentityFile ~/.ssh/dqar-rsa
Port 900
IdentitiesOnly yes
ForwardX11 yes
If I try commenting the IdentityFile line, it doesn't even read any id_* inside ~/.ssh:
OpenSSH_7.2p2 Ubuntu-4, OpenSSL 1.0.2g-fips 1 Mar 2016
debug1: Reading configuration data /home/manuth/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to r2d2.manuth.life [103.12.163.90] port 900.
debug1: Connection established.
debug1: identity file /home/manuth/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/manuth/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/manuth/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/manuth/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/manuth/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/manuth/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/manuth/.ssh/id_ed25519 type 4
debug1: key_load_public: No such file or directory
debug1: identity file /home/manuth/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4
ssh_exchange_identification: Connection closed by remote host
This seems to come up all of a sudden today.
Edit: contents inside ~/.ssh:
$ ls -la
insgesamt 36
drwx------ 2 manuth manuth 4096 Nov 7 16:28 .
drwxr-xr-x 53 manuth manuth 4096 Nov 7 13:31 ..
-rw-r--r-- 1 manuth manuth 805 Nov 7 12:10 authorized_keys
-rw-r--r-- 1 manuth manuth 543 Nov 7 13:47 config
-rw------- 1 manuth manuth 411 Nov 7 12:10 dqar-ed25519
-rw-r--r-- 1 manuth manuth 96 Nov 7 12:10 dqar-ed25519.pub
-rw------- 1 manuth manuth 3243 Nov 7 12:10 dqar-rsa
-rw-r--r-- 1 manuth manuth 740 Nov 7 12:10 dqar-rsa.pub
-rw-r--r-- 1 manuth manuth 1990 Nov 7 15:14 known_hosts
Edit 2: Ah hah, the console output was several lines of this:
Nov 7 13:51:32 dqar sshd [11316]: fatal: Missing privilege separation directory: /var/empty
The server in question is FreeBSD 10.3.
debug1: identity file /home/manuth/.ssh/dqar-rsa type 1
Says it read the file successfully. The failure is somewhere else.
ssh_exchange_identification: Connection closed by remote host
This is the real problem. You can't establish SSH connection to the remote host for some reason. You are probably blacklisted using /etc/hosts.deny or the server is failing to accept the connections and initiate SSH protocol for other reasons (missing directories, disk failures, full disk, etc.). The logs from the server will tell you more.
Nov 7 13:51:32 dqar sshd [11316]: fatal: Missing privilege separation directory: /var/empty
Explains it pretty much. You need to create this directory if it was removed for some reasons and set proper permissions (not writable by any other users than root).
So the reason was that, somehow, there was no /var/empty. I created it back from this forum post (I know it's for Juniper, but it's working on this FreeBSD as well):
http://forums.juniper.net/t5/Ethernet-Switching/Missing-privilege-separation-directory-var-empty/td-p/173832