The Problem

Trying to connect to my QNAP server from two different computers, which previously could connect via ssh without a password -- but now a password is asked for each time. What went wrong, and how do I fix it?

What I've tried

I've rooted around for ideas and checked the following, all with no success:

• Add public keys from client(s) to server: (~/.ssh/authorized_keys).

• Set file permissions: The admin (root) account on the server has run both:

  chmod 700 ~/.ssh/

  chmod 600 ~/.ssh/*

• Recreating and uploading the keys

Addtional info

I can't think of updates that would have happened that match the timeline of the keys no longer working, so I don't know why thy would have stopped functioning suddenly.

@Yevhen-Stasiv suggested checking my config -- turns out this file is reset every time the server reboots; nonetheless the values are:

[~/.ssh] # cat sshd_config
Protocol 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
PermitRootLogin yes
UseDNS no
Subsystem sftp /usr/libexec/sftp-server
AllowTcpForwarding no
AllowUsers admin

The version of OpenSSL on my server is:

# ssh -V
OpenSSH_8.0p1, OpenSSL 1.0.2k  26 Jan 2017

I'm not sure if that makes any difference, but there it is.

Finally, increasing verbosity of the command (and preventing password authentication), this is what I'm seeing:

$ ssh -vv -o PasswordAuthentication=no admin@nasdekleer
OpenSSH_8.1p1, OpenSSL 1.1.1d  10 Sep 2019
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolving "nasdekleer" port 22
debug2: ssh_connect_direct
debug1: Connecting to nasdekleer [10.132.197.180] port 22.
debug1: Connection established.
debug1: identity file /c/Users/Ryan/.ssh/id_rsa type 0
debug1: identity file /c/Users/Ryan/.ssh/id_rsa-cert type -1
debug1: identity file /c/Users/Ryan/.ssh/id_dsa type -1
debug1: identity file /c/Users/Ryan/.ssh/id_dsa-cert type -1
debug1: identity file /c/Users/Ryan/.ssh/id_ecdsa type -1
debug1: identity file /c/Users/Ryan/.ssh/id_ecdsa-cert type -1
debug1: identity file /c/Users/Ryan/.ssh/id_ed25519 type -1
debug1: identity file /c/Users/Ryan/.ssh/id_ed25519-cert type -1
debug1: identity file /c/Users/Ryan/.ssh/id_xmss type -1
debug1: identity file /c/Users/Ryan/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to nasdekleer:22 as 'admin'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal

[...output omitted for brevity...]

debug1: Server host key: ssh-rsa SHA256:/WeVhQddipMcY6dZdQ+HgL+kMhsUEqRqLEsdpK14MNM
debug1: Host 'nasdekleer' is known and matches the RSA host key.
debug1: Found key in /c/Users/Ryan/.ssh/known_hosts:1
debug2: set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /c/Users/Ryan/.ssh/id_rsa RSA SHA256:1sWIuTIkFdDZ9cxqz1va1Y+SnAXAkL61/0lLTe9fW5c
debug1: Will attempt key: /c/Users/Ryan/.ssh/id_dsa
debug1: Will attempt key: /c/Users/Ryan/.ssh/id_ecdsa
debug1: Will attempt key: /c/Users/Ryan/.ssh/id_ed25519
debug1: Will attempt key: /c/Users/Ryan/.ssh/id_xmss
debug2: pubkey_prepare: done
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering public key: /c/Users/Ryan/.ssh/id_rsa RSA SHA256:1sWIuTIkFdDZ9cxqz1va1Y+SnAXAkL61/0lLTe9fW5c
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Trying private key: /c/Users/Ryan/.ssh/id_dsa
debug1: Trying private key: /c/Users/Ryan/.ssh/id_ecdsa
debug1: Trying private key: /c/Users/Ryan/.ssh/id_ed25519
debug1: Trying private key: /c/Users/Ryan/.ssh/id_xmss
debug2: we did not send a packet, disable method
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
admin@nasdekleer: Permission denied (publickey,password,keyboard-interactive).

Update

I haven't been able to track down the logs on the server, however, I found I could run my own ssh daemon on a different port by logging into the server and running:

/usr/sbin/sshd -p 50023 -D -d -e -f /etc/config/ssh/sshd_config

...and then going to my client and attempting a login like so:

ssh -v -p 50023 admin@nasdekleer

and then can see the output on the server-side:

[ ... ]

debug1: userauth-request for user admin service ssh-connection method none [preauth]
debug1: attempt 0 failures 0 [preauth]
debug1: userauth-request for user admin service ssh-connection method publickey [preauth]
debug1: attempt 1 failures 0 [preauth]
debug1: userauth_pubkey: test pkalg rsa-sha2-512 pkblob RSA SHA256:1sWIuTIkFdDZ9cxqz1va1Y+SnAXAkL61/0lLTe9fW5c [preauth]
debug1: temporarily_use_uid: 0/0 (e=0/0)
debug1: trying public key file /root/.ssh/authorized_keys
debug1: fd 4 clearing O_NONBLOCK
Authentication refused: bad ownership or modes for directory /mnt/HDA_ROOT

[...]

So it's seeming to me that this might be a file permission issue still?

The issue is most likely server side. A software update may have rewritten the sshd config, verify that the configuration is still set to permit key auth.

Have a look at the sshd log file for any clues, if there is nothing there try running sshd in foreground & debug mode to quickly see what is going on.

So getting more info from the server was definitely the ticket.

Open a ssh daemon on the server with:

/usr/sbin/sshd -p 50023 -D -d -e -f /etc/config/ssh/sshd_config

and try to log in from a client with :

ssh -v -p 50023

Seeing the message:

Authentication refused: bad ownership or modes for directory /mnt/HDA_ROOT

Checking the directory with ll /mnt, got this output:

drwxr-xr-x   10 admin    administ       220 Jan  8 17:44 ./
drwxr-xr-x   20 admin    administ       480 Jan  8 19:35 ../
drwxrwxrwx   12 httpdusr administ      4.0k Jan  8 17:44 HDA_ROOT/

Yikes. I must have somehow changed the ownership of the root home directory when setting up some stuff for a webserver!

Switched the ownership back with chown admin /mnt/HDA_ROOT.

And now everything is hunky-dory again! :D