SSH has started prompting for password

I am desperate now.....I have trawled the internet, I have tried umpteen different workarounds, permission resets etc but nothing has helped so far.

SSH connectivity was previously working for years without issue, it stopped working on 30th December for no apparent reason. I have tried re-creating the keys, copying them to the target server, adding them to AUTHORIZED_KEYS but it made no difference. I have tried setting the permissions as per many web suggestions but no luck.

I have 2 servers, both Solaris 10.

Server 1 (Source of connection)

[~/.ssh] $ ls -altr
total 106
-rwxr-xr-x   1 informat informat     606 May  5  2016 id_dsa.pub.pfolio-ukbhu051p
-rwxr-xr-x   1 informat informat     627 May  5  2016 id_dsa.pub.eadmzu082p
-rwxr-xr-x   1 informat informat     606 May  5  2016 id_dsa.pub.51p-pfolio
-rwxr-xr-x   1 informat informat    1688 May  5  2016 authorized_keys
-rwxr-xr-x   1 informat informat    6449 Aug 24  2016 known_hosts_F0074033
-rwxr-xr-x   1 informat informat    7093 Apr 23  2017 known_hosts_20170423
-rwxr-xr-x   1 root     root        5422 Mar 14  2018 known_hosts-140318
drwxr-xr-x  13 informat informat    1024 Nov  9  2019 ..
-rwxr-xr-x   1 informat informat    6452 Jan  6 13:45 known_hosts.adw.bak
-rwxr-xr-x   1 informat informat    1688 Jan  7 10:27 authorized_keys.20210107
-rw-------   1 informat informat     887 Jan  7 10:27 id_rsa.20210107
-rw-r--r--   1 informat informat     229 Jan  7 10:27 id_rsa.pub.20210107
-rwxr-xr-x   1 informat informat    5954 Jan  7 10:27 known_hosts.20210107
-rw-------   1 informat informat     887 Jan  7 11:00 id_rsa
-rw-r--r--   1 informat informat     229 Jan  7 11:01 id_rsa.pub
drwx------   2 informat informat    1024 Jan  8 08:50 .
-rw-r--r--   1 informat informat     229 Jan  8 08:50 identity
-rwxr-xr-x   1 informat informat    5954 Jan  8 09:09 known_hosts

Server 2 (Target for connection)

 -> ls -altr
total 194
-rw-r--r--   1 pfolio   cms         5975 Sep 15  2012 known_hosts.bck
-rw-r--r--   1 pfolio   cms         9880 Sep 15  2012 known_hosts_old
-rw-r-----   1 pfolio   cms          226 Sep 15  2012 id_rsa.pub
-rw-r-----   1 pfolio   cms          226 Sep 15  2012 id_rsa.020p
-rw-------   1 pfolio   cms          887 Sep 15  2012 id_rsa
-rw-r-----   1 pfolio   cms          606 Sep 15  2012 id_dsa.pub
-rw-------   1 pfolio   cms          668 Sep 15  2012 id_dsa
-rw-------   1 pfolio   cms         1167 Sep 15  2012 authorized_keys.old
-rw-r--r--   1 pfolio   cms        14750 Apr 17  2016 known_hosts_UKBHSFILES_CR
-rw-r-----   1 pfolio   cms          229 May 25  2016 id_rsa.informat-eadmzu061p.pub
-rw-r--r--   1 pfolio   cms        16648 Apr 23  2017 known_hosts_20170423
-rw-r--r--   1 pfolio   cms        15836 Mar 19  2018 known_hosts.bak
-rw-r--r--   1 pfolio   cms        16241 Mar 19  2018 known_hosts
-rw-------   1 pfolio   cms          458 Jan  6 13:40 authorized_keys.adw.bak
-rw-r-----   1 pfolio   cms          229 Jan  6 13:41 id_rsa.pub.informat-eadmzu091p.bak
-rwx------   1 pfolio   cms          229 Jan  6 13:42 id_rsa.pub.informat-eadmzu091p
-rw-r--r--   1 pfolio   cms          229 Jan  7 10:35 id_rsa_new.pub.informat-eadmzu091p
-rw-------   1 pfolio   cms          458 Jan  7 10:36 authorized_keys.20210107
-rw-r--r--   1 pfolio   cms          229 Jan  7 11:02 informat_eadmzu091p.pub
drwx------   2 pfolio   cms         1024 Jan  7 11:03 .
-rw-------   1 pfolio   cms          458 Jan  7 11:03 authorized_keys
drwxrwxr-x  19 pfolio   cms         1024 Jan  7 16:00 ..

Debug of the connection attempt.....it appears to send the public key but is not accepted. I have checked on the target server for logs but can't see anything of interest.

debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug3: check_host_in_hostfile: filename /export/informatica64/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 17
debug3: check_host_in_hostfile: filename /export/informatica64/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 16
debug1: Host 'ukbhc052p' is known and matches the RSA host key.
debug1: Found key in /export/informatica64/.ssh/known_hosts:17
debug1: bits set: 2080/4095
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug3: kex_reset_dispatch -- should we dispatch_set(KEXINIT) here? 0 && !0
debug2: set_newkeys: mode 1
debug1: set_newkeys: setting new keys for 'out' mode
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: set_newkeys: setting new keys for 'in' mode
debug1: SSH2_MSG_NEWKEYS received
debug1: done: ssh_kex2.
debug1: send SSH2_MSG_SERVICE_REQUEST
debug2: service_accept: ssh-userauth
debug1: got SSH2_MSG_SERVICE_ACCEPT
debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive
debug3: start over, passed a different list gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-keyex
debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-keyex
debug1: Next authentication method: gssapi-keyex
debug2: we did not send a packet, disable method
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Failed to acquire GSS-API credentials for any mechanisms (No credentials were supplied, or the credentials were unavailable or inaccessible
Unknown code 0
)
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
**debug1: Next authentication method: publickey
debug1: Trying public key: /export/informatica64/.ssh/identity
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply**
debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive
debug1: Trying public key: /export/informatica64/.ssh/id_rsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive
debug1: Trying private key: /export/informatica64/.ssh/id_dsa
debug3: no such identity: /export/informatica64/.ssh/id_dsa
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1
Password:

Target server SSHD_CONFIG

cat sshd_config
#
# Copyright 2004 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)sshd_config        1.8     04/05/10 SMI"
#
# Configuration file for sshd(1m)

# Protocol versions supported
#
# The sshd shipped in this release of Solaris has support for major versions
# 1 and 2.  It is recommended due to security weaknesses in the v1 protocol
# that sites run only v2 if possible. Support for v1 is provided to help sites
# with existing ssh v1 clients/servers to transition.
# Support for v1 may not be available in a future release of Solaris.
#
# To enable support for v1 an RSA1 key must be created with ssh-keygen(1).
# RSA and DSA keys for protocol v2 are created by /etc/init.d/sshd if they
# do not already exist, RSA1 keys for protocol v1 are not automatically created.

# Uncomment ONLY ONE of the following Protocol statements.

# Only v2 (recommended)
Protocol 2

# Both v1 and v2 (not recommended)
#Protocol 2,1

# Only v1 (not recommended)
#Protocol 1

# Listen port (the IANA registered port number for ssh is 22)
Port 22

# The default listen address is all interfaces, this may need to be changed
# if you wish to restrict the interfaces sshd listens on for a multi homed host.
# Multiple ListenAddress entries are allowed.

# IPv4 only
#ListenAddress 0.0.0.0
# IPv4 & IPv6
ListenAddress ::

# Port forwarding
AllowTcpForwarding no

# If port forwarding is enabled, specify if the server can bind to INADDR_ANY.
# This allows the local port forwarding to work when connections are received
# from any remote host.
GatewayPorts no

# X11 tunneling options
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes

# The maximum number of concurrent unauthenticated connections to sshd.
# start:rate:full see sshd(1) for more information.
#MaxStartups 10:30:100

# Banner to be printed before authentication starts.
#Banner /etc/issue

# Should sshd print the /etc/motd file and check for mail.
# On Solaris it is assumed that the login shell will do these (eg /etc/profile).
PrintMotd yes

# KeepAlive specifies whether keep alive messages are sent to the client.
# See sshd(1) for detailed description of what this means.
# Note that the client may also be sending keep alive messages to the server.
KeepAlive yes

# Syslog facility and level
SyslogFacility auth
LogLevel info

#
# Authentication configuration
#

# Host private key files
# Must be on a local disk and readable only by the root user (root:sys 600).
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key

# Default Encryption algorithms and Message Authentication codes
#Ciphers        aes128-ctr,aes128-cbc,arcfour,3des-cbc,blowfish-cbc
#MACS   hmac-md5,hmac-sha1,hmac-sha1-96,hmac-md5-96

# Length of the server key
# Default 768, Minimum 512
ServerKeyBits 768

# sshd regenerates the key every KeyRegenerationInterval seconds.
# The key is never stored anywhere except the memory of sshd.
# The default is 1 hour (3600 seconds).
KeyRegenerationInterval 3600

# Ensure secure permissions on users .ssh directory.
#StrictModes yes

# Length of time in seconds before a client that hasn't completed
# authentication is disconnected.
# Default is 600 seconds. 0 means no time limit.
LoginGraceTime 600

# Maximum number of retries for authentication
# Default is 6. Default (if unset) for MaxAuthTriesLog is MaxAuthTries / 2
MaxAuthTriesLog 3

# Are logins to accounts with empty passwords allowed.
# If PermitEmptyPasswords is no, pass PAM_DISALLOW_NULL_AUTHTOK
# to pam_authenticate(3PAM).
PermitEmptyPasswords no

# To disable tunneled clear text passwords, change PasswordAuthentication to no.
PasswordAuthentication yes

# Use PAM via keyboard interactive method for authentication.
# Depending on the setup of pam.conf(4) this may allow tunneled clear text
# passwords even when PasswordAuthentication is set to no. This is dependent
# on what the individual modules request and is out of the control of sshd
# or the protocol.
PAMAuthenticationViaKBDInt yes

# Are root logins permitted using sshd.
# Note that sshd uses pam_authenticate(3PAM) so the root (or any other) user
# maybe denied access by a PAM module regardless of this setting.
# Valid options are yes, without-password, no.
PermitRootLogin yes

# sftp subsystem
Subsystem       sftp    internal-sftp


# SSH protocol v1 specific options
#
# The following options only apply to the v1 protocol and provide
# some form of backwards compatibility with the very weak security
# of /usr/bin/rsh.  Their use is not recommended and the functionality
# will be removed when support for v1 protocol is removed.

# Should sshd use .rhosts and .shosts for password less authentication.
IgnoreRhosts yes
RhostsAuthentication no

# Rhosts RSA Authentication
# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts.
# If the user on the client side is not root then this won't work on
# Solaris since /usr/bin/ssh is not installed setuid.
RhostsRSAAuthentication no

# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication.
#IgnoreUserKnownHosts yes

# Is pure RSA authentication allowed.
# Default is yes
RSAAuthentication yes

#TFS specific Settings for USer access
AllowUsers root orca waltersg mon oracle dbaop2 burmane winsera kennedn qualys smithd doublem pfolio lipedee adminsc bowryc gibsonc mckennae harrisj waltersj kucukeksiv robinsona godfreyr robinsond wilsoni reillyj heardo futerss skerrittt bowryc gatesb nortjed egglesj detarantoa

MaxAuthTries 3

Any help gratefully received

If the SSH connection attempt has started to prompt for a password for the user you're trying to connect as, change:

PasswordAuthentication yes

to

PasswordAuthentication no

Inside your sshd_config.

Furthermore it is trying to find your private key in a location it possibly isn't from the looks of things:

debug1: Trying private key: /export/informatica64/.ssh/id_dsa
debug3: no such identity: /export/informatica64/.ssh/id_dsa

I also cannot see AuthorizedKeysFile location in your sshd_config to point to where your authorized_keys files are located.

A quick suggestion: uncomment #StrictModes yes to make sure it's using the correct permissions on the ~/.ssh directories for your users.

See: https://docs.oracle.com/cd/E86824_01/html/E54775/sshd-config-4.html

Once you disable PasswordAuthentication and restart the ssh/sshd service it may prompt you for a different error and you will be able to specifically see what is happening or it should fix it.