Can't password-connect with SSH to CentOS7 machine we sent a password packet, error

I'm trying to reach a machine (CentOS7) over SSH with password authentication.

The machine is on a private network off the internet. I tried to connect from a Windows 10 host and from another CentOS7 server (and from VSCode SSH Remote Extension). From the 3 client i'm having a similar error (describe a little below)

It was working perfectly last Friday. The only thing different I have noticed is that at the first connection try SSH throwed "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!". So deleted the know_hosts files on both client.

On the target server

  • I tried disabling SElinux and reboot
  • I have checked that SSHD service was OK
  • Restarted the service
  • Checked the permission on /dev/tty (like suggested here it is equal to : crw-rw-rw- 1 root tty 5, 0 Jun 28 16:15 /dev/tty
  • And exec : sudo chown $USER:$USER ~/ -R; sudo chmod o-rwx ~/ -R

None of those tries changed the behavior of the error.

The errors seems to be related to :

admin@XXXXXXXX's password:
debug3: send packet: type 50
debug2: we sent a password packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
Permission denied, please try again.

OR 

admin@XXXXXXX's password:
debug3: send packet: type 50
debug2: we sent a password packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
Permission denied, please try again.
debug3: failed to open file:C:/dev/tty error:3
debug1: read_passphrase: can't open /dev/tty: No such file or directory

Edit 1 :

I checked the /var/log/secure file while doing ssh connection test but nothing is logged. I checked that the port 22 was opened with firewall-cmd I restarted it and the service with it. Still no log and same error message from both client.

Here the output log from the Windows cmd:

C:\Users\212682057>ssh -vvv admin@XXXXXXX
OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
debug1: Reading configuration data C:\\Users\\212682057/.ssh/config
debug3: Failed to open file:C:/ProgramData/ssh/ssh_config error:2
debug2: resolve_canonicalize: hostname XXXXXXX is address
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to XXXXXXX [XXXXXXX] port 22.
debug1: Connection established.
debug3: Failed to open file:C:/Users/212682057/.ssh/id_rsa error:2
debug3: Failed to open file:C:/Users/212682057/.ssh/id_rsa.pub error:2
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\212682057/.ssh/id_rsa type -1
debug3: Failed to open file:C:/Users/212682057/.ssh/id_rsa-cert error:2
debug3: Failed to open file:C:/Users/212682057/.ssh/id_rsa-cert.pub error:2
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\212682057/.ssh/id_rsa-cert type -1
debug3: Failed to open file:C:/Users/212682057/.ssh/id_dsa error:2
debug3: Failed to open file:C:/Users/212682057/.ssh/id_dsa.pub error:2
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\212682057/.ssh/id_dsa type -1
debug3: Failed to open file:C:/Users/212682057/.ssh/id_dsa-cert error:2
debug3: Failed to open file:C:/Users/212682057/.ssh/id_dsa-cert.pub error:2
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\212682057/.ssh/id_dsa-cert type -1
debug3: Failed to open file:C:/Users/212682057/.ssh/id_ecdsa error:2
debug3: Failed to open file:C:/Users/212682057/.ssh/id_ecdsa.pub error:2
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\212682057/.ssh/id_ecdsa type -1
debug3: Failed to open file:C:/Users/212682057/.ssh/id_ecdsa-cert error:2
debug3: Failed to open file:C:/Users/212682057/.ssh/id_ecdsa-cert.pub error:2
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\212682057/.ssh/id_ecdsa-cert type -1
debug3: Failed to open file:C:/Users/212682057/.ssh/id_ed25519 error:2
debug3: Failed to open file:C:/Users/212682057/.ssh/id_ed25519.pub error:2
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\212682057/.ssh/id_ed25519 type -1
debug3: Failed to open file:C:/Users/212682057/.ssh/id_ed25519-cert error:2
debug3: Failed to open file:C:/Users/212682057/.ssh/id_ed25519-cert.pub error:2
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\212682057/.ssh/id_ed25519-cert type -1
debug3: Failed to open file:C:/Users/212682057/.ssh/id_xmss error:2
debug3: Failed to open file:C:/Users/212682057/.ssh/id_xmss.pub error:2
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\212682057/.ssh/id_xmss type -1
debug3: Failed to open file:C:/Users/212682057/.ssh/id_xmss-cert error:2
debug3: Failed to open file:C:/Users/212682057/.ssh/id_xmss-cert.pub error:2
debug1: key_load_public: No such file or directory
debug1: identity file C:\\Users\\212682057/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_7.7
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.3
debug1: match: OpenSSH_4.3 pat OpenSSH_2*,OpenSSH_3*,OpenSSH_4* compat 0x00000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to XXXXXXX:22 as 'admin'
debug3: hostkeys_foreach: reading file "C:\\Users\\212682057/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file C:\\Users\\212682057/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from XXXXXXX
debug3: Failed to open file:C:/Users/212682057/.ssh/known_hosts2 error:2
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2
debug3: order_hostkeyalgs: prefer hostkeyalgs: [email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: [email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa,[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none
debug2: compression stoc: none
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,ssh-dss
debug2: ciphers ctos: aes128-cbc,aes192-cbc,aes256-cbc,[email protected],aes128-ctr,aes192-ctr,aes256-ctr
debug2: ciphers stoc: aes128-cbc,aes192-cbc,aes256-cbc,[email protected],aes128-ctr,aes192-ctr,aes256-ctr
debug2: MACs ctos: hmac-sha1,hmac-ripemd160,[email protected]
debug2: MACs stoc: hmac-sha1,hmac-ripemd160,[email protected]
debug2: compression ctos: none,[email protected]
debug2: compression stoc: none,[email protected]
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: diffie-hellman-group-exchange-sha1
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha1 compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha1 compression: none
debug3: send packet: type 34
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(2048<7680<8192) sent
debug3: receive packet: type 31
debug1: got SSH2_MSG_KEX_DH_GEX_GROUP
debug2: bits set: 3078/6144
debug3: send packet: type 32
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug3: receive packet: type 33
debug1: got SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: ssh-rsa SHA256:HYbSyZf7zESLuQUH81smbvmGafC2oAXVPzI+q9IQtJs
debug3: hostkeys_foreach: reading file "C:\\Users\\212682057/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file C:\\Users\\212682057/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from XXXXXXX
debug3: Failed to open file:C:/Users/212682057/.ssh/known_hosts2 error:2
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2
debug1: Host 'XXXXXXX' is known and matches the RSA host key.
debug1: Found key in C:\\Users\\212682057/.ssh/known_hosts:1
debug2: bits set: 3014/6144
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 4294967296 blocks
debug3: unable to connect to pipe \\\\.\\pipe\\openssh-ssh-agent, error: 2
debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory
debug2: key: C:\\Users\\212682057/.ssh/id_rsa (0000000000000000)
debug2: key: C:\\Users\\212682057/.ssh/id_dsa (0000000000000000)
debug2: key: C:\\Users\\212682057/.ssh/id_ecdsa (0000000000000000)
debug2: key: C:\\Users\\212682057/.ssh/id_ed25519 (0000000000000000)
debug2: key: C:\\Users\\212682057/.ssh/id_xmss (0000000000000000)
debug3: send packet: type 5
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 53
debug3: input_userauth_banner
This system is for authorized use only.
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: C:\\Users\\212682057/.ssh/id_rsa
debug3: no such identity: C:\\Users\\212682057/.ssh/id_rsa: No such file or directory
debug1: Trying private key: C:\\Users\\212682057/.ssh/id_dsa
debug3: no such identity: C:\\Users\\212682057/.ssh/id_dsa: No such file or directory
debug1: Trying private key: C:\\Users\\212682057/.ssh/id_ecdsa
debug3: no such identity: C:\\Users\\212682057/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: C:\\Users\\212682057/.ssh/id_ed25519
debug3: no such identity: C:\\Users\\212682057/.ssh/id_ed25519: No such file or directory
debug1: Trying private key: C:\\Users\\212682057/.ssh/id_xmss
debug3: no such identity: C:\\Users\\212682057/.ssh/id_xmss: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
debug3: failed to open file:C:/dev/tty error:3
debug1: read_passphrase: can't open /dev/tty: No such file or directory
admin@XXXXXXX's password:
debug3: send packet: type 50
debug2: we sent a password packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
Permission denied, please try again.
debug3: failed to open file:C:/dev/tty error:3
debug1: read_passphrase: can't open /dev/tty: No such file or directory

Here is the output log from the centos7 client:

➜  ~ ssh -vvv admin@XXXXXXX
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 58: Applying options for *
debug2: resolving "XXXXXXX" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to XXXXXXX [XXXXXXX] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /home/neito/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/neito/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/neito/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/neito/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/neito/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/neito/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/neito/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/neito/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.3
debug1: match: OpenSSH_4.3 pat OpenSSH_4* compat 0x00000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to XXXXXXX:22 as 'admin'
debug3: hostkeys_foreach: reading file "/home/neito/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/neito/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from XXXXXXX
debug3: order_hostkeyalgs: prefer hostkeyalgs: [email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ext-info-c
debug2: host key algorithms: [email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa,[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,ssh-dss
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],aes128-cbc,aes192-cbc,aes256-cbc
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],aes128-cbc,aes192-cbc,aes256-cbc
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected],zlib
debug2: compression stoc: none,[email protected],zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,ssh-dss
debug2: ciphers ctos: aes128-cbc,aes192-cbc,aes256-cbc,[email protected],aes128-ctr,aes192-ctr,aes256-ctr
debug2: ciphers stoc: aes128-cbc,aes192-cbc,aes256-cbc,[email protected],aes128-ctr,aes192-ctr,aes256-ctr
debug2: MACs ctos: hmac-sha1,hmac-ripemd160,[email protected]
debug2: MACs stoc: hmac-sha1,hmac-ripemd160,[email protected]
debug2: compression ctos: none,[email protected]
debug2: compression stoc: none,[email protected]
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: diffie-hellman-group-exchange-sha1
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha1 compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha1 compression: none
debug1: kex: diffie-hellman-group-exchange-sha1 need=20 dh_need=20
debug1: kex: diffie-hellman-group-exchange-sha1 need=20 dh_need=20
debug3: send packet: type 34
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<7680<8192) sent
debug3: receive packet: type 31
debug1: got SSH2_MSG_KEX_DH_GEX_GROUP
debug2: bits set: 3064/6144
debug3: send packet: type 32
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug3: receive packet: type 33
debug1: got SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: ssh-rsa SHA256:HYbSyZf7zESLuQUH81smbvmGafC2oAXVPzI+q9IQtJs
debug3: hostkeys_foreach: reading file "/home/neito/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/neito/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from XXXXXXX
debug1: Host 'XXXXXXX' is known and matches the RSA host key.
debug1: Found key in /home/neito/.ssh/known_hosts:1
debug2: bits set: 3092/6144
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 4294967296 blocks
debug2: key: /home/neito/.ssh/id_rsa ((nil))
debug2: key: /home/neito/.ssh/id_dsa ((nil))
debug2: key: /home/neito/.ssh/id_ecdsa ((nil))
debug2: key: /home/neito/.ssh/id_ed25519 ((nil))
debug3: send packet: type 5
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 53
debug3: input_userauth_banner
This system is for authorized use only.
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /home/neito/.ssh/id_rsa
debug3: no such identity: /home/neito/.ssh/id_rsa: No such file or directory
debug1: Trying private key: /home/neito/.ssh/id_dsa
debug3: no such identity: /home/neito/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /home/neito/.ssh/id_ecdsa
debug3: no such identity: /home/neito/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /home/neito/.ssh/id_ed25519
debug3: no such identity: /home/neito/.ssh/id_ed25519: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
admin@XXXXXXX's password:
debug3: send packet: type 50
debug2: we sent a password packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
Permission denied, please try again.
admin@XXXXXXX's password:

A coworker deployed another machine on the same ip. Shouldn't have overlooked the "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!" which was indicating exactly the problem.