ssh: "Access denied by PAM account configuration" for one non-root user but not another

On a VM I am initializing I am able to log in as one non-root user (admin) but not another (tbbscraper) over SSH with public key authentication. The only error message I can find in any log file is

Sep 18 17:21:04 [REDACTED] sshd[18942]: fatal: Access denied for user tbbscraper by PAM account configuration [preauth]

On the client side, the syndrome is

$ ssh -v -i [REDACTED] tbbscraper@[REDACTED]
...
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: [REDACTED]
debug1: Authentications that can continue: publickey
debug1: Trying private key: [REDACTED]
debug1: read PEM private key done: type RSA
Connection closed by [REDACTED]

Changing 'tbbscraper' to 'admin' allows a successful login: debug1: Authentication succeeded (publickey). appears instead of the "Connection closed" message.

This doesn't seem to be a permissions problem...

# for x in admin tbbscraper
> do ls -adl /home/$x /home/$x/.ssh /home/$x/.ssh/authorized_keys
> done
drwxr-xr-x 3 admin admin 4096 Sep 18 17:19 /home/admin
drwx------ 2 admin admin 4096 Sep 18 16:53 /home/admin/.ssh
-rw------- 1 admin admin  398 Sep 18 17:19 /home/admin/.ssh/authorized_keys
drwxr-xr-x 3 tbbscraper tbbscraper 4096 Sep 18 17:18 /home/tbbscraper
drwx------ 2 tbbscraper tbbscraper 4096 Sep 18 17:18 /home/tbbscraper/.ssh
-rw------- 1 tbbscraper tbbscraper  398 Sep 18 17:18 /home/tbbscraper/.ssh/authorized_keys

# cmp /home/{admin,tbbscraper}/.ssh/authorized_keys ; echo $?
0

... nor a PAM-level access control problem ...

# egrep -v '^(#|$)' /etc/security/*.conf
#

... so none of the existing answers to similar questions would seem to apply. The only other piece of evidence I've got is:

root@[REDACTED] # su - admin
admin@[REDACTED] $

but

root@[REDACTED] # su - tbbscraper
su: Authentication failure
(Ignored)
tbbscraper@[REDACTED] $

which suggests some larger-scale PAM issue, but I can't find anything obviously wrong with the stuff in /etc/pam.d. Any ideas?

The VM is an EC2 instance, OS is Debian 7.1 (Amazon's off-the-shelf AMI).

Solution 1:

After all that, it turns out to have been a one-character typo in /etc/shadow. Spot the difference:

admin:!:15891:0:99999:7:::
tbbscraper:!::15966:0:99999:7:::

That's right, there are two colons after the exclamation point on the tbbscraper line. That shoves all the fields over one and makes PAM think that the account expired on January 8, 1970.

Solution 2:

Thank you for posting your question. I was getting the same error, but my problem wasn't related to the shadow file. I found my fix and wanted to post an answer as well for anyone else Googling this error. This serverfault question comes up first.

Try checking the /etc/security/access.conf !

We're using Active Directory for authentication, but I needed to login in as a local, non-AD user (jenkins). My boss had originally setup the box with these line in the /etc/security/access.conf:

+:root:ALL
-:ALL:ALL

I changed it to the following and logins now work; I didn't even need to restart any services.

+:jenkins:ALL
+:root:ALL
-:ALL:ALL

Solution 3:

In my case I was renaming local CentOS 6 users, and forgot to rename them in /etc/shadow (who are password-less key-authenticated, didn't pop up in my mind), so the records for the new usernames were just absent in /etc/shadow. In /var/log/secure it was giving me unix_chkpwd error and Access denied by PAM:

    unix_chkpwd[12345]: could not obtain user info (user2)
    sshd[12354]: fatal: Access denied for user user2 by PAM account configuration

Solution 4:

Had the same error message. Shutted down the sshd and restarted it in debug mode

    /usr/sbin/sshd -ddd

this indicated the reason:

    debug3: User autossh not allowed because account is locked
            ...
    input_userauth_request: invalid user <username> [preauth]

Checked account:

    passwd -S <username>

which showed that the account was locked (flag "L") Un-locked the account by setting a new password:

    passwd <username>

Done.

Solution 5:

I got the same problem this morning but the server authenticates users against Active Directory. Turns out the user's domain password had expired.