Kerberos pre-authentication failed on nfs mount

I have the following nfs export:

/home/users     192.168.1.0/24(rw,sec=krb5p,no_subtree_check,nohide,async,anonuid=65534,anongid=65534)

When trying to mount that on a client I get:

client:/home # mount -t nfs4 -o sec=krb5p server:/home/users /home/users -vvv
mount.nfs4: timeout set for Sun May 12 21:13:56 2013
mount.nfs4: trying text-based options 'sec=krb5p,addr=192.168.1.2,clientaddr=192.168.1.62'
mount.nfs4: mount(2): Permission denied
mount.nfs4: access denied by server while mounting server:/home/users

On the server syslog I get:

May 12 19:59:48 server krb5kdc[2704]: AS_REQ (4 etypes {18 17 16 23}) 192.168.1.62: NEEDED_PREAUTH: nfs/client.localdomain@REALM for krbtgt/REALM@REALM, Additional pre-authentication required
May 12 19:59:48 server krb5kdc[2704]: preauth (encrypted_timestamp) verify failure: Decrypt integrity check failed
May 12 19:59:48 server krb5kdc[2704]: AS_REQ (4 etypes {18 17 16 23}) 192.168.1.62: PREAUTH_FAILED: nfs/client.localdomain@REALM for krbtgt/REALM@REALM, Decrypt integrity check failed

As far as I can tell the keytabs have been set correctly:

client:

client:/home # ktutil 
ktutil:  rkt /etc/krb5.keytab 
ktutil:  list
slot KVNO Principal
---- ---- ---------------------------------------------------------------------
   1    4                        nfs/client.localdomain@REALM
   2    4                        nfs/client.localdomain@REALM
   3    4                        nfs/client.localdomain@REALM
   4    4                        nfs/client.localdomain@REALM
   5    4                       host/client.localdomain@REALM
   6    4                       host/client.localdomain@REALM
   7    4                       host/client.localdomain@REALM
   8    4                       host/client.localdomain@REALM

Server:

root@server:~# ktutil 
ktutil:  rkt /etc/krb5.keytab 
ktutil:  list
slot KVNO Principal
---- ---- ---------------------------------------------------------------------
   1    3                     host/server.localdomain@REALM
   2    3                     host/server.localdomain@REALM
   3    3                     host/server.localdomain@REALM
   4    3                     host/server.localdomain@REALM
   5    2                     cifs/server.localdomain@REALM
   6    2                   HTTP/server.localdomain@REALM
   7    2                   HTTP/server.localdomain@REALM
   8    2                   HTTP/server.localdomain@REALM
   9    2                   HTTP/server.localdomain@REALM
  10    2                      nfs/server.localdomain@REALM
  11    2                      nfs/server.localdomain@REALM
  12    2                      nfs/server.localdomain@REALM
  13    2                      nfs/server.localdomain@REALM

Kerberos user authentication with kinit on the client works fine.

When I attempt the same nfs mount on server itself, it succeeds.

What is pre-authentication and what are the possible causes for the pre-authentication to fail?

Solution 1:

Turns out that regenerating the client keytab fixed the problem somehow.

Solution 2:

For future reference for people with a similar issue, I managed to solve it by purging the keys from the keytab file for the user, then removing the user, then adding the user back. Example:

# kadmin.local

purgekeys THEUSER

delprinc THEUSER

addprinc -randkey THEUSER

ktadd THEUSER

Then redistribute the /etc/krb5.keytab file to all the clients.

It was not sufficient to just delete and remake the user. I had to purge the keys first.