Why does redis report limit of 1024 files even after update to limits.conf?

I see this error at the top of my redis.log file:

Current maximum open files is 1024. maxclients has been reduced to 4064 to compensate for low ulimit.

I have followed these steps to the letter (and rebooted):

Moreover, I see this when I run ulimit:

ubuntu@ip-XX-XXX-XXX-XXX:~$ ulimit -n
65535

Is this error specious? If not, what other steps do I need to perform? I am running redis 2.8.13 (tip of the tree) on Ubuntu LTS 14.04.1 (again, tip of the tree).

Here is the user info:

ubuntu@ip-XX-XXX-XXX-XXX:~$ ps aux | grep redis
root      1027  0.0  0.0  66328  2112 ?        Ss   20:30   0:00 sudo -u ubuntu /usr/local/bin/redis-server /etc/redis/redis.conf
ubuntu    1107 19.2 48.8 7629152 7531552 ?     Sl   20:30   2:21 /usr/local/bin/redis-server *:6379               

The server is therefore running as ubuntu.

Here are my limits.conf file without comments:

ubuntu@ip-XX-XXX-XXX-XXX:~$ cat /etc/security/limits.conf | sed '/^#/d;/^$/d'
ubuntu soft nofile 65535
ubuntu hard nofile 65535
root soft nofile 65535
root hard nofile 65535

And here is the output of sysctl fs.file-max:

ubuntu@ip-XX-XXX-XXX-XXX:~$ sysctl -a| grep fs.file-max
sysctl: permission denied on key 'fs.protected_hardlinks'
sysctl: permission denied on key 'fs.protected_symlinks'
fs.file-max = 1528687
sysctl: permission denied on key 'kernel.cad_pid'
sysctl: permission denied on key 'kernel.usermodehelper.bset'
sysctl: permission denied on key 'kernel.usermodehelper.inheritable'
sysctl: permission denied on key 'net.ipv4.tcp_fastopen_key'

as sudo

ubuntu@ip-10-102-154-226:~$ sudo sysctl -a| grep fs.file-max
fs.file-max = 1528687

Also, I see this error at the top of the redis.log file, not sure if it's related. It makes sense that the ubuntu user isn't allowed to change max open files, but given the high ulimits I have tried to set he shouldn't need to:

[1050] 23 Aug 21:00:43.572 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
[1050] 23 Aug 21:00:43.572 # Redis can't set maximum open files to 10032 because of OS error: Operation not permitted.

You should edit your files inside /etc/pam.d/ directory.

In your case when you run sudo -u ubuntu /usr/local/bin/redis-server, you should add following line to /etc/pam.d/sudo or to /etc/pam.d/common-session-noninteractive in case /etc/pam.d/sudo includes this one:

session required pam_limits.so

This should help setting configuration provided inside /etc/security/limits.conf.

Another suggestion for users finding this post whose issue is not related to pam_limits.so. In my case, Redis was being launched via supervisord. In this case, the method by which supervisord switched user contexts caused the new session to be run with the system default limits, not those I'd configured for the user.

The solution in that case was found here. The short of it is, you need to define the limit via ulimit -n as part of the startup command in supervisord.

Example:

[program:redis-a]
command=bash -c "ulimit -n 32768; exec /usr/local/bin/redis-server /etc/redis/a.conf"