php-fpm invokes oom-killer

I'm seeing an issue where PHP-FPM works perfectly for a few days, and then decides to fill up all the memory + swap and invoke OOM-Killer. After this happens, the server is completely dead and you can't even SSH into it anymore. A hard restart has to happen to bring things back to normal.

I'm wondering why this is happening and if there's either a fix or a workaround, such as limiting the amount of memory it can use or restarting the process if it's starts using too much.

I've captured the kernel dumps the last few times its happened.

1. http://pastebin.com/raw.php?i=rX6jYDe0
2. http://pastebin.com/raw.php?i=f2qx5GcS

My php-fpm.conf

1. http://pastebin.com/raw.php?i=27hvN27q

My www.conf:

1. http://pastebin.com/raw.php?i=VgYtut9j

Let me know if there's anything else I can give you for more information on why this could be happening.

Solution 1:

Try to reduce pm.max_requests to some lower values (1000). Also, you have high pm.start_servers, pm.start_servers, pm.max_spare_servers values and extremely high pm.max_children

Solution 2:

There is a lot of great information around about OOM. If you want to understand what is happening I recommend to search for it with some more keywords like this

I had this myself once in a customer project and had to take care of this. Unfortunately none of the recommended above did help as this is related to the kernel. OOM is a dirty workaround for overcommitting memory in linux.

What helped (me) was to set some kernel parameters that avoid the OOM altogether. To quickfix it, add the following lines to /etc/sysctl.conf to make the change persistent over boot:

vm.overcommit_memory = 2
vm.overcommit_ratio = 80

I would however recommend to read up on the OOM. You might find some more useful informations about it.