Memory usage of php-cgi processes is growing steadily

I'm trying to set up a web server on a VPS. My problem is that memory usage of php-cgi processes increases over time even though the website is not receiving any traffic at all. (it is behind a firewall for the time being)

The VPS has 360MB RAM. I'm using Debian Lenny 32bit and its lighttpd and php5-cgi packages. Apart from some config changes (listed below), I'm using the stock setup by Debian.

The website is based on Drupal. Using Drupal's devel module, I can tell that memory usage of PHP scripts is less than 20KB on average, and it never exceeds 8MB.

Here are the relevant parts from the output of ps aux:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
www-data 29871  0.0  1.7  54552  6368 ?        Ss   Aug12   0:00 /usr/bin/php-cgi
www-data 29873  0.0  7.4  65808 27468 ?        S    Aug12   0:00 /usr/bin/php-cgi
www-data 29874  0.0  3.7  55808 13736 ?        S    Aug12   0:00 /usr/bin/php-cgi
www-data 29875  0.0  4.3  58040 16204 ?        S    Aug12   0:00 /usr/bin/php-cgi
www-data 29876  0.0  4.4  57444 16288 ?        S    Aug12   0:00 /usr/bin/php-cgi
www-data 29877  0.0  1.7  54552  6368 ?        Ss   Aug12   0:00 /usr/bin/php-cgi
www-data 29879  0.0  9.6  67140 35684 ?        S    Aug12   0:26 /usr/bin/php-cgi
www-data 29880  0.0  6.6  59172 24492 ?        S    Aug12   0:23 /usr/bin/php-cgi
www-data 29881  0.0  7.1  59784 26388 ?        S    Aug12   0:22 /usr/bin/php-cgi
www-data 29882  0.0  7.4  60880 27440 ?        S    Aug12   0:23 /usr/bin/php-cgi

Is it normal to have php-cgi this large?
Is it possible to estimate php-cgi memory usage based on settings?
Any tips for reducing memory consumption of php-cgi processes?

Searching for known memory leaking bugs didn't yield anything relevant. And I'd be surprised if the default Debian packages/config had such an obvious memory leak. Other users on the same host do not have this problem.

What I've done so far is set PHP_FCGI_MAX_REQUESTS to a low value so that php-cgi processes are recycled quickly. When I use ab to simulate high load, this works very well. Processes die quickly before they grow higher than 10MB. However, under low-to-medium load, all processes grow steadily (because of load balancing) and most of them consume 28MB+ simultaneously, putting my VPS at risk of swapping. Please note than even without any sort of traffic, the processes grow steadily.

I can reduce the number of php-cgi processes, but this feels like a workaround more than a fix. I'd be surprised if php-cgi normally grew like this.

Also, summing the total RSS numbers for php-cgi processes gives:

$ ps -C php-cgi -o rss= | awk '{s+=$1}END{print s/1024}'
195.738

Yet, free -m gives the following output:

             total       used       free     shared    buffers     cached
Mem:           360        351          8          0         33        190
-/+ buffers/cache:        127        232
Swap:          255          0        255

Am I missing something? How come the used memory (without buffers) is lower than the total resident memory of php-cgi processes on the host?

I have the following PHP extensions:

php5-cgi php5-common php5-curl php5-gd php5-mysql php5-xcache

xcache.size is set to 24M. It used to be 32M but reducing it didn't help. xcache.var_size is set to 0. The remaining plugins are using the stock configuration. The xcache admin pages shows that xcache is using less than 1MB.

PHP's memory_limit is set to 32M.

Here is my FastCGI config:

fastcgi.server    = ( ".php" =>
  ((
    "bin-path" => "/usr/bin/php-cgi",
    "socket" => "/tmp/php.socket",
    "max-procs" => 2,
    "idle-timeout" => 20,
    "bin-environment" => (
      "PHP_FCGI_CHILDREN" => "4",
      "PHP_FCGI_MAX_REQUESTS" => "1000"
    ),
    "bin-copy-environment" => (
      "PATH", "SHELL", "USER"
    ),  
    "broken-scriptfilename" => "enable" 
  ))
)

I'm using more-or-less the stock lighttpd.conf that ships with Debian.

Please let me know if there is any other data that I can provide.

Any help is appreciated. I've been trying to troubleshoot this for days. I've run out of ideas.

Try to lower down var_size. If we had value at 64MB, after a few hours it started to swap a lot, and after next few hours it was completely down. Try to keep original settings at 32M, maybe this should help you a lot - we had the same problem at our travel site Xcache is still a lot of buggy software :(

Setting the max requests is the right idea. That is the way to keep your system RAM from filling up when there is a memory leak.

One thing I suggest you try is switching to apache + mod_php. If that works without leaking memory, then that means your problem was CGI related. If it continues to leak with mod_php, then there is probably a memory leak in the code somewhere.

You said you are using Drupal. Do you have any sort of Drupal modules installed? I doubt that a stable version of Drupal has memory leaks in the core, so problems are most likely to occur in modules and other 3rd party add-ons and customizations.

Memory usage of php-cgi processes is growing steadily

Related

Recent Posts