Fatal error: Out of memory, but I do have plenty of memory (PHP)

Since my question is getting longer and longer, I decide to re-write the whole question to make it better and shorter.

I run my website on dedicated server with 8GB memory. I am fully aware that I need to raise the memory limit on php.ini setting. I have set it from 128M to 256M and to -1. Still the problem is persistence.

Fatal error: Out of memory (allocated 786432) (tried to allocate 24576 bytes) in D:\www\football\views\main.php on line 81

The out of memory does not make sense because it said only 786432 bytes is allocated and it needed 24576 bytes more.

786432 bytes is only 768 kilobytes and is fairly small.

Hints

• The error occurs on a very random line. It does not always error on line number 81.
• At peak time, Apache only takes around 500mb of memory. I still have 6GB to spare.
• There is no infinite loop.
• The script takes 1,042,424 bytes. Getting this number from echo memory_get_peak_usage();
• The resultset from MySQL is small (at most 12 of rows, purely text, no blob data)
• (Important) If I restart Apache once every two days, the error is gone. It usually happens when Apache is running more than 2 days.
• I have included the profiling the script and you can get it here.
• This dedicated server is purely used to run only one website. This website is a high traffic website with average of 1,000 visitors every minute. At peak time, there will be 1,700 to 2,000 visitors accessing at the same time.

Server Spec

OS: Windows 2008 R2 64-Bit
CPU: Intel Core i5 - 4 cores
RAM: 8 GB
Apache 2.2
PHP 5.3.1
Storage: 2 x 1 TB hard drives
Bandwidth: 10 TB per month

Solution

I have finally tuned up and fixed the problem and I would like to share it here what I have done to improve:

1. favicon.ico was missing which mess up with my route engine. Although my route engine is very small, but by including favicon.ico, it helps reduce memory usage by not running my route engine. Most of part of my website has it and I forgot to put it for this new section.

2. Limit MaxRequestPerChild helps. In my other dedicated server, I have my MaxRequestPerChild limited. For this server, I set it to 0. I always thought that each script is isolated. Lets say if my script takes 800kb to run. Upon its completion, Apache or PHP should free 800kb memory. It seem like it doesn't work this way. Limited MaxRequestPerChild does help to prevent memory leak by creating new process after limited MaxRequestPerChild and the old process is dying. This is my new setting.

   ThreadsPerChild      1500
   MaxRequestsPerChild  10000 
   

3. ob_flush(); does reduce slightly more memory. It does not help much but every bit of optimization helps.

4. I have used xdebug which I have never used before as suggested by people who attempt to answer this question. I have to say it is great tool and I have optimized a few stuffs to make it run slightly faster.
5. I have disable a few unnecessary Apache module. I am trying to disable it one by one and leave it a few days test to ensure it works perfectly before I disable another one. I do have all unnecessary PHP extension disable now.
6. Most of my script in this server used traditional way (no template, no database layer, pure PHP, HTML, and legacy mysql_* function). To be honest, it runs very fast and used extremely small memory. However, maintenance the script is not very easy as the website is getting longer. I have tried to convert some parts of the website into proper framework (my own tiny framework). The reason that I used my own framework because it is tiny (3kb for the whole framework and include only what I need).
7. Switching to IIS7.5 solving this problem completely.

I ran accross the same kind of problem with the server dying when trying to use the swap. This is because mod_php does not free memory ever. So Apache processes keep growing either reaching apache or PHP's memory limit or, if there's no limit, crashing the server.

Restarting apache makes it to spawn new fresh slim processes but as they run PHP scripts over time, they grow until problems arise.

The solution is to make apache to kill processes after a certain number of queries served so it will create new ones ( There are some questions related to that) reducing the MaxRequestsPerChild configuration option to, let's say 100 (Defaults to 1000).

Of course this may reduce server performances as it takes ressources to kill and spawn new processes but at least it keeps the site working. You might be tempted to raise the number of running processes to keep performances high, be sure PHP (or apache) memory limit x max number of processes do not get over your server's physical ram.

Here's my experience, hope it helps.

For starters, memory_get_peak_usage() will not be helpful here. It will only return the amount of memory which was allocated, and that is the same number which caused the error.

memory_get_usage will return the active amount of memory which is being allocated when it is called.

ini_set('memory_limit', '256M'); will set the maximum allowance of PHP's footprint on your systems Memory. If you are getting OOM at 768K, upping it will not fix the problem.

There is no indication as to what version of PHP you are using, but I would suggest an upgrade immediately. There are several bugs where Zend's Memory Manager fails to deallocate memory, which would lead you exactly to the same problem.

Are both your local server and your production server running the same version of OS, the same long bit and the same version of PHP? The answer will be no.

If it is unrelated to the windows malloc() issue, being it is a sub domain and probably within a VirtualHost, and allocating only 768k, it almost sounds like an OS issue.

Run tasklist from the command prompt when you access your script. Do you see an additional Apache thread, or Memory usage across the processes spike?

One last idea is, run flush() and/or ob_flush(); after each loop for the table row/column. This should clear your buffer and save you some memory in the event this is where the issue is occurring.

I would start by upgrading PHP to 5.4+ as it's up to 50% faster for some applications. They fixed a large number of memory leaks. Please see becnhamrks: http://news.php.net/php.internals/57760

Please note that the error is Out of memory and is not Allowed memory size [..] exhausted.

So the memory leak is elsewhere on the system. It's possible that mysql server use a lot of system memory after this heavy query, leaving apache/php without it physical and swap.

This should explain the error always on the same line (and/or in the same script).

Install xdebug and enable profiler trigger. Generate a profiler file, then post the cachegrind file if you still would not be able to tell the source of the problem.

EDIT: profiler file of the page where the memory leak happens of course!