Unable to remove limit on memory usage for PHP script
The Situation
I am having an issue with a PHP script getting the following error message:
Fatal error: Out of memory (allocated 359923712) (tried to allocate 72 bytes) in /path/to/piwik/core/DataTable.php on line 969
The script I'm running is: /path/to/piwik/misc/cron/archive.sh
I am assuming the numbers are Bytes, which means that total is approximately 360MB.
For all intents and purposes, I have increased the memory limits on the server well above 360MB, yet this is the number (give or take a byte) it consistently errors out at.
Please note: This question is not about fixing a memory leak in the script, nor about why the script itself is using so much memory. The script is part of the Piwik archiving process, so I cannot just fix any memory leaks, etc. For more info on this script and why I am increasing the memory limit, see "How to setup auto archiving"
The question
Given that the script is attempting to use over 360MB of memory, which I cannot change, why does it not seem possible for me to increase the amount of memory available to php on my server?
Update 23rd June: See "What I've tried" > "Increasing the per process memory limits of Linux" below for background... If I set ulimit -v 1024000, then check it with ulimit -v I get the correct value of '1024000'. If I run the script again, it will progress much further, but will eventually error out with the same memory limit (~360MB) having been reached. If I immediately check ulimit -v, it has reset to the original value of '524288'. This appears as though it may be the root cause of the issue.
What I've tried
Increasing PHP's memory_limit
Given the php.ini file:
php -i | grep php.ini
Configuration File (php.ini) Path => /usr/local/lib
Loaded Configuration File => /usr/local/lib/php.ini
I have edited that file, so the memory_limit directive reads;
memory_limit = -1
Restart Apache, and check the new value has stuck;
$ php -i | grep memory_limit
memory_limit => -1 => -1
Run the script, and get the same error.
I've also tried 1G, 768M, etc, all to the same result (ie; no change).
Update 22nd June: Based on Vangel's help, I have attempted to set post_max_size to 20M in combination with setting memory_limit. Again, this has no effect.
Update 23rd June: Based on olefebvre's help, I can confirm that the user running the script has read/write permission to the php.ini file containing the settings of memory_limit.
Removing the memory limit on child processes of Apache
I have found and edited the httpd.conf file to make sure there is no RLimitMEM directive.
I then used WHM's Apache Configuration > Memory Usage Restrictions to generate a restriction, which it claimed was at 1000M (and confirmed by checking httpd.conf).
Both of these resulted in no change to the script erroring at 360MB.
Increasing the per process memory limits of Linux
The current limits set on the system:
$ ulimit -m
524288
$ ulimit -v
524288
I have attempted to set both of these to unlimited:
$ ulimit -m unlimited
$ ulimit -v unlimited
$ ulimit -m
unlimited
$ ulimit -v
unlimited
Once again, this has resulted in absolutely no improvement in my problem.
Update 23rd June: I have run into a related issue here. If I set ulimit -v 1024000, then check it with ulimit -v I get the correct value of '1024000'. If I run the script again, it will progress much further, but will eventually error out with the same memory limit having been reached. If I immediately check ulimit -v, it has reset to the original value of '524288'. This appears as though it may be the root cause of the issue.
My setup
$ cat /etc/redhat-release
CentOS release 5.5 (Final)
$ uname -a
Linux example.com 2.6.18-164.15.1.el5 #1 SMP Wed Mar 17 11:30:06 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
$ php -i | grep "PHP Version"
PHP Version => 5.2.9
$ httpd -V
Server version: Apache/2.0.63
Server built: Feb 2 2011 01:25:12
Cpanel::Easy::Apache v3.2.0 rev5291
Server's Module Magic Number: 20020903:13
Server loaded: APR 0.9.17, APR-UTIL 0.9.15
Compiled using: APR 0.9.17, APR-UTIL 0.9.15
Architecture: 64-bit
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D HTTPD_ROOT="/usr/local/apache"
-D SUEXEC_BIN="/usr/local/apache/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="logs/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
Output of $ php -i: http://pastebin.com/EiRut6Nm
Background:
I just grabbed the current version of Piwik -- In version 1.5, line 969 appears as:
public function addRowsFromSerializedArray( $stringSerialized )
{
$serialized = unserialize($stringSerialized);
if($serialized === false)
{
throw new Exception("The unserialization has failed!");
}
$this->addRowsFromArray($serialized);
}
and is specifically $serialized = unserialize($stringSerialized);. The call to unserialize can be incredibly memory intensive. There is an excellent post here.
As you've noted, this clearly isn't a bug in your script and is a valid Out of Memory.
Suggestion: In the config file, as noted above in my comment:
/.../piwik/config/global.ini.php
I think you may need to increase one of these limits:
# during archiving, Piwik will limit the number of results recorded, for performance reasons
# maximum number of rows for any of the Referers tables (keywords, search engines, campaigns, etc.)
# this limit will also be applied to the Custom Variables names and values reports
datatable_archiving_maximum_rows_referers = 1000
# maximum number of rows for any of the Referers subtable (search engines by keyword, keyword by campaign, etc.)
datatable_archiving_maximum_rows_subtable_referers = 50
# maximum number of rows for any of the Actions tables (pages, downloads, outlinks)
datatable_archiving_maximum_rows_actions = 500
# maximum number of rows for pages in categories (sub pages, when clicking on the + for a page category)
# note: should not exceed the display limit in Piwik_Actions_Controller::ACTIONS_REPORT_ROWS_DISPLAY
# because each subdirectory doesn't have paging at the bottom, so all data should be displayed if possible.
datatable_archiving_maximum_rows_subtable_actions = 100
# maximum number of rows for other tables (Providers, User settings configurations)
datatable_archiving_maximum_rows_standard = 500
where I changed the semi-colons to # signs just to make sf's autocolor readable.
You might also try adding:
CMD_TO_CHECK_SETTINGS="$PHP_BIN -i > /tmp/piwik-php-env.out"
$CMD_TO_CHECK_SETTINGS
to archive.sh to determine if there are other settings afoot which override the php.ini file.
M. Tibbits answer alludes to allowing Piwik to use more resources during the archiving process.
There is a limitation on how large an SQL command can be executed, which defaults to just 1MB. For more info; Packet too large
To increase the size limitation in MySQL, edit /etc/my.cnf and set max_allowed_packet=32M
Ensure the PHP memory limit is set to be high enough per forked process by editing /usr/local/lib/php.ini and set memory_limit = 512M
Finally, ensure all processes have at least a 1G hard limit before the system shuts them off by executing ulimit -v 1048576
Update
ulimit -v 1048576 will only raise the soft limit. If the hard limit is not high enough, the system will automatically reset the soft limit to that of the hard limit.
To set the hard limit, add the -H switch:
ulimit -vH 1048576
followed by increasing the soft limit up to this value:
ulimit -vS 1048576