"Too many files open" on Mac OSX after running apache in PHP with XDebug for some time

I am running Mac OS X 10.9.4, including the builtin apache2 webserver with PHP 5.5.14 from brew (packages: php55, php55-intl, php55-pdo-pgsql, php55-xdebug).

When running this setup it works quite well. However, after some time, I will run on 403 errors for every request. I have looked up the apache error log, and found something like the following:

[Fri Jul 25 05:28:18 2014] [error] [client 127.0.0.1] PHP Warning:  require_once(/Users/daniel/Development/massiveart/sulu-complete/app/bootstrap.php.cache): failed to open stream: Too many open files in /Users/daniel/Development/massiveart/sulu-complete/web/website.php on line 10, referer: http://sulu.lo/de
[Fri Jul 25 05:28:18 2014] [error] [client 127.0.0.1] PHP Stack trace:, referer: http://sulu.lo/de
[Fri Jul 25 05:28:18 2014] [error] [client 127.0.0.1] PHP   1. {main}() /Users/daniel/Development/massiveart/sulu-complete/web/website.php:0, referer: http://sulu.lo/de
[Fri Jul 25 05:28:18 2014] [error] [client 127.0.0.1] PHP Fatal error:  require_once(): Failed opening required '/Users/daniel/Development/massiveart/sulu-complete/web/../app/bootstrap.php.cache' (include_path='.:/usr/local/Cellar/php55/5.5.14/lib/php') in /Users/daniel/Development/massiveart/sulu-complete/web/website.php on line 10, referer: http://sulu.lo/de
[Fri Jul 25 05:28:18 2014] [error] [client 127.0.0.1] PHP Stack trace:, referer: http://sulu.lo/de
[Fri Jul 25 05:28:18 2014] [error] [client 127.0.0.1] PHP   1. {main}() /Users/daniel/Development/massiveart/sulu-complete/web/website.php:0, referer: http://sulu.lo/de
[Fri Jul 25 05:28:40 2014] [crit] [client 127.0.0.1] (24)Too many open files: /Users/daniel/Development/massiveart/sulu-complete/web/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable, referer: http://sulu.lo/de
[Fri Jul 25 05:28:41 2014] [crit] [client 127.0.0.1] (24)Too many open files: /Users/daniel/Development/massiveart/sulu-complete/web/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable, referer: http://sulu.lo/de
[Fri Jul 25 05:28:41 2014] [crit] [client 127.0.0.1] (24)Too many open files: /Users/daniel/Development/massiveart/sulu-complete/web/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable, referer: http://sulu.lo/de
[Fri Jul 25 05:28:41 2014] [crit] [client 127.0.0.1] (24)Too many open files: /Users/daniel/Development/massiveart/sulu-complete/web/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable, referer: http://sulu.lo/de
[Fri Jul 25 05:28:45 2014] [crit] [client 127.0.0.1] (24)Too many open files: /Users/daniel/Development/massiveart/sulu-complete/web/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable, referer: http://sulu.lo/de
[Fri Jul 25 05:28:45 2014] [crit] [client 127.0.0.1] (24)Too many open files: /Users/daniel/Development/massiveart/sulu-complete/web/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable, referer: http://sulu.lo/de

It looks to me like the file cannot be read anymore, and it returns the 403 somehow. I already found out about certain limits, but launchctl returns I have an unlimited hard limit on open files:

 ~ $ launchctl limit
    cpu         unlimited      unlimited
    filesize    unlimited      unlimited
    data        unlimited      unlimited
    stack       8388608        67104768
    core        0              unlimited
    rss         unlimited      unlimited
    memlock     unlimited      unlimited
    maxproc     709            1064
    maxfiles    256            unlimited

I have also already tried to set the maxfiles to 4096 with the command launchctl limit maxfiles 4096 16384, but the issue still returns after some time. Any idea what else I can check?

UPDATE: When running the lsof -c httpd command as suggested by Gordon Davisson, I can see that there are loads of entries like the following:

httpd   1361 _www   15u    IPv4 0xb306b48659f63853       0t0     TCP localhost:50603->localhost:cslistener (CLOSED)

I can say that the application I use is using websockets, and is also using a fallback when websockets are not available or the counterpart is not running on the server. What confuses me is the (CLOSED)-part, why is it still listed?

UPDATE: After some time I looked up the cslistener port, which actually is 9000, which again is which port xdebug is listening for remote debugging. So I guess I have got some wrong configuration there, or it is a bug in xdebug (I am using XDebug 2.2.5, installed by brew)

Are you using PHPStorm with XDEBUG on Mac?

I have the same problem. I found an open bug filed with XDEBUG here:

http://bugs.xdebug.org/view.php?id=1070

Update

This bug has now been fixed:

I just merged a patch by Sean Dubois, which should fix this \o/! The patch is going to be in 2.3.4 and 2.4.0.

I believe this is the commit: https://github.com/xdebug/xdebug/commit/6efc6588efc277d648a78b69c11c721992c996f9

Make sure you are using an updated version with this patch.

I'm pretty sure you have something running in apache (probably the PHP module, but it's hard to be sure) that's leaking file descriptors. That is, it's opening files and then just leaving them open indefinitely. If this is the case, increasing the open file limit just makes it take longer to hit the limit. What you really need to do is track down what's opening all the files and leaving them open.

You can probably get some idea what's going on with the lsof("LiSt Open Files") command:

sudo lsof -c httpd

Run it when apache hasn't been running for long to see what's normal, then again when it hits the limit. Look in the second output for lots of additional files that aren't there in the first listing. Note that this will be complicated somewhat by the fact that it'll list the files opened by all httpd processes, and (depending on your apache settings and server load) there may be a large number of them; the important thing is the number of files opened by a single process, not the total across all server processes. You can also use sudo lsof -p someprocessID to list just a single server process at a time.

Hopefully seeing what the extra open files are will give you a good idea what's opening them and leaving them open.

Adding following line to xdebug.ini also solved the problem for me

xdebug.remote_autostart = 0