Why python has limit for count of file handles?

Solution 1:

The number of open files is limited by the operating system. On linux you can type

ulimit -n

to see what the limit is. If you are root, you can type

ulimit -n 2048

now your program will run ok (as root) since you have lifted the limit to 2048 open files

Solution 2:

I see same behavior on Windows when running your code. The limit exists from C runtime. You can use win32file to change the limit value:

import win32file

print win32file._getmaxstdio()

The above shall give you 512, which explains the failure at #509 (+stdin, stderr, stdout as others have already stated)

Execute the following and your code shall run fine:

win32file._setmaxstdio(2048)

Note that 2048 is the hard limit, though (hard limit of the underlying C Stdio). As a result, executing the _setmaxstdio with a value greater than 2048 fails for me.