How to solve "java.io.IOException: error=12, Cannot allocate memory" calling Runtime#exec()?

This is the solution but you have to set:

echo 1 > /proc/sys/vm/overcommit_memory

What's the memory profile of your machine ? e.g. if you run top, how much free memory do you have ?

I suspect UnixProcess performs a fork() and it's simply not getting enough memory from the OS (if memory serves, it'll fork() to duplicate the process and then exec() to run the ls in the new memory process, and it's not getting as far as that)

EDIT: Re. your overcommit solution, it permits overcommitting of system memory, possibly allowing processes to allocate (but not use) more memory than is actually available. So I guess that the fork() duplicates the Java process memory as discussed in the comments below. Of course you don't use the memory since the 'ls' replaces the duplicate Java process.

This is solved in Java version 1.6.0_23 and upwards.

See more details at http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7034935

Runtime.getRuntime().exec allocates the process with the same amount of memory as the main. If you had you heap set to 1GB and try to exec then it will allocate another 1GB for that process to run.

I came across these links:

http://mail.openjdk.java.net/pipermail/core-libs-dev/2009-May/001689.html

http://www.nabble.com/Review-request-for-5049299-td23667680.html

Seems to be a bug. Usage of a spawn() trick instead of the plain fork()/exec() is advised.