How can I limit the amount of memory available to a process?
I am developing a program in go; it occasionally ends up allocating very large amounts memory (>>10G on a machine with 8G of physical memory), causing the system to become unresponsive. I want to limit the amount of memory the process can allocate. The usual way I would do this is:
ulimit -m 4000000 ; ./myprogram
...which should kill my program if it tries to use more than 4GB of memory.
On OS X El Capitan this appears to have no effect; even ulimit -m 1 (limiting all programs to just 1kB of memory!) is ineffective.
How can I set an upper bound on the memory available to a particular process?
Solution 1:
There are two approaches to constraining your memory usage: Ex post facto, and preemptive. That is to say, you can try to kill your program after it has gotten too large, or you can program it not to get too large in the first place.
If you insist on the ex post facto approach, you can use the following Bash script. This script first finds the amount of memory (as defined by "resident set size") the process with processid pid is using, filters out all non numerical data using grep, and saves the amount as variable n. The script then checks if n is greater than your specified x. If it is, the process with processid pid is killed.
Please note:
- You must replace
<pid>with the process id of your program. - You must replace
<x>with the rss = "resident set size" (ie real memory size) you don't want the program to exceed.
n=$(ps -<pid> -o rss | grep '[0-9]')
if [ $n -gt <x> ]; then kill -9 <pid>; fi
If you want this to run every y seconds just inclose it in a loop, and tell it to wait for y seconds after every iteration. You could also write a similar command using top. Your starting point would be top -l 1|grep "<pid>"|awk '{print $10}'.
@kenorb's answer helped me with my script
While I believe that answers the question, in the long run I believe it is better programming design to take a preemptive approach using manual memory allocation.
Firstly, are you sure the memory usage is really a problem? The Go documentation states:
The Go memory allocator reserves a large region of virtual memory as an arena for allocations. This virtual memory is local to the specific Go process; the reservation does not deprive other processes of memory.
If you still think you have a problem, then I encourage you to manually manage your memory as is done in the C programming language. Since go is written in C, I suspected there would be ways to get into the C memory management/allocations, and indeed there are. See this github repository which,
allows you to do manual memory management via the standard C allocator for your system. It is a thin wrapper on top of malloc, calloc and free from . See man malloc for details on these functions for your system. This library uses cgo.
The use case is given as:
Why would you want this?
When a program is causing memory pressure or the system is running out of memory it can be helpful to manually control memory allocations and deallocations. Go can help you control allocations but it is not possible to explicitly deallocate unneeded data.
This seems like a better long term solution.
If you want to learn more about C (including memory management), The C Programming language is the standard reference.