How to find out which process eats up memory?

My MacBook Pro has 8G RAM. Recently memory leak offen happened. Page outs and swap used kept increasing. The last time it used ~8G swap space.

But in Activity Monitor, I can't find any process has a large value in 'Real Mem', 'Private Mem' nor 'Shared Mem' column.

I checked the %MEM column in output of 'ps -ev', all processes occupied less than 1% of memory. The full output is put in this gist: https://gist.github.com/aleung/4760556

What the way to diagnose OSX memory leak issue?

Check with this command (will show more processes):

sudo ps -awxm -o %mem,rss,comm | sort -nr | head

if won't help, check with fs_usage to display system calls and page faults related to filesystem:

sudo fs_usage -f filesys,diskio

especially page in and out by appending the following to above command:

| egrep -i "page_|pgin|pgout"

^{Add extra | grep -v kernel to ignore kernel process or | grep -v 0.00 to show these with higher time spent.}

If your swap is big, you can force disk cache to be flushed and emptied by: sudo purge.

Based on my experience, OS X have some issues with managing the memory after higher uptime (when it used all of free memory), so sometimes only full restart may help.

Check also sudo iotop or sudo vm_stat 1 commands which may help.

See also: How to investigate high kernel task memory usage?