Reserve RAM for cache and buffer
Solution 1:
If you want to reserve memory for cache and buffer: echo 10 > /proc/sys/vm/vfs_cache_pressure when 100 is the default value. Then you can limit the max ram used by each app: echo 8192 > /proc/sys/vm/max_map_count. I recommend a swapiness=30 and high /proc/sys/vm/dirty_writeback_centisecs and /proc/sys/vm/dirty_expire_centisecs values (both: 1250). It can also help to tweak the file system:
echo 8192 > /sys/block/mmcblk0/queue/max_sectors_kb
echo 8192 > /sys/block/mmcblk1/queue/max_sectors_kb
echo 0 > /sys/block/mmcblk0/queue/iosched/slice_idle
echo 0 > /sys/block/mmcblk1/queue/iosched/slice_idle
echo 160 > /sys/block/mmcblk0/queue/iosched/quantum
echo 160 > /sys/block/mmcblk1/queue/iosched/quantum
echo 800 > /sys/block/mmcblk0/queue/iosched/fifo_expire_sync
echo 800 > /sys/block/mmcblk1/queue/iosched/fifo_expire_sync
echo 180 > /sys/block/mmcblk0/queue/iosched/fifo_expire_async
echo 180 > /sys/block/mmcblk1/queue/iosched/fifo_expire_async
echo 1 > /sys/block/mmcblk0/queue/iosched/back_seek_penalty
echo 1 > /sys/block/mmcblk1/queue/iosched/back_seek_penalty
These values are from my mobile and my laptop and it works really stable and makes things pretty fast. My mobile is memory limited but this tweaks helps a lot. My mobile has cgroups enabled too and I would recommend the 4k wonderpatch because this adds custom cgroups to each app a user is running from the terminal (not desktop) but I don't know how to limit memory with cgroups.
Solution 2:
You can't directly reserve memory for buffers and cache. The kernel will use memory that isn't used for anything else for this, sometimes swapping out very old unused pages to make more room, so the way to resve memory for caches/buffers is to limit the amount of RAM applications (including your virtual machines) are using.
One common problem is allocating more memopry to VMs than is needed, so that the OS in the VM has room for disk caches. You can remove the need to do this by turning on caching support for your virtual disks (how to do this depends upon your virtualisation solution, which you don't state) thereby letting teh host decide how much memory globally is used for caching. That way you can reduce the amount of memory allocated to each VM with less I/O performance degredation than otherwise. A word of warning though: by doing this you do open yourself to extra danger of filesystem corruption on the virtual disks if your host machine has an unclean shutdown (such as caused by a power cut).
Solution 3:
If you have limited memory, don't try to to dictate memory-usage to the kernel. The kernel is probably already doing a better job than you can do with cgroups or compcache.
The only real solution, if memory increase is not an option, is to limit your usage of the RAM. Increasing swap space may "solve" the problem but cause lengthy swapping between processes. Reducing RAM usage by VM can improve the situation.
See the article Reduce your Linux memory footprint. Although from 2007, it can still help.
You could also compile your own kernel, turning off every service and option that you don't need. See for example this article: Kernel Size Tuning Guide.
You might instead move to a small-memory version of Linux, as in this article : 8 of the best tiny Linux distros.
But I repeat that there is no miracle solution that can let you use lots of memory on a memory-starved computer, at least not without lots of slow swapping. You will have to optimize your usage patterns. Optimizing the kernel can help a bit, but has its limits.