Linux not freeing large disk cache when memory demand goes up
I have discovered the answer to my own question - thanks to womble's help (submit an answer if you like).
lsof -s shows file handles in use, and turns out there were several gigabytes of mmap'd log files taking up the cache.
Implementing a logrotate should resolve the issue completely and allow me to take advantage of more memory.
I will also re-enable swap so we have no problems with the OOM killer in the future. Thanks.
Apparently, postgres' shared_buffers can show up in cached, while not really being easily discardable... See OOM despite available memory (cache)