Is it safe to delete ~/Library/Caches?
It's generally safe, though a little dangerous depending, to do it but often not worth the effort.
The caches in /System/Library/Caches are generally small and useful, the ones in /Library/Caches are less system caches and much more readily cleared.
If you have a look in ~/Library/Caches you will find a bunch of applications have a cache in there, none of them particularly large though dropbox sometimes has a fair sized cache. This folder can run quite large just because so many apps cache something in there.
If the cache /Library/Caches folder is over 3Gb then you have something that is caching quite a lot. On the three machines I just checked none had a /Library/Caches folder over .75 Gb so I'd go right ahead and remove some of it.
Don't worry about age, I'd worry about size.
In the terminal run the following to sort all of the files in that directory by size (ascending):
du -s /Library/Caches/* | sort -h
Of course the best way to clear the caches is to install AppleJack and do it with that in single user mode. Doing it with the System fully up can be a little dangerous. If you do it then I'd reboot immediately afterwards.
Yes, it is safe. That said, don't just delete all the contents of your cache folder without reason.
It can be beneficial to clear out those taking up the significant space in your ~/Library/Caches/ if you need to free some up, but you really shouldn't clear any contents of your /System/Caches unless there is a problem. As would be expected, you will find that the programs you access most often have the largest cache size.
There's a misconception that purging your caches regularly constitutes "maintenance," optimization, etc of your machine, but that's a fallacy; Always remember that caches improve the performance of your machine, and shouldn't be cleaned out regularly or blindly without a specific reason.
Really, the only two reasons to clear out a cache are 1) space considerations (it’s simply growing too large) or 2) You're troubleshooting a problem.