Impossibly low RAM
I remember the times when I got my macbook pro with 4GB of RAM: I could launch Firefox, VirtualBox and Firefox inside a VirtualBox and it worked reasonably well.
Two years later, when I start the system and have no programs launched, 3.99 out of 4GB of RAM is used; and launching any tiny program, making ⌘ Cmd+Tab or a new tab in a browser is a huge pain for Mac OS X: I have to wait for dozens of seconds for any little action to complete.
However, if I sum up the "Memory" column in Activity Monitor, it would barely come up to 700 MB; Swap is low, File Cache does not exceed 300 MB.
So I don't really understand what could have happened with my Mac and what can I do to take it back to the times when I could use Firefox and Skype together without any slowing down.
Screenshot with all processes as suggessted in comments:
Solution 1:
The "Memory Used" number doesn't mean what most people think it does. Modern operating systems pre-fetch a lot of things from the disk to make things seem faster when you actually request them. For example, in your screenshot, 1.47 GB of the 4 GB "used" is the file cache, which OS X will delete if an app needs more memory. A better metric to look at is App Memory (1.57 GB) compared to your total memory (4 GB).
In the screenshots, it should not be slow because you have barely used any swap. Once the OS starts using swap (more rare with Mavericks), it will start feeling slow. Some people think there are bugs introduced in Lion in OS X's dynamic pager (manages the swapfiles) that cause the system to thrash before it needs to. (I assume) Apple engineers use fast computers with SSDs, which hides this problem.
The solution is to buy a crapton of RAM so the OS doesn't have to hit the swap or downgrade to the earliest OS supported by your computer.