How to check if ESX hosts have enough RAM

I have noticed that some Linux based virtual machines become gradually slower until I reboot them. At first I suspected hacking and resource abusing but after several vm rebuilds from the scratch and being extra ultra careful security wise, I have ruled the hacking out. Now I'm suspecting that ESX hosts do not have enough RAM and are swapping it for the less busy VMs.

How could I verify whether this is the case or not?

Specs:

ESX#1: ESX 3.5 8x3GHz, 32GB RAM. 7 vms
ESX#2: ESX 3.5 8x3GHz, 32GB RAM. 25 vms
70GB configured guest RAM in total over all the vms.

ESX1 has occasionally an alert that memory is getting low but it's only 24GB of 32GB.

This could be a resource problem on the host, vm or both. Check performance tab in the VI client. Switch to memory. Does it use swap or ballooning? Now switch to disk and select performance. (requires 4.1) What is you latency in ms? Everythin less than 10ms is fine. SATA disks can spike at 300ms or worse. At last try running htop from the Linux VM. Does it swap? How many vcpu's do you use on each host. The host will waste a lot of cycles if you add to many vsmp VM's to the host. SSH into the host and run esxtop. Look at the number rdy. This is the time the VM is waiting for CPU access. if this is more than 10 you could have a problem. By the way you should upgrade to 4.1. 3.5 has huge performance problems.

I've only got v4.1u1 hosts to hand but on the VIClient.exe look on the host summary page under resources - it should say how much memory is being used on that host.

It doesn't seem as though you have a RAM resource problem on your hosts. It may be an issue with your individual VMs.

Looking inside of those specific linux VMs, can you see if you're swapping at all?
How have you configured those VMs in terms of RAM?
Are you using resource pools at all?