Solution 1:

womble's answer gives you great list of parameters to monitor.

i suggest you go for munin as simple tool for gathering stats and plotting charts with the trends. if you need something more advanced - maybe zabbix or cacti, if you need rock-solid alerting system - nagios.

few things to add to measured parameters:

  • baseline of mysql queries [ munin's standard mysql plugin will do it for you ] - how many queries of each type, how many slow queries [ enable in my.cnf logging of those eg longer than 1-2 sec - log will come handy when you'll be searching for biggest offenders ]
  • apache's stats - number of idle worker processes, number of page requested/sec [ again munin's plugin will do it for you ].
  • total number of process

most probably you'll find munin with it's plugins in your distro's standard repository. it's light and if you don't need more features - i suggest you start with it.

and yeah - already start thinking about shifting static content to some high performance server like nginx/mathopd/lighthttpd [ if you have just 1 public ip look here ].

Solution 2:

I use Munin to monitor my Apache server on running on Ubuntu. Getting Munin running was very easy, and getting the Apache plugin going required just a little more work.