Diagnosing possible intermittent network failures/slowdowns; tool?

Take a look at mtr. It's traceroute and ping consolidated into one tool and continuously monitors the path between two hosts. You get output like that below. It's available as package mtr-tiny for Ubuntu and mtr for CentOS.

                               My traceroute  [v0.75]
somehost.lan (0.0.0.0)                                  Thu Aug 18 20:52:49 2011
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                           Packets               Pings
 Host                                    Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. somehost.lan                                0.0%    56    0.3   0.3   0.2   0.5   0.0
 2. 192.168.1.254                         0.0%    55   82.8  48.1   2.9 100.6  31.5
 3. 94-192-160-1.zone6.bethere.co.uk     76.4%    55  11038 11232 11038 11450 156.1
 4. 10.1.3.245                           25.9%    55   17.6  17.6  16.8  20.6   0.7
 5. ???
 6. ???
 7. linx1-hex.webfusion.com              96.3%    55   23.6  23.8  23.6  24.1   0.3
 8. ???
 9. supanames-22.supanames.co.uk          0.0%    55   31.4  31.4  30.5  38.0   1.0

I would do this with a small shell script like this:

#!/bin/bash
while true; do
  date >> mylogfile.txt
  ping -c 5 sometestnode >> mylogfile.txt
  sleep 5
done

Run the script like this so it keeps running after you log out:

nohup scriptname.sh &

Once you have waited a suitable amount of time, you'll just need to kill the process and look through your log file. Of course this could be tweaked any which way you want.