What tools could help me figure-out what is wrong with my internet connection?

I've been having some problems with my internet connection, whereby sometimes sites take an age to load yet some (such-as Google) are generally available all the time. The pages also work eventually, as opposed to the connection dropping and the page loads failing.

I've tried using various tools to analyse the problem, such as Firebug and Tamperdata, but they've not been able to tell me anything beyond that some requests have taken a while (up to 3 minutes timed) to resolve. I'm thus thinking that the problem is at a lower level than they can tell me about.

The problem is most noticeable when I open a bunch of tabs to different sites, some of the content loads but some pages (and some sections of other pages) remain in a loading state for ages. I thought at first it could be an intermittent issue with my ISP's DNS, as I'd generally have a cached copy of Google's IP but might not for other sites, but I switched to Google's public DNS and still experienced the same problems.

So, any ideas where the problem could be, or what tool might help me pin this down further? I've contacted my ISP and they've been generally helpful, they sent me a new (better) router in the hopes that it would solve the problem, but unfortunately I'm still occasionally getting this issue.

I suppose it's possible your ISP has a Path MTU Discovery black hole somewhere and doesn't realize it. Try setting your MTU down to 1400 or even 1000 and see if the problem goes away. If it does, then try to find the highest setting that doesn't show the problem.

You could also leave a ping running against some host deep within your ISP (like a web or mail server, not just your local router), and see what kinds of latencies and drop percentages you get.

Also, some DNS resolvers (clients) use all DNS server addresses in a round-robin manner. See what DNS server addresses your hosts and your router are using, and use a DNS testing tool like host, dig, or nslookup, or any number of GUI tools or web apps, to make sure all of the DNS servers you're using a really up and reachable and responding quickly.

Whenever I feel my internet is playing up I always run Auslogics Internet Optimizer firstly.

Just launch the program, choose your internet connection speed, click Analyze and Internet Optimizer will recommend improvements to your system settings. After applying these with another click, reboot, then try a few internet apps and see if you spot any performance gains.