Cable modem speed test
Are the popular speed tests (cable, dsl, etc) that are available accurate? if so, how? Also, it seems that they only measure download speed of one file and not responsiveness, latency, robustness, etc. Does anyone know how it should be measured? any links to good articles on that topic?
I'd say most speed tests are flawed if you want to measure web browsing performance (which is what a sizable number of people will be doing). Almost all tests use Flash or Java, totally bypassing your browser HTTP stack. These tests also normally use a single large file, totally bypassing the benefits of persistent connections and HTTP/1.1 pipelining only some browsers support natively. If you do use Flash or Java to download single large files, then these tests may be more relevant.
Also, I've seen great variability in results across browser for some flash tests, which has to do with the way browsers handle flash animation, thus skewing results with no relation to network performance.
To make these tests better, the methodology needs to reflect:
- That tests are tailored to what is being tested! Don't use flash unless you are testing flash network performance on single files. A browser test should use multiple resources (remembering CSS and JS can block successive downloads in some browsers) across multiple tabs to reproduce real circumstances. If you are testing torrent speed, you need huge numbers of small bandwidth HTTP/UDP connections.
- Measuring variability. Most tests do something once, and a single sample is not sufficient to know something reliably. One needs to average across repeat tests, and critically, variance needs to be estimated. Is 180 really better than 150 when the variability is +- 100??? {answer: no}
Not sure of a perfect resource for this, but Steve Souder writes extensively on web performance at least:
http://stevesouders.com/
The tests others have suggested all fall foul of these technical problems. They will give you a rough number that may help generally, but you should be aware of the limitations. Particularly fancy flash animations seem utterly useless, and as I've said actually can skew results significantly.
A good connection is made up of key parts: Through put and responsiveness. There are more but I am sticking to the main 2 (in my opinion). Through put is how much data can be pushed or pulled through the connection (download/upload). Responsiveness is how fast does my request make it the destination and back. Both affect each other a bit.
Through put is a pretty easy test, just go to one of the sites listed here and it will test your upload/download speeds. Just remember that the connection you pay for from you ISP is an UPPER limit (most of the time). For example I pay for a 6mb down/ 1mb up connection. 6mb down/1mb up is the uppper limit of my connection. Some ISPs will have a lower limit. An example would be if an ISP says your connection will be between 3mb down and 6mb down. The other key part is the remote host that you are pulling from or pushing data too. If you connection is bigger, then you are limited by the remote host connection and effectively DoS the remote host.
Responsiveness is a bit hard to test. I am not 100% sure on the correct way to test it. commands like ping and tracert in window are a good start. They will give you a rough idea of how long a round trip to and from a remote host takes. This also takes into account the time the remote host takes to process your request and send it back so take this with a grain of salt. Responsiveness is not a big deal when surfing the web or chatting on line. It comes into play with applications like online games. A slow response time will cause you to "lag" and make the game less fluid to the point of unplayable. Online games rely on fast response over higher through put.
Other things to consider in evaluating a connection are Uptime, what ports are blocked, ISP policies regarding us of Bittorrent, IRC, SMTP, or other services.