Can excessive latency affect throughput? [closed]

In short: yes, too much latency can lower your total bandwidth, and the effect is more pronounced on serial-type connections - ie: connection which can not transfer any more data until an ACK for previous transmission is received.

The latency/bandwidth correlation become a problem only when the two values are much apart, for example having a very "wide", but high latency, link (or vice-versa - a very low latency but extremely "narrow" link).

Let me do an example using something as simple as a ping of 32 bytes on a WAN connection with about 50 ms latency (100 ms RTT). With such a connection, you can send a maximum of 10x 32b pings, for a grand total of 320b/s only in each direction! It is not a surprise that ping floods explicitly do not wait for the other party confirmation, or the flood would be non-existent.

If you can't lower latency, how can you work around the problem? Mainly using two ways:

• sending larger packets (eg: using 1400 bytes sized pings);
• sending more packets before waiting for confirmation.

TCP uses both systems: after a streaming connection is established (eg: a download is started), it both increases packet size up to a maximum value (the MSS - maximum segment size) and the number of packet sent without waiting for any confirmation/ack - aka TCP window scaling. Much work was spent on creating network stack capable of dynamically adjust the TCP window scaling, so this rarely needs to be manually adjusted nowadays. However, sometime a bad network driver and/or a strange corner case can prevent correct scaling, leading to reduced bandwidth.

Protocols which does not natively implement a window scaling (ie: IP and UDP) are more sensible to the bandwidth problem, and generally need assistence from the higher level application (where possible) or specific system tuning (for example, increasing maximum packet size, know as MTU).

Clearly, WAN connections are, due to their intrinsics higher latency, more prone to the problem. However, even very fast local connections can be affected.

Cases in point:

• on a 10 Gb/s ethernet network, with ping RTT in the range of 0.05 ms, I initially achieved only about 30-40% of the total available bandwidth. Increasing the interface MTU to 9000 bytes (up from the standard 1500 bytes) completely solved the problem;
• on a Gigabit ethernet connection, an Athereos interface driver interacted with the Linux network stack in a manner that prevented the TCP window to "open" to its maximum value, reducing performance. Manually setting an TCP window solved the problem.

Too long for a comment but not quite a complete answer, but latency can effect throughput (bandwidth is a fixed property) :

Your TCP network stack needs to keep a copy of all transmitted packets in memory until it receives confirmation from the recipient that the packet was received, so packets can be retransmitted in case of packet loss.

The minimal size of that buffer is a simple multiplication:

<throughput> * <roundtrip time> = <retransmit buffer size> 

i.e. To support the maximum throughput for a gigabit NIC with a round trip time (=latency) of 0.2 milliseconds in your LAN the needed retransmit buffer would be:

  (1Gigabit/s = 10^9 bit/s) * 0.0002 s = 200000 bits = 25 Kb 

A fairly modest amount. A more realistic WAN latency of 2 milliseconds and you would need 250 Kb.

Unless your network driver dynamically adjusts the amount of memory allocated to your TCP retransmit buffer size, that buffer effectively becomes a limiter on the throughput your system can sustain, regardless of the available bandwidth in the network:

<max throughput>  = <retransmit buffer size> / <roundtrip time>

And any increase in latency reduces the effective throughput of your system.

Now imagine that rather than for a low latency LAN/WAN that same 250 Kb buffer is set for a high latency , high bandwidth connection such as for instance a satellite uplink with a typical 500 ms latency and your throughput gets limited to:

250 Kb / 0,5 s = 2 000 000 / 0,5 = 4 Mbit/s 

rather than the 1 Gbit bandwidth that a satellite internet connection can actually support (according to Wikipedia)

Bandwidth = Capacity

Latency = Delay

Imagine that I have a 500 gallon pool and that I have a 1 gallon bucket for carrying water and a 5 gallon bucket for carrying water. It takes me 1 minute to carry either bucket from my water supply to the pool. If I use only the 1 gallon bucket to fill the pool then I'll need to make 500 trips, totaling 500 minutes of time spent carrying water from the water supply to the pool. If I use only the 5 gallon bucket to fill the pool then I need only make 100 trips, totaling 100 minutes of time spent carrying water from the water supply to the pool.

The time it takes to walk to the pool (latency) doesn't affect how much water I can carry with each bucket (bandwidth) and conversely, the size of the bucket (bandwidth) doesn't change the time it takes to walk to the pool (latency).

The end result is that the pool will fill up 5 times faster when using the 5 gallon bucket than when using the 1 gallon bucket because the 5 gallon bucket is bigger, not because it's faster.