How valuable is TCP Window Scaling?

On a busy† Debian Lenny server of mine I am considering disabling TCP Window Scaling. Why?

• I'd like to enable syn cookies, which disables TCP Window Scaling. This server is locally firewalled and the protection against syn flood attacks could be a good thing, right?
• The kernel logs have many "TCP: Treason uncloaked!" messages. It does not appear to be an attack, having occurred regularly for ages, but it still concerns me. From what I understand, this message is a result of a disagrement between client and server about the size of the TCP window and is generally not a big deal.

So I ask myself "Does the box really need TCP Window Scaling?" Before I even try to experiment and benchmark, it seems prudent to query the boffins of ServerFault.

Some relevant details:

• many (10-30%) of requests are for 5-50MB files
• the large files are sent at a regulated bit rate (~2Mbps)
• clients are on the Internet and 90% within 250km

How valuable is TCP Window Scaling?

• Is the CPU dramatically impacted? If so, how much?
• Is networking performance degraded? Latency does not bother me, but throughput below a minimum threshold would.
• Anything else I might be missing?

† = 3Gbps worth of LACP'd NICs + hundreds of millions of HTTP requests + tens of terabytes of traffic per month

Solution 1:

"Does the box really need TCP Window Scaling?"

Well, you didn't really say what the box does, so it's kind of hard to say really. But in general, TCP Window Scaling is critical for good end user performance on modern WAN/Internet connections. It's less needed on LAN.

Is networking performance degraded? Latency does not bother me, but throughput would.

It depends on your users' network connections. But if some of your users are

• physically far away from you (i.e. have high latency), and
• have fast network connections (i.e. good DSL, fiber, etc)

... then without Window Scaling, these users will only be able to use a small fraction of their connection speed when downloading from you. Here is a good tutorial on bandwith-delay product and window sizes -- complete with nice animated cartoons.

"Enabling High Performance Data Transfers" is a classical document about TCP window scaling. It correctly mentions that if you're using a recent Linux kernel in the 2.6 series, then you normally don't need to tweak the TCP settings, because Linux now has agressive auto-tuning of these. It fails to mention that Windows 2008+ also auto-tunes TCP settings well, using Compound TCP.

Update:

the large files are sent at a regulated bit rate (~2Mbps) clients are on the Internet and 90% within 250km

With this updated info, it gets trickier. Since you are capping the maximum speed, maybe TCP windows are not a limitation in your case. Have a look at what kind of connections your end users are using, and do the math. You don't need to run benchmarks, you can calculate the required window size, and compare it to your servers actual default transmit window size.

Solution 2:

In many cases, TCP autothrottling means that a given transfer is limited to:

BW = windowsize / RTT

since old TCP standards limited the maximum window size to 64K, you could easily get a "Long Fat Network" (LFNs, or 'elephan') if that figure was lower than the physical bandwith. Window scaling to the rescue!

To check if you need it, do some packet capturing and analyze the RTT (don't rely too much on ping times, WireShark has a tool to do that on a sample of real traffic). The average throughput a user would see if you turn it off would be 64KB/RTT (in bytes/sec).

Solution 3:

Is networking performance degraded?

Yes - but only for transferring files larger than RTTxBW. But when you start hauling around big files on a long fat network the performance degrades very quickly.

So whether it affects you will depend on what you're serving (you didn't say).