Linux Slow Start: Changing ip route does not have any effect on initial window

I think you are misunderstanding how TCP works.

Each packet sent will always advertise a receiver window (aka. RWIN) and an optional scaling factor, see RFC 1323

The sender is not allowed to send more than the amount of data specified in the RWIN without it getting acknowledged. Depending on the congestion window, the sender may decide to fill up the RWIN or not.

So, there are two bits of information that are public in the TCP packets. The RWIN on the Server and the RWIN on the client. Both of these figures dictate what the maximal size of the congestion window can be on both ends.

The RWIN on the server is interesting when we are trying to optimise performance for say file uploads.

The RWIN on the client is interesting when we are trying to determine download speed.

Neither of these numbers make the congestion window on the other end public.

SO if I have an RWIN of 64k, the congestion window on the server can be ANY number lower than 64k.

The only way to determine what the actual congestion window is to count packets.

If I know:

1. My round trip time (RTT) is ~200ms.
2. I just requested a resource that is 100k.
3. I have an RWIN of 64k.

If I get 2 packets back from the server that are 1452 bytes long within ~200ms, it is likely the congestion window on the server is smaller than 4356, cause if it were bigger 3 packets would be sent. If IW was set to 10, I would see a burst of 10 packets around the 200ms mark.

If you change your IW and want to confirm the change worked, you need to count packets to get an estimate on the congestion window size on the server.

Keep in mind, you probably want to look at the conversation directly after the SYN, SYN-ACK, ACK to ensure you are not looking at the middle of a conversation (where the congestion window could have already grown).

The window size will be whichever is smaller: server init window size or client RWIN. Since 5840 is the default RWIN for Linux 2.6, it seems that your client is the limiting factor here.

Try from a windows box. Windows XP has an RWIN of 64k, newer version 8k.

Source: http://www.cdnplanet.com/blog/tune-tcp-initcwnd-for-optimum-performance/ (The interesting part is below the video)

Edit: Expanding the answer to make it clearer:

• In the TCP handshake, the client sends a SYN packet to the server sending its maximum allowed window size. (As your tcpdump output shows, these are 5840 bytes)
• The server now responds with SYN ACK and the window size it would like to agree on. This window size can only be smaller than what the client proposed, not bigger. No matter how the server is configured, it can never have bigger window sizes than 5840 bytes with that client.
• Client returns ACK and they happily exchange data ever after.

Edit2: The tcpdumps added to the question show the server opening connections to google and itself ACTING AS THE CLIENT.