Linux fails to interpret ACK, keeps resending SYN+ACK
The following is a wireshark dump of the problem occurring, IP-adresses replaced with 'client' and 'server':
4414.229553 client -> server TCP 62464 > http [SYN] Seq=0 Win=65535 Len=0 MSS=1452 WS=3 TSV=116730231 TSER=0
4414.229633 server -> client TCP http > 62464 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=2406364374 TSER=116730231 WS=6
4414.263330 client -> server TCP 62464 > http [ACK] Seq=1 Ack=1 Win=524280 Len=0 TSV=116730231 TSER=2406364374
4418.812859 server -> client TCP http > 62464 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=2406365520 TSER=116730231 WS=6
4418.892176 client -> server TCP [TCP Dup ACK 778#1] 62464 > http [ACK] Seq=1 Ack=1 Win=524280 Len=0 TSV=116730278 TSER=2406365520
4424.812864 server -> client TCP http > 62464 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=2406367020 TSER=116730278 WS=6
4424.891240 client -> server TCP [TCP Dup ACK 778#2] 62464 > http [ACK] Seq=1 Ack=1 Win=524280 Len=0 TSV=116730337 TSER=2406367020
So the normal SYN, SYN+ACK, ACK sequence seems to occur, except that the server doesn't seem to interpret the ACK. Instead it keeps resending the SYN+ACK, which the client dutifully keeps responding to with a duplicate of the previous ACK. I don't see how this could happen.
I noticed the problem because iptables connection tracking considers these connections established and keeps them in memory for a full 120-hour timeout. I have some firewall rules to prevent large numbers of concurrent connections, the limits of which people were hitting, without actually having that many connections active. The netstat command doesn't show these phantom connections.
Other info:
The server is a standard debian lenny system with a stock kernel:
Linux tb 2.6.26-2-686 #1 SMP Wed Aug 19 06:06:52 UTC 2009 i686 GNU/Linux
running:
Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g
I don't have all the info on the client (I can't reproduce locally), but it's a Mac running the Chrome browser.
I don't have any firewall rules messing with ACK packets. Basically I only filter SYN packets, all other TCP packets are allowed through. So I don't actually use the connection tracking for firewalling, other than counting concurrent connections and some graphing of TCP-Established packets compared to other packet types.
Edit: my iptables-rules pertaining to TCP port 80:
iptables -P INPUT ACCEPT
iptables -A INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above 50 -j LOGDROP-CONN
iptables -A INPUT -p tcp --syn -m multiport --dports 80,443 -j ACCEPT
iptables -A INPUT -p tcp --syn -j REJECT --reject-with tcp-reset
iptables -A LOGDROP-CONN -m limit --limit 1/minute --limit-burst 1 -j LOG --log-prefix "ConConn "
iptables -A LOGDROP-CONN -j DROP
Edit 2: another dump, this time using tcpdump -vv:
16:05:52.999525 IP (tos 0x0, ttl 55, id 46466, offset 0, flags [DF], proto TCP (6), length 64) client.50538 > server.www: S, cksum 0x4429 (correct), 38417001:38417001(0) win 65535 <mss 1452,nop,wscale 3,nop,nop,timestamp 117224762 0,sackOK,eol>
16:05:52.999580 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) server.www > client.50538: S, cksum 0xa2ab (correct), 3062713115:3062713115(0) ack 38417002 win 5792 <mss 1460,sackOK,timestamp 2418739698 117224762,nop,wscale 6>
16:05:53.321788 IP (tos 0x0, ttl 55, id 24299, offset 0, flags [DF], proto TCP (6), length 52) client.50538 > server.www: ., cksum 0xe813 (correct), 1:1(0) ack 1 win 65535 <nop,nop,timestamp 117224765 2418739698>
16:05:56.252697 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) server.www > client.50538: S, cksum 0x9f7a (correct), 3062713115:3062713115(0) ack 38417002 win 5792 <mss 1460,sackOK,timestamp 2418740512 117224765,nop,wscale 6>
16:05:56.277250 IP (tos 0x0, ttl 55, id 15533, offset 0, flags [DF], proto TCP (6), length 52) client.50538 > server.www: ., cksum 0xe4c4 (correct), 1:1(0) ack 1 win 65535 <nop,nop,timestamp 117224798 2418740512>
Solution 1:
I'd brute force it. First I'd try if it works with iptables stopped. If it does, then it's something in the iptables.
Then I'd add rules one by one and watch which one causes connection failure. Then I'd play with that rule until it did what I wanted without totally disrupting the traffic.
If it doesn't work with iptables stopped, then it would start to be really strange.