Routing via Cisco ASA is changing TCP sequence/ACK numbers
Our network has a dedicated VPN appliance that sits inside the office network. We have a Cisco ASA with a static route that routes the VPN subnets to the VPN appliance. So a typical request from the client to the remote site (192.168.161.28 -> 192.168.101.28) goes:
Client ASA Local VPN Remote VPN Remote Server 192.168.161.28 -> 192.168.161.17 -> 192.168.161.10 -> 192.168.101.1 -> 192.168.101.28
With this route, the firewall on the remote VPN endpoint of 192.168.101.1 rejects the 3-way TCP handshake:
Status: A TCP packet was rejected because it has an invalid sequence number or an invalid acknowledgement number
However, if I bypass the ASA (with a static route on the client machines directly):
Client Local VPN Remote VPN Remote Server 192.168.161.28 -> 192.168.161.10 -> 192.168.101.1 -> 192.168.101.28
The TCP streams are correctly handshaked and everything is smooth sailing.
What could it be? Is there some inspection rules on the ASA that could be breaking this? I suspect it's because the return route of the traffic is different to the sending route (i.e. the packets will be going directly from the VPN Endpoint to the client, rather than going via the ASA as they are on the same LAN).
Disabling ACK randomisation on the ASA resolves this issue (this scenario matches Example B - Multiple Internet Paths):
access-list tcp_bypass extended permit tcp 192.168.161.0 255.255.255.0 any
class-map tcp_bypass
match access-list tcp_bypass
policy-map tcp_bypass_policy
class tcp_bypass
set connection advanced-options tcp-state-bypass
set connection timeout idle 0:10:00
service-policy tcp_bypass_policy interface inside
This solution sort of sucks - make sure you read the implications before doing it.
Most likely the the firewall(192.168.101.1) misses the SYN-ACK because it is routed in a different route back and since its a stateful firewall it rejects it.
- A sends SYN to B through Z while being inspected by C.
- B sends SYN-ACK directly to A (Because B is on the same subnet and has the default route to A)
- A sends ACK to B through Z while being inspected by C and dropped because it never saw SYN-ACK.
This is commonly referred to as asymmetric routing.