How do i get Strongswan / IPTables to route data back to my road warrior client correctly?
I have a simple VPN. I have a client on 10.185.28.241 who gets a virtual IP of 10.42.42.0/24 from the VPN which is located at 10.112.18.105 and is providing access to machines in the 10.112.0.0/16 CIDR. I'm running StrongSwan U5.3.2/K3.10.0-229.el7.x86_64 on Centos 7. My network layout looks something like this:
[Windows 2012 Client ]--------[VPN ]---------[Hidden network]
10.185.28.241 / 10.42.42.1 10.112.18.105 10.112.0.0/16
Yes, everything about this network is internal. The Windows Client is actually in AWS while the 10.112 network is in my colo, but all over a pre-existing site to site VPN. I'm setting this up because I eventually want to route external traffic back through the VPN so it will appear to come through our colo where we're whitelisted by vendors. But I'm not there yet.
I can connect to the VPN from Windows and it's connecting over the very fancy IKEv2. When I ping an IP on the hidden network, I see the packets coming over from tcpdump but the 10.112.18.105 machine refuses to return the response.
So on the Windows machine I'm executing:
C:\temp> ping 10.112.8.102
Pinging 10.112.8.102 with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Ping statistics for 10.112.8.102:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
C:\temp> ping 10.112.8.102
And on the VPN server I see:
[root@localhost strongswan]# tcpdump -n -i any host 10.112.8.102 or host 10.42.42.1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
17:38:42.616871 IP 10.112.18.105 > 10.112.8.102: ICMP echo request, id 1, seq 161, length 40
17:38:42.617324 IP 10.112.8.102 > 10.112.18.105: ICMP echo reply, id 1, seq 161, length 40
17:38:47.280369 IP 10.42.42.1 > 10.112.8.102: ICMP echo request, id 1, seq 162, length 40
17:38:47.280457 IP 10.112.18.105 > 10.112.8.102: ICMP echo request, id 1, seq 162, length 40
17:38:47.280872 IP 10.112.8.102 > 10.112.18.105: ICMP echo reply, id 1, seq 162, length 40
17:38:52.280116 IP 10.42.42.1 > 10.112.8.102: ICMP echo request, id 1, seq 163, length 40
17:38:52.280213 IP 10.112.18.105 > 10.112.8.102: ICMP echo request, id 1, seq 163, length 40
17:38:52.280672 IP 10.112.8.102 > 10.112.18.105: ICMP echo reply, id 1, seq 163, length 40
17:38:57.279917 IP 10.42.42.1 > 10.112.8.102: ICMP echo request, id 1, seq 164, length 40
17:38:57.280018 IP 10.112.18.105 > 10.112.8.102: ICMP echo request, id 1, seq 164, length 40
17:38:57.280488 IP 10.112.8.102 > 10.112.18.105: ICMP echo reply, id 1, seq 164, length 40
^C
11 packets captured
12 packets received by filter
0 packets dropped by kernel
[root@localhost strongswan]#
The reply is coming back to the VPN server ("10.112.8.102 > 10.112.18.105: ICMP echo reply") but the VPN server isn't passing the response back to the client. I've spent a good 10 hours trying to track this down and all I've found are lots of people with a similar issue :-(.
Some background info:
ipsec.conf:
config setup
charondebug="ike 2, knl 2, cfg 2, net 2, esp 2, dmn 2, mgr 2"
conn %default
keyexchange=ikev2
ike=aes128-sha256-ecp256,aes256-sha384-ecp384,aes128-sha256-modp2048,aes128-sha1-modp2048,aes256-sha384-modp4096,aes256-sha256-modp4096,aes256-sha1-modp4096,aes128-sha256-modp1536,aes128-sha1-modp1536,aes256-sha384-modp2048,aes256-sha256-modp2048,aes256-sha1-modp2048,aes128-sha256-modp1024,aes128-sha1-modp1024,aes256-sha384-modp1536,aes256-sha256-modp1536,aes256-sha1-modp1536,aes256-sha384-modp1024,aes256-sha256-modp1024,aes256-sha1-modp1024!
esp=aes128gcm16-ecp256,aes256gcm16-ecp384,aes128-sha256-ecp256,aes256-sha384-ecp384,aes128-sha256-modp2048,aes128-sha1-modp2048,aes256-sha384-modp4096,aes256-sha256-modp4096,aes256-sha1-modp4096,aes128-sha256-modp1536,aes128-sha1-modp1536,aes256-sha384-modp2048,aes256-sha256-modp2048,aes256-sha1-modp2048,aes128-sha256-modp1024,aes128-sha1-modp1024,aes256-sha384-modp1536,aes256-sha256-modp1536,aes256-sha1-modp1536,aes256-sha384-modp1024,aes256-sha256-modp1024,aes256-sha1-modp1024,aes128gcm16,aes256gcm16,aes128-sha256,aes128-sha1,aes256-sha384,aes256-sha256,aes256-sha1!
dpdaction=clear
dpddelay=300s
rekey=no
left=10.112.18.105
leftsubnet=10.112.0.0/16
leftfirewall=yes
leftcert=vpnHostCert.der
right=%any
rightdns=10.112.1.140,10.112.1.141
rightsourceip=10.42.42.0/24
conn IPSec-IKEv2
keyexchange=ikev2
auto=add
conn IPSec-IKEv2-EAP
also="IPSec-IKEv2"
rightauth=eap-mschapv2
rightsendcert=never
eap_identity=%any
conn CiscoIPSec
keyexchange=ikev1
forceencaps=yes
authby=xauthrsasig
xauth=server
auto=add
Strongswan statusall:
[root@localhost strongswan]# strongswan statusall
Status of IKE charon daemon (strongSwan 5.3.2, Linux 3.10.0-229.el7.x86_64, x86_64):
uptime: 26 minutes, since Apr 15 17:31:18 2016
malloc: sbrk 1630208, mmap 0, used 515488, free 1114720
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 1
loaded plugins: charon aes des rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints acert pubkey pkcs1 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp xcbc cmac hmac ctr ccm curl attr kernel-netlink resolve socket-default farp stroke vici updown eap-identity eap-md5 eap-gtc eap-mschapv2 eap-tls eap-ttls eap-peap xauth-generic xauth-eap xauth-pam xauth-noauth dhcp
Virtual IP pools (size/online/offline):
10.42.42.0/24: 254/1/0
Listening IP addresses:
10.112.18.105
192.168.122.1
Connections:
IPSec-IKEv2: 10.112.18.105...%any IKEv2, dpddelay=300s
IPSec-IKEv2: local: [10.112.18.105] uses public key authentication
IPSec-IKEv2: cert: "C=US, O=Aurea, CN=cis-migration-vpn.aes.aurea.com"
IPSec-IKEv2: remote: uses public key authentication
IPSec-IKEv2: child: 10.112.0.0/16 === dynamic TUNNEL, dpdaction=clear
IPSec-IKEv2-EAP: 10.112.18.105...%any IKEv2, dpddelay=300s
IPSec-IKEv2-EAP: local: [10.112.18.105] uses public key authentication
IPSec-IKEv2-EAP: cert: "C=US, O=Aurea, CN=cis-migration-vpn.aes.aurea.com"
IPSec-IKEv2-EAP: remote: uses EAP_MSCHAPV2 authentication with EAP identity '%any'
IPSec-IKEv2-EAP: child: 10.112.0.0/16 === dynamic TUNNEL, dpdaction=clear
CiscoIPSec: 10.112.18.105...%any IKEv1, dpddelay=300s
CiscoIPSec: local: [10.112.18.105] uses public key authentication
CiscoIPSec: cert: "C=US, O=Aurea, CN=cis-migration-vpn.aes.aurea.com"
CiscoIPSec: remote: uses public key authentication
CiscoIPSec: remote: uses XAuth authentication: any
CiscoIPSec: child: 10.112.0.0/16 === dynamic TUNNEL, dpdaction=clear
Security Associations (1 up, 0 connecting):
IPSec-IKEv2-EAP[1]: ESTABLISHED 26 minutes ago, 10.112.18.105[10.112.18.105]...10.185.28.241[10.185.28.241]
IPSec-IKEv2-EAP[1]: Remote EAP identity: ryan
IPSec-IKEv2-EAP[1]: IKEv2 SPIs: 9b0fd7b8f81a8c6c_i 966e28db92550c47_r*, rekeying disabled
IPSec-IKEv2-EAP[1]: IKE proposal: AES_CBC_256/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024
[root@localhost strongswan]#
Network Information
[root@localhost strongswan]# cat /etc/sysctl.conf
# System default settings live in /usr/lib/sysctl.d/00-system.conf.
# To override those settings, enter new settings here, or in an /etc/sysctl.d/<name>.conf file
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
# VPN
net.ipv4.ip_forward = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
[root@localhost strongswan]# ifconfig
ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.112.18.105 netmask 255.255.254.0 broadcast 10.112.19.255
inet6 fe80::250:56ff:fe9a:b6c prefixlen 64 scopeid 0x20<link>
ether 00:50:56:9a:0b:6c txqueuelen 1000 (Ethernet)
RX packets 132983 bytes 33121546 (31.5 MiB)
RX errors 0 dropped 678 overruns 0 frame 0
TX packets 62082 bytes 11920649 (11.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 20 bytes 2100 (2.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 20 bytes 2100 (2.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255
ether 52:54:00:a1:a6:70 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@localhost strongswan]# ip -4 a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 10.112.18.105/23 brd 10.112.19.255 scope global ens192
valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
[root@localhost strongswan]# ip -4 r s t 0
default via 10.112.18.1 dev ens192 proto static metric 100
10.112.18.0/23 dev ens192 proto kernel scope link src 10.112.18.105 metric 100
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
broadcast 10.112.18.0 dev ens192 table local proto kernel scope link src 10.112.18.105
local 10.112.18.105 dev ens192 table local proto kernel scope host src 10.112.18.105
broadcast 10.112.19.255 dev ens192 table local proto kernel scope link src 10.112.18.105
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
broadcast 192.168.122.0 dev virbr0 table local proto kernel scope link src 192.168.122.1
local 192.168.122.1 dev virbr0 table local proto kernel scope host src 192.168.122.1
broadcast 192.168.122.255 dev virbr0 table local proto kernel scope link src 192.168.122.1
[root@localhost strongswan]# ip route show table 220
10.42.42.1 via 10.112.18.1 dev ens192 proto static src 10.112.18.105
[root@localhost strongswan]# ip xfrm policy show
src 10.42.42.1/32 dst 10.112.0.0/16
dir fwd priority 2883 ptype main
tmpl src 10.185.28.241 dst 10.112.18.105
proto esp reqid 1 mode tunnel
src 10.42.42.1/32 dst 10.112.0.0/16
dir in priority 2883 ptype main
tmpl src 10.185.28.241 dst 10.112.18.105
proto esp reqid 1 mode tunnel
src 10.112.0.0/16 dst 10.42.42.1/32
dir out priority 2883 ptype main
tmpl src 10.112.18.105 dst 10.185.28.241
proto esp reqid 1 mode tunnel
src 0.0.0.0/0 dst 0.0.0.0/0
socket in priority 0 ptype main
src 0.0.0.0/0 dst 0.0.0.0/0
socket out priority 0 ptype main
src 0.0.0.0/0 dst 0.0.0.0/0
socket in priority 0 ptype main
src 0.0.0.0/0 dst 0.0.0.0/0
socket out priority 0 ptype main
src ::/0 dst ::/0
socket in priority 0 ptype main
src ::/0 dst ::/0
socket out priority 0 ptype main
src ::/0 dst ::/0
socket in priority 0 ptype main
src ::/0 dst ::/0
socket out priority 0 ptype main
[root@localhost strongswan]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.112.18.1 0.0.0.0 UG 100 0 0 ens192
10.112.18.0 0.0.0.0 255.255.254.0 U 100 0 0 ens192
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
[root@localhost strongswan]#
IPTables
[root@localhost strongswan]# iptables-save
# Generated by iptables-save v1.4.21 on Fri Apr 15 18:03:58 2016
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [152:18472]
-A INPUT -p udp -m udp --dport 1701 -j ACCEPT
-A INPUT -p udp -m udp --dport 4500 -j ACCEPT
-A INPUT -p udp -m udp --dport 500 -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p esp -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Fri Apr 15 18:03:58 2016
# Generated by iptables-save v1.4.21 on Fri Apr 15 18:03:58 2016
*nat
:PREROUTING ACCEPT [866:126303]
:INPUT ACCEPT [9:2520]
:OUTPUT ACCEPT [14:1198]
:POSTROUTING ACCEPT [14:1198]
-A POSTROUTING -s 10.42.42.0/24 -o ens192 -m policy --dir out --pol ipsec -j ACCEPT
-A POSTROUTING -s 10.42.42.0/24 -o ens192 -j MASQUERADE
COMMIT
# Completed on Fri Apr 15 18:03:58 2016
[root@localhost strongswan]#
I originally started out using the new firewall-cmd in Centos 7 using the following commands, but I ran into the exact same problem, hoping that moving back to old iptables would help since I honestly understand a bit better. Here were the commands I ran back then:
firewall-cmd --zone=dmz --permanent --add-rich-rule='rule protocol value="esp" accept' # ESP (the encrypted data packets)
firewall-cmd --zone=dmz --permanent --add-rich-rule='rule protocol value="ah" accept' # AH (authenticated headers)
firewall-cmd --zone=dmz --permanent --add-port=500/udp #IKE (security associations)
firewall-cmd --zone=dmz --permanent --add-port=4500/udp # IKE NAT Traversal (IPsec between natted devices)
firewall-cmd --permanent --add-service="ipsec"
firewall-cmd --zone=dmz --permanent --add-masquerade
firewall-cmd --set-default-zone=dmz
firewall-cmd --reload
firewall-cmd --list-all
Any ideas on what I'm doing wrong? Why does the ping request/response make it all the way to the target and back to the StrongSwan server but not back to the client?
Well, I got this to work by giving my Strongswan machine a public IP. That was the only change needed. So now my network looks like:
[Windows 2012 Client ]------[External Nat]-----[VPN ]---------[Hidden network]
10.185.28.241 / 10.42.42.1 x.x.x.71 10.112.18.105 10.112.0.0/16
That's all that's required. I didn't have to change any of my configs. I just changed the DNS name that was pointing to 10.112.18.105 to x.x.x.71 and left everything else alone.
To the best of my understanding, the AWS network where my Windows client is was somehow eating the returning ESP packets so it never got a response from the server when going through the site to site VPN. I didn't have to make any changes at all on my Strongswan server. Just going over a public IP and NOT going through the site to site VPN was all I needed.