Keep the original IP after forwarding through iptables

EDIT: I've made a post on Medium about how I did these things: Link

I've been trying to forward trafic from my VPS with public IP to my home server (behind CGNAT) using wireguard VPN. I want to forward traffic for ssh, minecraft server (25565) and minecraft bedrock server (19132). It seems to work correctly with this setup:

Wireguard configuration (VPS):

[Interface]
PrivateKey = (redacted)

PostUp = iptables -t nat -A PREROUTING -p tcp --dport 555 -j DNAT --to-destination 192.168.4.2:22
PostUp = iptables -t nat -A PREROUTING -p tcp --dport 25565 -j DNAT --to-destination 192.168.4.2:25565
PostUp = iptables -t nat -A PREROUTING -p udp --dport 19132 -j DNAT --to-destination 192.168.4.2:19132
PostUp = iptables -t nat -A POSTROUTING -j MASQUERADE


PostDown = iptables -t nat -D PREROUTING -p tcp --dport 555 -j DNAT --to-destination 192.168.4.2:22
PostDown = iptables -t nat -D PREROUTING -p tcp --dport 25565 -j DNAT --to-destination 192.168.4.2:25565
PostDown = iptables -t nat -D PREROUTING -p udp --dport 19132 -j DNAT --to-destination 192.168.4.2:19132
PostDown = iptables -t nat -D POSTROUTING -j MASQUERADE

ListenPort = 51820
Address = 192.168.4.1

[Peer]
PublicKey = (redacted)
AllowedIPs = 192.168.4.2

Wireguard configuration (home server):

[Interface]
PrivateKey = (redacted)
Address = 192.168.4.2


[Peer]
PublicKey = (redacted)
AllowedIPs = 192.168.4.1
Endpoint = (redacted):51820
PersistentKeepalive = 25

When I connect to my server or login to ssh, it says that I'm connecting from the VPN (192.168.4.1), which makes sense, but is there a way how to retain the original IP? It would be useful since I could ip-ban players on the Minecraft server and block IPs on ssh after a few failed attempts.

EDIT Newer and updated configs

VPS config:

[Interface]
PrivateKey = (redacted)

PostUp = iptables -t nat -A PREROUTING -p tcp --dport 555 -j DNAT --to-destination 10.20.4.2:22
PostUp = iptables -t nat -A PREROUTING -p tcp --dport 25565 -j DNAT --to-destination 10.20.4.2:25565
PostUp = iptables -t nat -A PREROUTING -p udp --dport 19132 -j DNAT --to-destination 10.20.4.2:19132
PostUp = iptables -t nat -A POSTROUTING -s 10.20.4.2 -j MASQUERADE

PostDown = iptables -t nat -D PREROUTING -p tcp --dport 555 -j DNAT --to-destination 10.20.4.2:22
PostDown = iptables -t nat -D PREROUTING -p tcp --dport 25565 -j DNAT --to-destination 10.20.4.2:25565
PostDown = iptables -t nat -D PREROUTING -p udp --dport 19132 -j DNAT --to-destination 10.20.4.2:19132
PostDown = iptables -t nat -D POSTROUTING -s 10.20.4.2 -j MASQUERADE

ListenPort = 51820
Address = 10.20.4.1

[Peer]
PublicKey = (redacted)
AllowedIPs = 0.0.0.0/0

And this is the home server's config:

[Interface]
PrivateKey = (redacted)
Address = 10.20.4.2
Table = 1

PostUp = ip -4 rule add pref 500 from 10.20.4.2 lookup 1
PostDown = ip -4 rule del pref 500

[Peer]
PublicKey = (redacted)
AllowedIPs = 0.0.0.0
Endpoint = (redacted):51820
PersistentKeepalive = 25

The IPs:

10.20.4.1 - VPS

10.20.4.2 - Home server

Solution 1:

Source addresses are changed by the MASQUERADE rule. This is the whole point of MASQUERADE, but it shouldn't have been applied so broadly.

You'll need to limit the rule so that it only applies to packets going out from the homeserver to Internet but not the other way around. (You can limit this by interface using -i wg0 and/or -o eth0 and/or by source address -s 192.168.4.0/24.)

Now the home server's WireGuard configuration must also accept packets with any source IP address from the VPN server, by changing its [Peer] AllowedIPs to 0.0.0.0/0.

When wg-quick is used to set up WireGuard, it'll automatically generate routes based on AllowedIPs, so the previous change will also cause all your home server's network traffic to go through the tunnel. To avoid this, you could also set [Interface] Table to off on the client.

However, the home server still needs to send some traffic through the VPN – if wg-quick was told to not add any routes, then the home server will receive packets through VPN but still respond to them directly through the Internet connection. This can be solved using policy routing, first by setting the WireGuard tunnel's [Interface] Table to an actual table number (1 is fine), and then by running the command

PostUp = ip -4 rule add pref 500 from 192.168.4.2 lookup 1
PostDown = ip -4 rule del pref 500

so that only reply packets coming from 192.168.4.2 will use the routes set up by wg-quick in table 1, but other unrelated packets won't.