How to troubleshoot network flows across peered VPC

I'm trying to accomplish this: https://cloud.google.com/vpc/docs/vpc-peering#global and it isn't working. Specifically, i'm trying to access an external server via a VPN on a peered network.

My question is what tools are available for troubleshooting this? Are there any tricks to get insight into what is going on?

I've enabled flow logs, and i can see the SRC logs in the source VPC, but they have no dest information. I can see no logs in the peered network at all.

I'm new to google VPCs and networking generally, so i could be missing something simple. I've checked the routes and firewall rules they look correct on both VPCs.

Solution 1:

VPC flow logs is a great tool to debug network connection issues. However, In your case, I would recommend to go through the following troubleshooting steps:

1-In your scenario you need to make sure that the vpn connection is working and that you are able to reach the external server on-prem from a vm in Network-b using cloud vpn.

2-Make sure the vpc peering is working and that vms in Network-a are able to reach vms in Network-b.

3-Make sure you add Cloud Router custom route advertisements in your VPC network(Network-b) to announce peered network subnets to your on-premises network.

EDIT to address the additional comment

There are many tools and different approaches to troubleshoot networking issue between GCP and On-prem servers, you can use the following troubleshooting steps to help you in the future:

1- running a packet capture using wireshark or tcpdump on the GCP VM and the on-prem server to see the ingress/egress traffic.

2- Check firewall rules on GCP and the on-prem firewall to make sure you are allowing ingress/egress traffic from the VPN gateways on both sides.

3-Check the routing table of the on-prem firewall/router and see if there is a route to the VPC subnet and that the peer VPN gateway(cloud VPN) is the next hop.

4-Check routes on GCP and see if there is a route to the VPC subnet and that the remote peer VPN gateway(on-prem router/firewall) is the next hop. This is created automatically on the GCP side once you create a vpn tunnel.