GCP VPC Peering active, connectivity tests ok, but any request (ssh, icmp...) times out

I have 2 GCP projects with 2 different VPCs:

VPC1 has the default subnet VPC2 has a default subnet and a custom subnet I then have 2 instances, one in each project:

instance1 has one network interface to VPC1 default subnet (internal IP 10.128.0.2)

instance2 has 2 network interfaces to:

a) VPC2 default: default 10.128.0.1 (not a problem since default is not peered)

b) VPC2 custom: custom 10.230.0.3 (the one peered)

Subnets are existing in the right regions and IP Ranges are not overlapping. VPC Peering on the VPC1default to VPC2custom is active and routes have been exported/imported.

Both instances are in regions covered by the regional subnets declared in the VPC peering.

However, when I try to ping using the internal IP from one instance to another, it times out. From 10.128.0.2, ping 10.230.0.3

Firewall rules are updated to allow all ports from both VPC IP Ranges. I popped up instance3 in VPC2 to check if it could be firewall or DHCP because of second network interface. Instance3 is configured the same way as instance2, ie 2 subnets, default and custom. I manage to make both instance communicate without issues... but still can't communicate with any of these 2 from the other peered VPC

I tried running a Network intelligence connectivity test, but it seems that packets are going through. However ssh, ping, or any service is timing out....

Solution 1:

I have reproduced the environment you have detailed and encountered the same issue when setting the peered network as a secondary interface. But it works when the peered network is set as a sole interface or as a primary interface.

As per the official GCP documentation in order to connect to secondary interfaces internal IP from VPC peered networks you will have to configure routes inside your VM, as DHCP defaults traffic to the primary interface.