IPv6 on Amazon VPC: missing default route in Ubuntu
Now that Amazon has extended IPv6 support for VPC to most of their global regions including eu-west-1, I'm trying to get my instances connected. Unfortunately I can't get routing to work.
I've followed the steps in the migration guide, i.e. I've associated an IPv6 CIDR to our VPC, assigned a part of that to our 'public' subnet, updated the VPC route table to send ::/0 through the igw (internet gateway), made sure that route table is assigned to the public subnet and assigned IPv6 addresses to some new Ubuntu 16.04 instances from the Console.
I then configured Ubuntu to get the assigned address via DHCPv6 as described here, by adding iface eth0 inet6 dhcp to the networking setup and rebooting.
When I reboot the instance it takes a few minutes longer to start up but eventually I can log in and ip a s shows both a IPv4 and a global IPv6 address configured.
However, the v6 network isn't working:
# ping6 www.google.com
connect: Network is unreachable
The route table is indeed missing a default route:
# ip -6 route
2001:DB8:1234:1234:1234:1234:1234:1234 dev eth0 proto kernel metric 256
fe80::/64 dev eth0 proto kernel metric 256 mtu 9001
Manually adding a default v6 route, via ip -6 route add default dev eth0 leads to a routing table that look correct:
# ip -6 route
2001:DB8:1234:1234:1234:1234:1234:1234 dev eth0 proto kernel metric 256
fe80::/64 dev eth0 proto kernel metric 256 mtu 9001
default dev eth0 metric 1024
Unfortunately, this results in a different error:
# ping6 www.google.com
PING www.google.com(dh-in-x6a.1e100.net) 56 data bytes
From dh-in-x6a.1e100.net icmp_seq=1 Destination unreachable: Address unreachable
From dh-in-x6a.1e100.net icmp_seq=2 Destination unreachable: Address unreachable
From dh-in-x6a.1e100.net icmp_seq=3 Destination unreachable: Address unreachable
Isn't the DHCPv6-client supposed to take care of adding a default route? And why can I not reach the outside world even then?
Your routing table does not look correct. This line looks very wrong:
default dev eth0 metric 1024
This line says that the entire internet is connected directly to your eth0 interface without needing to go through any intermediate routers. This will cause your system to send neighor discovery requests onto the LAN for every host it tries to reach. And if that host is not directly connected to your LAN, it will not see the neighbor discovery request.
So you cannot really expect anything to work with that routing table. With some routers it is possible to configure a neighboring router to work around your misconfiguration. But you shouldn't count on it. Instead you should find out what the correct gateway address is and configure that.
Here is an example of what the routing table entry looks like on one particular machine with functional connectivity:
default via fe80::1 dev eth0 metric 1024 advmss 1220
The via fe80::1 part is what is missing from yours. The address you are supposed to use may be different from fe80::1, you would need to ask your provider what gateway address to use if they haven't told you so. The two ways I have mostly seen providers choose to address their gateway is either fe80::1 or the /64 prefix followed by ::1 which in your case would become 2001:DB8:1234:1234::1.
The advmss 1220 part is not absolutely necessary, but I include it because it will work around some MTU issues.
Once you have fixed the routing table entry the next steps to test is to verify that the router shows up in your neighbor cache. And then use traceroute6 or mtr to see how far you can get packets before they get lost.
It turned out I missed a step in the migration guide.
When enabling IPv6 on an existing VPC some things, like route tables and security groups, have to be manually updated if you've made modifications to the default ones.
I had updated our route table (as per http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-migrate-ipv6.html#vpc-migrate-ipv6-routes) and security-groups (as per http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-migrate-ipv6.html#vpc-migrate-ipv6-sg-rules), but had forgotten to update our Network ACL, as mentioned on that same page.
So, I was effectively firewalling all IPv6 traffic. Adding inbound and outbound ALLOW rules for ::/0 fixed my problem for Ubuntu 16.04.
For Ubuntu 14.04 there was actually an error in Amazon's migration guide, which has since been fixed. The advice to add iface eth0 inet6 dhcp to /etc/networking/interfaces.d/eth0.cfg didn't work, leading to a configured IPv6 address, but a missing default route.
Instead, I had to start the dhcp-client when the interface came up, like so: up dhclient -6. I ended up with the following working configuration in the /etc/networking/interfaces.d/eth0.cfg file:
# The primary network interface
auto eth0
iface eth0 inet dhcp
up dhclient -6 -v -pf /run/dhclient6.$IFACE.pid -lf /var/lib/dhcp/dhclient6.$IFACE.leases $IFACE
It appears Amazon has updated their migration guide to say something similar (http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-migrate-ipv6.html#ipv6-dhcpv6-ubuntu-14).