AWS VPC Endpoint SecurityGroupEgress rule

I am trying to make use of an AWS VPC Endpoint to access an S3 resource with no luck. Could you help?

Details:

I have an EC2 Instance inside a Security Group. This is inside a VPC and subnet with an internet endpoint and a routing table entry to it for 0.0.0.0/0. However, I don't want this specific EC2 instance to be able to talk to the internet but I do want it to be able to talk to an S3 Bucket located at https://s3-eu-west-1.amazonaws.com/XXXXX/YYYYYY.sh.

In response, I have overloaded the default allow Egress rule on the securityGroup so outbound connections are denied by adding an egress rule to port 22. The amazon documentation tells me that local (private) AWS addresses will still be allowed. With this in mind, I added a VPC endpoint into my VPC and updated my routing table with pl-6da54004 (com.amazonaws.eu-west-1.s3) :: vpce-9f47a1f6.

With this done, I cant access my S3 resource from within my EC2 instance. I try and wget https://s3-eu-west-1.amazonaws.com/XXXXX/YYYYYY.sh with no luck.

Do you have any idea what is wrong? My thoughts are:

• my ec2 instance seems to resolve the S3 domain to 54.231.130.244, but this is not the same for every instance. Do i need to add a securityGroupEgress rule or a route for an S3 IP range? if so, what is the range?
• Perhaps I should be accessing my S3 resources via a different domain name that instead resolved to an internal IP address?

Thank you, Toby

First, a bit of background. The DNS resolver for VPC instances is a virtual component that is built in to the infrastructure. It's immune to the outbound security group rules... but the resolution of the hostnames for S3 endpoints doesn't change when you provision an S3 endpoint for your VPC.

What a VPC endpoint for S3 does is a couple of different things. Understanding what those things are is key to understanding whether it will do what you need. tl;dr: it will, in this case.

First, you notice they are configured in the route tables as "prefix lists." A VPC endpoint takes a set of predefined IPv4 network prefixes, and hijacks the routes to those prefixes for every route table that includes the respective prefix list so that your traffic to any of those networks will traverse the VPC endpoint instead of the Internet Gateway and any intermediate NAT instance.

In essence, this opens a new path out from your VPC to the AWS service's IP address ranges... but where those IP addresses take you is not, initially, and he same place as they would take you without the VPC endpoint in place.

The first place you hit looks just like S3 but it isn't identical to the Internet-facing S3, because it knows about your VPC endpoint's policies, so that you can control which buckets and actions are accessible. These do not override the other policies, they augment them.

An endpoint policy does not override or replace IAM user policies or S3 bucket policies. It is a separate policy for controlling access from the endpoint to the specified service. However, all types of policies — IAM user policies, endpoint policies, S3 policies, and Amazon S3 ACL policies (if any) — must grant the necessary permissions for access to Amazon S3 to succeed.

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html#vpc-endpoints-access

Note that if you do not restrict bucket access with an appropriate policy, and instead enable full access, the instances will be able to access any bucket in the S3 region if the bucket's policies allow it, including public buckets.

Now, the tricky part. If your instance's security group doesn't allow access outbound to S3 because the default "allow" rule has been removed, you can allow the instance to access S3 via the VPC endpoint, with a specially-crafted security group rule:

Add a new outbound rule to the security group. For the "type," choose HTTPS. For the destination, choose "Custom IP."

The documentation is not consistent with what I see in the console:

The Destination list displays the prefix list IDs and names for the available AWS services.

http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html#vpc-endpoints-security

Well... no, it doesn't. Not for me, least, not as of this writing.

The solution is to choose "Custom IP" and then, instead of an IP address block or security group ID, type the prefix list id for your VPC endpoint, in the form of pl-xxxxxxxx in the box for the IP address. You can find this in the VPC console, by looking at the destinations in one of the subnets associated with the VPC endpoint.