AWS elastic beanstalk: Errno 404 downloading file from S3 on deployment
I'm following the docs on fetching certificates from s3 when a new instance is deployed to elastic beanstalk. The instructions are fairly straightforward: create a config file under app-root/.ebextensions
that reads:
Resources:
AWSEBAutoScalingGroup:
Metadata:
AWS::CloudFormation::Authentication:
S3Auth:
type: "s3"
buckets: ["elasticbeanstalk-us-west-2-123456789012"]
roleName:
"Fn::GetOptionSetting":
Namespace: "aws:autoscaling:launchconfiguration"
OptionName: "IamInstanceProfile"
DefaultValue: "aws-elasticbeanstalk-ec2-role"
files:
# Private key
/etc/pki/tls/certs/server.key:
mode: "000400"
owner: root
group: root
authentication: "S3Auth"
source: https://s3-us-west-2.amazonaws.com/elasticbeanstalk-us-west-2-123456789012/server.key
where the links point to the location on your certificate / key / whatever.
However, when I deploy my application bundle (zip), the build fails with [Errno 404] HTTP Error 404...
suggesting that my files cannot be found at the specified location contrary to the fact that the files are at the precise specified location since I can aws s3 cp s3://<pasted-link-to-file> .
them from my command line (same region of course).
Also, on the surface, it doesn't seem to be a permissions problem because I'd expect something like "Access Denied" if it was; the bucket policy grants the following actions to aws-elasticbeanstalk-ec2-role
:
"Action": [
"s3:ListBucket",
"s3:ListBucketVersions",
"s3:GetObject",
"s3:GetObjectVersion"
]
I also attached full s3 access policy to aws-elasticbeanstalk-ec2-role
in IAM.
But the fact that I can access the files from cli and aws-elasticbeanstalk-ec2-role
cannot find them, suggests that a) something could be wrong with my setup: files on s3, s3 permissions, config in .ebextensions
or b) aws docs on this matter are completely out of whack.
- This is a bit of a broad question but can anyone suggest what else could go wrong with this sort of setup?
Solved it by explicitly specifying:
roleName: "aws-elasticbeanstalk-ec2-role"
instead of:
roleName:
"Fn::GetOptionSetting":
Namespace: "aws:autoscaling:launchconfiguration"
OptionName: "IamInstanceProfile"
DefaultValue: "aws-elasticbeanstalk-ec2-role"
in .ebextensions/my_conf.config
.
This could be because the config file key and the S3 policy refer to "roleName" and not "IamInstanceProfile".
After doing some troubleshooting, I realized the issue with :
"Fn::GetOptionSetting":
Namespace: "aws:autoscaling:launchconfiguration"
OptionName: "IamInstanceProfile"
DefaultValue: "aws-elasticbeanstalk-ec2-role"
This option returns the Instance Profile. However, to fetch credentials, we need the AWS Role associated with the instance profile.
Following command will return 404:
curl -vvv http://169.254.169.254/latest/meta-data/iam/security-credentials/{Instance-Profile}
However, following command works and returns temporary credentials:
curl -vvv http://169.254.169.254/latest/meta-data/iam/security-credentials/{Instance-ROLE}