Getting files from an s3 bucket using IAM role credentials

I am trying to retrieve some files from a private s3 bucket to a filesystem location elastic beanstalk ec2 instance, but with no success.

I've created a bucket named dev-config containing a file named local.properties.

I've created a IAM policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::dev-config",
                "arn:aws:s3:::dev-config/*"
            ]
        }
    ]
}

And associated that policy to a IAM role, that in turn is associated with the EC2 instance. I have confirmed that I can fetch files from the s3 bucket using the aws-cli without providing any additional credentials. i.e. aws s3 ls s3://dev-config/local.properties

To my project I've added the following file:

.ebextensions/01_files.config

"/usr/share/tomcat7/lib/local.properties" :
    mode: "000777"
    owner: ec2-user
    group: ec2-user
    source: http://s3.amazonaws.com/dev-config/local.properties

I've also tried a few variations of the source url

    source: http://dev-config.s3.amazonaws.com/dev-config/local.properties
    source: http://dev-config.s3.amazonaws.com/local.properties
    source: s3://dev-config/local.properties

And I've also tried adding an authentication attribute with no success (there seem to be no docs on possible values for authentication). authentication: S3Access

None of the approaches have worked so far.

In some cases I get access denied messages in the logs:

    <?xml version="1.0" encoding="UTF-8"?>
      <Error><Code>AccessDenied</Code><Message>Access Denied</Message>
      <RequestId>blahblah</RequestId>
      <HostId>blahblah</HostId>
    </Error>

In other cases I have had error messages in the local.properties file itself PermanentRedirect The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. dev-config dev-config.s3.amazonaws.com
blahlblah blahlblah

Has managed to get this working?

After taking a look at this answer to Using environment properties with files in elastic beanstalk config files I added the following section to the .ebextensions/01_files.config

Resources:
  AWSEBAutoScalingGroup:
    Metadata:
      AWS::CloudFormation::Authentication:
        S3Access:
          type: S3
          roleName: aws-elasticbeanstalk-ec2-role
          buckets: dev-config

and updated the s3 url to include the bucket name in the host, so the final file looked like this:

"/usr/share/tomcat7/lib/local.properties" :
    mode: "000777"
    owner: ec2-user
    group: ec2-user
    source: https://dev-config.s3.amazonaws.com/local.properties

Resources:
  AWSEBAutoScalingGroup:
    Metadata:
      AWS::CloudFormation::Authentication:
        S3Access:
          type: S3
          roleName: aws-elasticbeanstalk-ec2-role
          buckets: dev-config

This enabled the elastic beanstalk ec2 instance to use the IAM role associated with it to access the s3 bucket containing the files.

PS: For this configuration to work, make sure that you've granted access to the S3 bucket in question to the aws-elasticbeanstalk-ec2-role principal. You can get the ARN from IAM console.

Try with this IAM. It works for me.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::dev-config",
                "arn:aws:s3:::dev-config/*"
            ]
        }
    ]
}

If you need to have read/write/delete permissions you need something like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": [
                 "arn:aws:s3:::dev-config",
                 "arn:aws:s3:::dev-config/*"
            ]
        }
    ]
}

Regards.