Is it possible to populate an S3 bucket through a CloudFormation template?

Solution 1:

I'm afraid it's not possible to upload to a bucket created from the same template. However if the bucket is pre-existing, say an account-wide deployment bucket, or one of those cf-templates-... buckets, you've got some options.

• Upload the auxiliary files using Ansible playbook and in the same playbook create/update the CloudFormation stack. That's a very popular method for CI/CD pipelines. In your Ansible playbook you'll have something like this:

  tasks:
  - name: "Upload files to s3://{{deployment_bucket}}/lambda/"
    s3_sync:
      bucket: "{{deployment_bucket}}"
      file_root: lambda/
      prefix: lambda/
      permission: private
  
  - name: "Create CloudFormation Stack"
    cloudformation:
      stack_name: "some-stack-name"
      state: present
      template: template.yml
      template_parameters:
        DeploymentS3Path: "s3://{{deployment_bucket}}/lambda/"
  

• Let aws cloudformation package zip up and upload the files to S3 and then aws cloudformation deploy create and execute a CloudFormation change set. Again quite a popular method that works well with CI/CD.

  In your CloudFormation template you can then refer to local files like this:

  MyLambda:
    Type: AWS::Lambda::Function
    Properties:
      Code: lambda/
  

  And when aws cloudformation package is run it will output a modified template with the code path expanded:

  MyLambda:
    Properties:
      Code:
        S3Bucket: cf-templates-1a2b3c4d5e6f-ap-southeast-2
        S3Key: e24e45d4f5f2ab3c5d437659fa2246a7
  

  You then aws cloudformation deploy this expanded template. Or use some other method for deployment - the template is ready to use.

  The beauty of this package / deploy method is that it handles the upload of local files to S3 for you.

In different projects we use both methods in various CI/CD pipelines and both work great.

Hope that helps :)

Solution 2:

Another alternative is to use cfpack tool. It allows you to define what artifacts need to be uploaded to a S3 bucket. Take a look at the Artifacts section for more details, it explains how to deal with Graphql schema and resolvers.

In addition to this, cfpack allows you to split your big CloudFormation template into smaller files. This is especially useful when you have a huge CloudFormation template. So, give it a try.

Solution 3:

Some AWS tools, like λ#, are able to create redeployable CloudFormation templates with assets. Take a look at this example: https://github.com/LambdaSharp/StaticWebsite-Sample