Adding CloudFlare to S3 bucket

Solution 1:

Here is a list of steps you will have to take to get Cloudflare to work for your S3 bucket, I will try to elaborate later as necessary, but there are quite a few steps here:

Step 1: Set up your domain-based bucket

Note: This hostname will have to be onboarded into Cloudflare later (in step 3), so choose it based on that. The entire hostname will have to be used to serve the bucket, so don't choose something that is already in use. I will use static.example.com for this.

Go into S3 and create a bucket with the name of this domain, you will want to select the existing bucket with your images as the copy settings from bucket. You will want the bucket to be in the same region as the existing bucket for Step 2:

Step 1.5 (optional): Test the custom domain without Cloudflare

Add a test image image.jpg to the bucket with public viewable permissions, load it into the root.

CNAME the hostname's DNS entry (static.example.com) to the endpoint of your Amazon bucket. This is usually the name of the bucket (in this case, static.example.com) plus the standard S3 URL including the region. For our example, it would be: static.example.com.s3.us-west-2.amazonaws.com, but replace us-west-2 with your actual region.

Now see if the test resource is available at http://static.example.com/image.jpg, make sure to use http because https will not work here. It may take a little while to propagate DNS from the above step.

Step 2: Copy all the resources from the old bucket

You will need to follow this guide to copy all the resources from the old bucket to the new. The guide is in depth enough to follow to copy all your resources over to the new bucket, although if you have a very large bucket you may run into issues here.

Verify that the expected resources are at http://static.example.com/old_image_path.jpg.

Step 3: Sign up and setup Cloudflare

This step is to onboard your domain into Cloudflare. Cloudflare has a set of instructions for this. If you have any existing records, make sure they are there in the onboarding list. One entry that should be there is the static.example.com -> static.example.com.s3.us-west-2.amazonaws.com CNAME that was set up in step 1.

Once this is set up, make sure the DNS entry is in "orange cloud" mode A.K.A. proxied (and cached). Make a test request again from step 2, but look that the image has signs it is coming from Cloudflare servers. This will include a Server: cloudflare header as well a cf-cache-status header that is indicating whether you are retrieving from Cloudflare's cache and saving bandwidth. This may take some time as the previous step needed a DNS change to propagate.

You should be able to use https in requests at this point, but if not, go to the SSL/TLS > Edge Certificates tab of Cloudflare and verify that Universal SSL is enabled.

Step 4: Switch image link references

This is specific to the original question. All image link references will have to switch from the old s3 bucket to the new hostname in Cloudflare, static.example.com.

Step 5: Tweak cache settings

You may want to tweak the Cache settings to save bandwidth. There are two main ways to achieve this. Cloudflare will use the Cache-Control header to determine how to determine how long to cache. You can mass update the S3 cache control header according to these instructions. Additionally Cloudflare gives you the option to override this header with the Caching > Configuration > Browser Cache TTL option. This will modify effectively modify the Cache-Control: max-age value to the larger of the two. A longer TTL will keep the images cached longer in the users' browsers, but will also keep images in Cloudflare's edge cache longer and reduce the load from your S3 bucket.

Disclaimer:

Although cloudflare does offer seemingly free bandwidth, there is a limit to their generosity. Please refer to Section 2.8 of Cloudflare's terms. It seems like the above mentioned website is probably in violation of these terms and if Cloudflare decides your usage is too much it is possible that your site will be removed. At that point you will have to DNS-only mode and go back to paying for S3 or negotiate with cloudflare some terms and payment to continue using Cloudflare's bandwidth.