Is Fargate a good fit for independent containers that have low resource utilization?
I'm new to docker and ECS, so I might not use the right terms. Please let me know if I need to clarify.
My Scenario: I have a number of independent containers. Each container represents a web site. Each container should run all the time, but might see long periods of inactivity (e.g. zero CPU usage). For illustration, let's assume I have 10 containers and each container requires up to 200MB RAM and up to 1vCPU. Let's further assume that I only need 2vCPU total to handle the combined load from all 10 containers (since they don't see high load all at the same time).
Fargate Option 1: I create a different task for each container: (2GB, 1vCPU) x 10 (2GB is minimum RAM for 1vCPU).
Fargate Option 2: I create a single task with all containers: (2GB, 2vCPU).
EC2 Option: I create a task per container, all mapped to a single EC2 instance.
If I understand correctly, Fargate Option 2 is much cheaper than Fargate Option 1 because I know I only need a maximum of 2vCPU. But Option 2 is a lot less flexible since it is tasks that are stopped/started/scaled and I want to treat my containers as independent of each other (e.g. stop/start/scale independently).
Furthermore, if I understand correctly, the EC2 Option is the only way I can get both the flexibility of having a task per container as well as paying for the resources I actually need.
So: It seems that for independent containers with low resource utilization, Fargate is not a good fit at this time.
Is my understanding correct?
Solution 1:
That's correct, Fargate is more expensive than EC2 for the same vCPU/RAM amount.
For example:
The smallest Fargate container with 0.25 vCPU and 0.5 GB RAM costs $0.019/hr, that's ca $14/month per container.
If you need 1 vCPU then the minimum amount of RAM is 2GB (see Fargate Supported Configurations) and suddenly the price is ca $55/month per container. In your case times 10 tasks makes it $550/month.
On the other hand if you're confident that you can squeeze all that to a single t3.small (2 vCPU, 2 GB RAM) it will cost you $0.0208/hr which is ca $15/month. Even if you happen to need 2 or 3 t3.small instances to support your load it's still far cheaper than Fargate.
If you lump up all your containers into a single Fargate task (as suggested in your Option 2) it's still more expensive than using EC2 ECS, plus the complications with having multiple independent containers in one task. It's not worth the trouble.
So to wrap up - if you want to run your containers 24x7 and they are not fully utilised all the time you'll be much better off running them on an EC2-based ECS cluster.
With Fargate you pay premium for the flexibility.
If your containers only run briefly to complete a task and then exit, or if they scale up and down based on demand it will be much easier for you to run them in Fargate - you won't need to scale up and down the underlying EC2 cluster to support the load.
In many cases it works out better to run on Fargate even if it's more expensive per vCPU/RAM. We spin up batches of hundreds of containers at a time a couple times per day for some processing and each container runs for only about 10 minutes. If we had to scale up the EC2/ECS cluster before each run, wait for it to settle, deal with the failures, then run our batch job and then scale down again the overhead would be quite high and our batch processing would take much longer.
Here Fargate works great for us. I wouldn't use it for an always-on service though.
Hope that helps :)