How to scale up one container using Amazon EC2 Container Service
Solution 1:
Do I need to create separate task definitions?
Yes
Do I need to create separate services?
Not necessarily. You can simply run tasks on their own without the "service". But "service" allows association with Load Balancer, Application autoscaling as well as zero downtime deployments.
The only way to "Docker link" your containers is to define them in one task definition like you are currently doing. This way ECS will place all containers on the same instance. Splitting into different tasks means no linking as the containers might be started on different instances.
So if you decide to split them, then each container will have to connect to other containers via "service" urls.
My advise is to
- Create ALB/ELB
- Split all containers into individual tasks.
- Create "services" for all tasks
- Associate each service container with ALB/ELB
- Update each service config to use DNS:PORT of the ALB/ELB used by each service
- Stop using rabitMQ and migrate to SQS.
This way you can scale each "service" individually.
If you decide to stay with rabbitMQ you will have to use ELB for rabbitMQ container and manually associate container port used by rabbitMQ with the ELB.
ALBs will automagically discover container ports used by your services.
See this for more details on ALB and ECS:
https://aws.amazon.com/blogs/compute/microservice-delivery-with-amazon-ecs-and-application-load-balancers/