EC2 - How do you tell the ELB when the machine is "ready"

I have a autoscaling setup with monitoring and all is well. The process for code deployment is now well in hand and basically does this.

  1. Create a new launch_config which defines the git commit to pull
  2. Update the autoscale group with the new launch config.
  3. Scale up the min-size of the servers to ~2x what we have.
  4. Wait for them all to come online
  5. Scale down and the old ones get knocked off.

The problem that I am seeing is that while we are "waiting for them to come online" they are reporting "healthy" and "in service" before they are "ready" for service as they are updating the packages and pulling the relevant source code. This time varies greatly from 2 minutes to > 15 minutes per EC2 instance.

What I'm looking for is a way do you set the "in service/healtny" flag manually, or a better strategy for dealing with this lag.

I will note that when the instance fires off Nginx is started automatically. Is that what determines whether the instance is healthy? There is so little documentation (that I can find) on this process of boot to ready - any help would be appreciated.

This is behind an ELB if that helps. Health checks are enabled "/". I wonder if it's as simple as that..

Thanks


Solution 1:

The Elastic Load Balancer health check determines whether ELB sends traffic to any particular instance.

What you want to do is very simple:

Don't start the web server on your instances until you've finished bringing them up and configuring them.

Solution 2:

You can easily setup a URI that will be active on your EC2 instance and set the ELB health check to call it. Until that health check URI returns a 200 status code the instance will not be active in the ELB to receive traffic.

In most cases the default is just to query the server for the URI / but you can change that to be anything you wish. In my case / will fail if the webapp setup for it is not ready for traffic. You could also point it to a CGI or PHP page that would do a system check and return with a good 200 status code when everything is ready.