How to automatically update nginx upstream server list when aws ec2 hostname changes or increases?

I want to setup autoscaling in AWS. I don't want to use Elastic Load Balancer.

Autoscalling in Amazon creates EC2 instances seamlessly during demand spikes to maintain performance, and decreases automatically during demand lulls to minimize costs.

Since this EC2 instances are created automatically, their host names are unknown to NGINX.

I know and already have upstream setup in nginx to 10 EC2 instances.

I want to be able to add/update/delete automatically server names to my upstream nginx configuration, when autoscaling adds/updates/deletes EC2 instances.

This can be achieved by using Amazon SDK ( I am almost done with it, will put it on github ), utilizing the SNS, EC2 and Autoscaling service.

I have followed the below steps to achieve this:

1. Enable HTTP notification and subscribed my webserver.
2. Added a lifecycle-hook with heartbeat of 1 min (to wait for 1 min before terminating) to my autoscaling group for terminating server
3. Created an index file to parse the message to detect what kind of message it is ( i.e Launch or Terminate)
4. Once type of event is decided i queried EC2 to get the private ip of the instance
5. In case of Launch wait till header 200 is recieved and then add the ip to nginx config and reload
6. In case of Terminate remove the IP from config and reload nginx

Please find the script here https://github.com/singhupendra/aws-autoscale

Thank you @talonx, I've done some research, Amazon Autoscale has an api to query current autoscaling group status, and enumerates its members. It returns instance id (http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/api_requests.html#query-example), then you can use the describe tools to get the server name (http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DescribeInstances.html) and finally recreate the upstream include file. I could sense the Autoscaling notifications to launch a process that performs these tasks.

I still didn't implement it but its a way to go.

One can also use Autocaling with SNS http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/ASGettingNotifications.html

I haven't implemented this yet myself, but I'm looking into using the On-the-fly Reconfiguration of NGiNX Plus. I'm thinking that either the AMI, or the configuration management (Puppet, Salt, or such) that sets up an Auto Scaling Group instance, could reach the NGiNX reconfiguraiton API (perhaps, via an internal Route53 domain name so no fixed IP would need to be used), and add itself to the upstream cluster for the reverse proxy. After that NGiNX's built-in health check would then take over for that [added] instance, and drop it in case it becomes unavailable. This seems the cleanest solution and there is no delay in adding the instance, and hardly any delay in dropping it since NGiNX Plus features out-of-band health check.

This approach avoids needing to set up an auto-discovery system (Consul, Serf, or such) which for smaller setups often seems like much overhead both in the terms of setup/administration as well as required EC2 instances. Consul, for example, requires minimum of three instances to be stable. Serf could perhaps run on the ASG instances themselves, but there's still the overhead of maintaining it, and if the ASG scales down to one or two instances, you'd lose the quorum.

Finally, this could be combined with automatic notification of Auto Scaling Group changes, perhaps on the NGiNX server(s) that is/are used for load balancing. A listener triggered by such notification (this may be what Upendra also referred to) could then instantly add the new instance to NGiNX via the On-the-fly modification API. Besides the cost of NGiNX Plus, this makes one wonder why would anyone use Elastic Load Balancer with its numerous issues in the first place.

Edit 2015-12-07: ngx_openresty's balancer-by-lua (see this GitHub thread) offers a another possible open source solution for hot-adding/removing servers from NGiNX upstream group. I have not yet experimented with this myself, but wanted to add a mention here for anyone stumbling across this post.