ansible and reloading AWS dynamic inventory

See also: https://stackoverflow.com/questions/29003420/reload-ansibles-dynamic-inventory.

My question: is there a better way of doing what's below?

I have an ansible role that provisions AWS machines, and runs correctly (notice the provision tag):

- name: AWS provision
  hosts: localhost
  gather_facts: no
  vars_files:
    - vars/dev.yml
  user: ec2-user
  roles:
    - provision
  tags:
    - provision

I then have a base role, that I want to be able to run independently (for example during development, so I don't have to wait for re-provisioning (notice the base tag). I run a play find running instances that filters and stores the hosts in the group started:

- name: find running instances
  hosts: localhost
  vars_files:
    - vars/dev.yml
  gather_facts: no
  tags:
    - base
  tasks:
    - name: gather remote facts
      ec2_remote_facts:
        region: "{{ target_aws_region }}"
        filters:
          instance-state-name: running
          "tag:Name": "{{ instance_name }}"
      register: ec2_facts

    - debug: var=ec2_facts

    - name: add hosts to groups
      add_host:
        name: "{{ item.id }}"
        ansible_ssh_host: "{{ item.public_dns_name }}"
        groups: started
      changed_when: false
      with_items: ec2_facts.instances

- name: base setup
  hosts: started
  gather_facts: no
  vars_files:
    - vars/dev.yml
  user: ec2-user
  roles:
    - base
  tags:
    - base

My question: the plays are working, but is there a better way of doing this? For example I'm got gather_facts: no followed by ec2_remote_facts and the filters - it all seems rather convoluted.

A clarification: thanks for the comment about ec2.py -- I'm already using it in my first play (when I call the provision role).

But for testing purposes I want to jump into subsequent plays without re-doing the (slow) provisioning. So how do I re-populate my hosts data? Is ec2_remote_facts followed by add_host the right way? Or can I somehow use gather_facts: yes?

I'd probably use the EC2 dynamic inventory script instead, which you can employ by configuring ec2.ini and passing -i ec2.py to ansible-playbook.

See http://docs.ansible.com/ansible/intro_dynamic_inventory.html#example-aws-ec2-external-inventory-script for more info.

Note that there are plenty of options in ec2.ini. Be sure to have a look at those, e.g. cache_max_age. You can also make the inventory generation faster by filtering unnecessary resources (e.g. set rds = False if you are only interested in EC2 instances).

UPDATE: With Ansible 2.x+ you can also use - meta: refresh_inventory mid-play.

While the meta: refresh_inventory is the "preferred method", I tend to like OP's proposal of using ec2_remote_facts in conjunction with add_host. I've setup such a playbook and it has the strength to be 100% dynamic without caching glitches.

Assuming your ASG fired up instances with the env: cool_asg_instance tag, just add the following under the ec2_asg playbook call:

- ec2_remote_facts:
    filters:
      "tag:env": "cool_asg_instance"
  register: instance_facts

You'll then gather a full JSON dataset containing all the needed informations, from there you can use Jinja2 capabilities within the playbook to extract newly created IP addresses, i.e.:

- name: group hosts
  add_host: hostname={{ item }} groups=launched
  with_items: "{{ instance_facts.instances|selectattr('state', 'equalto', 'running')|map(attribute='private_ip_address')|list }}"

Filter is courtesy of this wonerfull blog post: https://bonovoxly.github.io/2016-02-11-ansible-stuffs-ec2_remote_facts_instead_of_ec2_py

From now on you can use the launched group on your parent deployment YAML file like this:

- hosts: launched
  gather_facts: no

  tasks: 
    - name: wait for SSH
      wait_for: port=22 host="{{ inventory_hostname }}" search_regex=OpenSSH delay=5

Some may ask why the headache, well imagine that instead of having an hideous userdata which will git clone both Ansible and a playbook from the Internet, you can trigger the instance setup from your own deployment center by setting up a simple SNS topic which will publish to a SQS queue, watched by a 10 lines python code (https://github.com/alexandregama/python-sqs-consumer/blob/master/sqs-message-consumer-polling.py) which will trigger Ansible when a new instance comes out.