Callbacks or hooks, and reusable series of tasks, in Ansible roles

I'm struggling with making my Ansible roles somewhat composeable and re-usable.

I have a role that performs several series of steps. Depending on the other roles being applied to a host, other roles need to run tasks at each step.

I'm working with PostgreSQL and PostgreSQL extension management, but to keep it simple let's use an example of a webserver and some apps that run on it.

• role: webserver
• role: myapp
• role: myotherapp

The webserver role:

• installs the webserver
• templates its configuration
• sets up its database
• starts the webserver

The myapp role needs to be able to add some settings to the webserver's config that must be applied before the webserver's database is created. It must also perform some actions before the webserver is first stated.

The myotherapp role also needs to add some settings to the webserver's config before first-run, without clobbering the settings from myapp. It must also perform some actions after the webserver has first started, restart it, do some more work, and restart it again.

(This is contrived, but expresses all the needs of my real use case without going into way more about PostgreSQL innards that you ever wanted to know).

I thought I should be able to use handlers for this. Have webserver just fire notifys at each relevant step, so other roles could hook those notifications to append to lists or add keys to hashes used for config templating, fire their own tasks, etc. Have a restart_webserver handler that myotherapp can notify each time it wants to issue a restart, without having to know how that restart is actually performed. etc.

However:

• If there's a notification, the handler must exist. Ansible doesn't permit optional handlers.

• If there's more than one handler for a notification, Ansible just picks the first one and ignores all the rest.

• Notifications fire only at the end of a play section. You can't fire them immediately. There's meta: flush_handlers but it flushes all handlers, even those from other unrelated roles and tasks that might not be expecting their handlers to run early.

• It's astonishingly difficult to append to a list, or add keys to a dictionary. So it's quite hard to compose multiple related roles, without just manually gluing the configuration together with some host_vars yml.

So handlers and notifications don't seem to be how to do this.

How does one write composeable tasks and roles?

Am I trying to use a bicycle wheel as a hammer? Are you just not meant to use roles this way? It seems consistent with the concept of a role.

Should anything vaguely re-usable or less than totally linear be written by dropping Ansible's tasks/roles/vars/etc system and doing it in Python as an Ansible module? If so, is there any practical way to implement most of the brains of such a module as Ansible task-lists, variables yml files, etc, and just having the control logic in the module, so the module doesn't have to re-implement tons of stuff?

The "PostgreSQL guts" version:

• A postgres_server role. Installs Pg from distro packages or from source code as required. Has lots of vars in defaults/main.yml that can be overridden to control behaviour. Provides the facility to template a PostgreSQL configuration to supply parameters for postgresql.conf. Lets the caller override or prepend lines to pg_hba.conf. initdb's PostgreSQL once installed. Starts PostgreSQL once configured. Knows how to start, stop and restart the postgres_server (handler). Exposes facts to tell other roles where PostgreSQL is installed.

• A bdr_extension role. Assigned to a host where the bdr extension will be installed in PostgreSQL. Needs to see facts exposed by postgres_sever to find out how to install. Can fetch/compile/install BDR from git or install from OS packages.

  PostgreSQL cannot start succesfully if bdr is in the configuration's shared_preload_libraries. So postgres_server's starting of the server must be delayed until this is installed, if this extension is enabled.

• A bdr_node role. Depends on postgres_server and bdr_extension. Knows the PostgreSQL configuration options required to allow BDR to run on PostgreSQL, e.g. max_wal_senders = 20. Needs to be able to ensure that the PostgreSQL config option shared_preload_libraries has bdr in its comma-separated list of values when postgres_server generates the config. Overrides the default pg_hba.conf template in postgres_server to enable replication entries.

  This role must run some SQL commands after PostgreSQL has started.

• A pg_stat_statements role. Can install pg_stat_statements extension from git or OS packages. Needs to see facts exposed by postgres_sever to find out how to install. Needs to be able to append some extension-specific lines to postgresql.conf. Needs to be able to append 'pg_stat_statements' to the shared_preload_libraries setting in the PostgreSQL server configuration.

  PostgreSQL cannot start succesfully if shared_preload_libraries is in the configuration's pg_stat_statements. So postgres_server's starting of the server must be delayed until this is installed, if this extension is enabled.

So a BDR node is-a PostgreSQL server, and has-a BDR extension and pg_stat_statements extension, as well as others.

I expected to be able to expose handlers in the postgres_server role, and invoke them from other roles as needed to invoke appropriate activities in the postgres_server role, like a DB server restart, but it doesn't seem to be designed that way at all.

Am I getting it all completely backwards by thinking of Ansible roles as actually modelling roles?

I'm not familiar with PostgreSQL so I'm unsure of the contents of the various configuration files you mentioned, but if you need to allow multiple roles to contribute to a single configuration file, I've accomplished this in one of two ways depending on what the configuration file supports:

1. Use a configuration file with lots of include statements to include additional configuration files located in sub-directories. This works well with something like Apache for rewrite rules, application-specific configuration directives, and virtual hosts.

2. Assemble the configuration file from multiple partial configuration files stored in a directory using the assemble module. I've used this method to construct a master.cf file for Postfix as I had a postfix role and another role that needed to add to the Postfix master.cf file. So I modified the postfix role to create a /etc/postfix/master.d directory and a handler that assembles the files in that directory into a master.cf file. Unfortunately, as you pointed out, since I can't call the handler in the postfix role from another role, I had to duplicate that handler in my additional role. It's slightly less than idea but it works and it's idempotent.