Deploying distributed apps with Puppet - advice?

I'm looking into migrating our deployment process over to using Puppet. I'm starting by deploying the third-party apps we use for the system, which include Solr Cloud orchestrated by a Zookeeper ensemble.

Deploying Solr cloud with the inbuilt Zookeeper is fairly simple, however if I want to use a seperate Zookeeper ensemble then I need to deploy Zookeeper with these crucial lines in the config:

server.1=ip1:2888:3888
server.2=ip2:2888:3888
server.3=ip3:2888:3888

(for example). Given that I can't assume anything about the ordering of nodes, and I'd like to keep this all Puppet-based: what should I look at for doing this?

Do I need to do it using templates and some Hiera lookups, and define all the Zookeeper servers in my Hiera config? Or is there a method that allows me not to specify where all the nodes are in advance, and have them identify each other (using custom facts maybe)?

I'd like to minimize the amount of site-specific config I have to write, so if I can just specify that "this node needs the zookeeper module" and it automatically finds any others that have the zookeeper module, that would be brilliant.

Any advice from sysadmins who've done something similar would be appreciated!

Using Hiera is the right approach and you can achieve the dynamic nature of what you want by using PuppetDB along with the PuppetDB Hiera backend.

This allows you to have something like the following in your Hiera YAML files:

zookeeper::servers::_nodequery: ['Class[Zookeeper]', 'ipaddress']

This will return an array of $::ipaddress facts for all of the machines PuppetDB knows about with the Zookeeper class as part of their catalog. You can then use the result of this query as a parameter to other classes.