Nagios 3 config ideas
Solution 1:
I recently divided files by business area, then into staging/production, but that required too much prior knowledge from other colleagues to just find something and change it. Also we moved some servers around and I realised it would be useful to have config-per-host files that could be moved as well. Also, having config-per-host allows you to neatly encapsulate all the monitoring for a particular host (or hostgroup) in one file, and see easily what you are doing for that host. On top of that we have a central commands.cfg file, divided up by each business area which can be easily searched.
However none of this is truly satisfactory - using flat-file storage for configuration commits you to favouring one particular ordering method over another. This sort of configuration really benefits from the relational model. Most new monitoring products have implemented configuration from a database for this reason, I guess.
Currently NDOUtils (warning: pdf) gets your config and event data into a database, but as far as I know there's no way to tell Nagios to use that database as config source. It does give you the opportunity to create tables aside from those required by NDOUtils, perhaps creating a separate nagios_config database where you can map things out and then script up a solution that pulls this information out of the database and creates your config.
I'm going to try and use Puppet to generate my Nagios configs in the near future, hopefully it will let me resolve these sorts of issues.
Solution 2:
Our setup for is one config file per host, but they are all contained in one "hosts" directory. Then we specify this directory with "cfg_dir=" in the nagios config and any hosts added to that directory are automatically picked up on the next reload.
Solution 3:
I have several groups of identical hosts. So I have a config file for each "type" of host and use a host group definition at the top of the file so that I only have to have 1 set of service definitions per "type". I also have some hosts that are unique which get their own config file and I keep escalations in a separate file per host/type so that I can turn them off simply by commenting out the line that imports that particular file.