How to split out Prometheus config files?
Right now we're using Prometheus for our monitoring and we have a lot of config (our prometheus.yml main config file is 1400+ lines long).
I would like to split this out into logical groupings (maybe DEV/TEST/PROD?) but I can't seem to find any documentation as to how to use "includes" (or similar) in the Prometheus config file syntax.
Has anyone done this with their Prometheus config file? If so, how did you do it?
Solution 1:
The Prometheus config file (and other config files in the ecosystem) explicitly do not support any form of templating. Instead this is left up to your configuration management system to handle.
Additionally it sounds a bit unusual that you have dev/test/prod sections in your config file. Usually a) you'd have a Prometheus per environment and b) the main difference between these Prometheus servers would be a different value for the env
label in your external_labels
.