At what point does a config file become a programming language?

I have been mulling over config files and their relationship to code for a while now and depending on the day and direction of the wind my opinions seem to change. More and more though I keep coming back to the realization I first had while learning Lisp: there is little difference between data and code. This seems doubly true for config files. When looked at in the right light a Perl script is little more than a config file for perl. This tends to have fairly heavy consequences for tasks such as QA and divisions of labor like who should be responsible for changing config files.

The creep from config file to full fledged language is generally slow and seems to be driven by the desire to have a generic system. Most projects seem to start out small with a few config items like where to write logs, where to look for data, user names and passwords, etc. But then they start to grow: features start to be able to be turned on or off, the timings and order of operations start to be controlled, and, inevitably, someone wants to start adding logic to it (e.g. use 10 if the machine is X and 15 if the machine is Y). At a certain point the config file becomes a domain specific language, and a poorly written one at that.

Now that I have rambled on to set the stage, here are my questions:

1. What is the true purpose of a config file?
2. Should an attempt be made to keep config files simple?
3. Who should be responsible for making changes to them (developers, users, admins, etc.)?
4. Should they be source controlled (see question 3)?

As I said earlier my answers to these questions shift constantly, but right now I am thinking:

1. to allow a non-programmers to change large chunks of behaviour quickly
2. yes, anything that is not coarsely grained should be in code
3. users should be responsible for config files and programmers should be responsible for a configuration layer between config files and code that gives more fine grained control of the application
4. no, but the finer grained middle layer should be

Solution 1:

Very interesting questions!

I tend to limit my config files to a very simple "key=value" format, because I fully agree with you that config files can very quickly become full-blown programs. For example, anyone who has ever tried to "configure" OpenSER knows the feeling you are talking about: it's not configuration, it's (painful) programming.

When you need your application to be very "configurable" in ways that you cannot imagine today, then what you really need is a plugins system. You need to develop your application in a way that someone else can code a new plugin and hook it into your application in the future.

So, to answer your questions:

1. What is the true purpose of a config file?

   I would say, to allow the people who will install your application to be able to tweek some deployment-related parameters, such as host name, number of threads, names of the plugins you need, and the deployment-parameters for those plugins (check out FreeRadius's configuration for an example of this principle), etc.. Definitely not the place to express business logic.

2. Should an attempt be made to keep config files simple?

   Definitely. As you suggested, "programming" in a config file is horrible. I believe it should be avoided.

3. Who should be responsible for making changes to them (developers, users, admins, etc.)?

   In general, I would say admins, who deploy the application.

4. Should they be source controlled (see question 3)?

   I usually don't source-control the configuration files themselves, but I do source-control a template configuration file, with all the parameters and their default values, and comments describing what they do. For example, if a configuration file is named database.conf, I usually source-control a file named database.conf.template. Now of course I am talking about what I do as a developer. As an admin, I may want to source-control the actual settings that I chose for each installation. For example, we manage a few hundred servers remotely, and we need to keep track of their configurations: we chose to do this with source-control.

Edit: Although I believe the above to be true for most applications, there are always exceptions, of course. Your application may allow its users to dynamically configure complex rules, for example. Most email clients allow the users to define rules for the management of their emails (for example, "all emails coming from 'john doe' and not having me in the To: field should be discarded"). Another example is an application that allows the user to define a new complex commercial offer. You may also think about applications like Cognos which allow their users to build complex database reports. The email client will probably offer the user a simple interface to define the rules, and this will generate a complex configuration file (or even perhaps a bit of code). On the other hand, the user-defined configuration for the commercial offers might be saved in a database, in a structured way (neither a simple key=value structure nor a portion of code). And some other applications might even allow the user to code in python or VB, or some other automation-capable language. In other words... your mileage may vary.

Solution 2:

Ok. You will have some users which want a really simple config, you should give it to them. At the same time, you will have constant requests of "Can you add this? How do I do in the config file?", I don't see why you can't support both groups.

The project I am currently working on uses Lua for its configuration file. Lua is a scripting language, and it works quite well in this scenario. There is available an example of our default configuration.

You'll note that it is mainly key=value statements, where value can be any of Lua's built-in types. The most complicated thing there are lists, and they aren't really complicated (it's just a matter of syntax).

Now I'm just waiting for someone to ask how to set their server's port to a random value every time they start it up...