What is the best way to keep track of changes people are making to your servers?

Solution 1:

You need bureaucracy like a CMDB ! , maybe.. but it's no silver bullet. The cheapest tool you can start with is MS word, or a wiki.

Servers in production need to be under change control, changes shouldn't be happening willy nilly.

You have to decide on the right level of bureaucracy for your business.

Why are there multiple people making changes to prod in such a small environment? It may be time to introduce a clear separation of roles & have one production person that that has admin access & rolls out all changes.

For rebuilding machines you can do something simple like a 'build guide' if you create lots of servers that are slightly different, then have a generic guide & fill in the blanks for specific servers.

You should also have your disaster recovery plan documented, so that the business knows what to do if a server / data is lost.

Solution 2:

Instituting a thorough deployment procedure, including use of a CMBD of some kind (whether the C is for Change or Configuration), is a great first step. Nick definitely covered it well in his answer. Process and procedure helps with the legitimate, intentional changes.

I would also recommend looking into a configuration monitoring tool such as Tripwire. These kinds of systems make use of your C(onfiguration)MDB and will alert whenever a device deviates. Not only will it help the sysadmins, since it detects the case where somebody unintentionally turns off HA when provisioning a new VM; but also will make your security folks happy when a rogue Domain Admin adds his buddy to the financials group.