How can we keep an organized inventory of server hardware, virtual guests, software licenses and associated services?

We frequently need to figure out questions like:

1. What services would be affected if we bring down a server x?
2. When does license x expire?
3. Which KVM guests are on host x?
4. Can our VMWare license support feature x?

How do you guys handle these questions? Is there a tool or software that helps keep an accurate inventory of all or most of these items?

Solution 1:

I can tell you what doesn't work: having a manually-edited document, spreadsheet, webpage or wiki.

I've yet to see an organization strict enough to manually keep their technical documentation 100% accurate and up-to-date. Inaccurate documentation is not just annoying; it's plain dangerous. What happens if someone forgets to note that service a is running on server x? If you trusted the documentation, you'd end up with unexpected downtime of service a.

The first part of the solution is to automate the process using a monitoring platform such as Xymon, Nagios or Zabbix (there are plenty of others). It's possible to extend these platforms with custom scripts to answer most questions about your software and hardware. The monitoring platform will ensure that this automatic documentation is kept up-to-date.

The second part of the solution is to have a mechanism which warns you when your coverage is incomplete. For example, I use a custom script which lists all the processes running on a system, filters out the processes that are monitored or ignorable, and alerts me of any that are unexpected -- this tells me that it's time to add those new processes to the monitoring platform. Another example is to cross-reference your Cisco port status with CDP information to make sure you've covered all of your servers.

The great benefit of this approach is that you're doing documentation and you get alerting and trending for free.