Implications of using VMware's templating and cloning for provisioning Linux servers instead of PXE

There seems to be speed benefits when using VMware's templates to create new VMs. However, I'm concerned about possible less obvious security and flexibility implications. For e.g.:

• First boot generated keys
• VMware proprietary methods of configuring server settings (IPs, hostname, etc.)

In a shop that's almost entirely VMware with a majority of Linux being RHEL-based distros, has anyone come across some pitfalls when provisioning from templates? How about concerns when cloning?

BTW: Regardless of the initial provisioning method, it would primarily be used as a bootstrap to Puppet for the rest of the configuration.

Solution 1:

Depending on the environment and your provisioning process, it may be faster to build anew than using VMware templates and the clone from template feature.

I did work in a large Linux-focused VMware environment, where the deployment process wasn't as automated as it should have been. We relied on vSphere templates of RHEL systems, but quite a bit of manual work was needed following the initial clone.

Template advantages:

• Templates are good if you're baking other applications, configuration settings and things that extend beyond the reach of config management into the template. (e.g. a complex Oracle application stack)
• The cloning time is a function of your storage and vSphere infrastructure. I've seen really slow and resource intensive cloning jobs occur.
• If using Red Hat/CentOS/Debian/Ubuntu, for example, you can take advantage of the sys-unconfig command to "unconfigure" a system ahead of templating. This is the Linux equivalent to Microsoft's Sysprep and removes network interface rules, SSH keys, network settings, etc.
• VMware's IP assignment tools are okay and haven't been too much of a problem with mainstream Linux operating systems.

PXE/kickstart deployment advantages:

• Usually faster than templating/cloning, assuming you have a local repository and a net boot image.
• More flexible in that you can make changes to your images/master without the cycle of converting template to VM, modifying, reconverting and cloning.
• Best when coupled with a configuration management solution.

In the end, it depends on your application and how much additional work is needed following the initial OS install. I've taken a hybrid approach, using clones for the most complex instances and new builds via a provisioning system for routine systems. They're not mutually exclusive.

Solution 2:

Specifics depend on the distribution, but I'll give you a few CentOS/RHEL things to remember when building your template.

• Delete the keys in /etc/ssh/
• Remove the SUBSYSTEM lines in /etc/udev/rules.d/70-persistent-net.rules that refer to your NICs

Both of those will be re-generated the first time you boot.

You need some way of running updates after you deploy the template.

With vSphere you can define the networking and hostname. As a Puppet user, I can just install the Puppet client in my template along with a standard puppet.conf. If I am using multiple environments, then I actually need a different template for each environment.

When you modify a template, you have to remember to remove the files above.

Personally, while deploying from template is marginally quicker, I prefer to provision from PXE anyway as it is more flexible.

Solution 3:

Have a look at Preparing Linux Template VMs and the post it references, Creating a puppet ready image (CentOS/Fedora). Maybe this helps.

Btw: I really like your idea to deploy JeOS VMs and then use puppet to customize it into a DB, Web or whatever server.