Best way to manage third-party / custom-built software with Puppet?

We use versions of Ruby, Collectd, Ant, Java (and more) that are not available in CentOS or EPEL repos. Up until now, our strategy for installing these has been kind of a hack:

• write a (version-controlled) script for each package that downloads the source or binaries off the host site with curl onto the Puppet master and compiles the source code if necessary; repackages into gzipped tarball if necessary

• use the Puppet file server to distribute the binaries to our servers, and Puppet manifests to unpack the tarball in /usr/local (or wherever)

Writing the scripts can be a pain, and they have to be updated if one of the sites we depend on for downloads changes their API. Also software is compiled separately in every environment, which seems wasteful and could potentially result in problems with missing compile-time dependencies (eg., Ruby: require 'readline' or require 'yaml' might work in some environments but not others)

So, I can think of two other options:

• Just check in the custom-built third-party binaries into subversion and distribute them with the rest of our Puppet code. I'm worried it will severely impact the performance of checkouts and Puppet code pushes; we're looking at nearly 800MB (and growing) of third-party code. Plus it just feels wrong to check, like, multiple versions and architectures of Java side-by-side into version control.

• Don't version control the binaries or write download scripts -- when we decide to upgrade Ruby, compile it on a dev box and upload the new package manually to all of our Puppet masters whenever we decide to upgrade. Except, what if the packages get wiped out? Or become out of sync on the different masters? Right now we can easily re-generate all of our custom packages from scratch.

Which of the three options is better? How do people typically manage custom-compiled / repackaged third-party software with Puppet? If you create a local Yum repository, do you version control the process you use to create the RPMs? What happens if your Yum repo is wiped out?

Puppet isn't really designed to distribute large files, so you should rather do this out of band. The best approach is to package the custom/third-party software as RPMs and host your own RPM repository. The RPM packaging (i.e. specfile and patches) should ideally be version controlled, and the RPM repository backed up or hosted on multiple machines.

I highly suggest utilizing a Yum repository for deploying all your software including that which you compile. Once properly configured you'll get all your dependency issues straightened out, version control, etc.

My leaning is to have a three branch system (dev, qa, prod) for my repositories. That allows for given systems to be added to different repositories and run through its paces accordingly.

I do that (and the scheduling for the yum updates) via Puppet.

Echoing the other two answers: use your distro's existing package management system.

We do this with Ubuntu and apt to install RabbitMQ and OAuth like this:

class apt::example {

  file  {  "/etc/apt/sources.list.d/repo.example.list":
    source => "puppet:///modules/apt/repo.example.list",
    ensure => present,
  }

  exec {"install-gpg-key":
    command => "/usr/bin/curl -s http://repo.example.com/[email protected] | /usr/bin/apt-key add -; /usr/bin/apt-get update",
    refreshonly => true,
    subscribe => File["/etc/apt/sources.list.d/repo.example.list"],
    require => File["/etc/apt/sources.list.d/repo.example.list"],
  }

}

and to use the repo:

class apache2::platform {

  package { ["librabbitmq0", "php5-amqp", "php5-oauth"]:
    ensure => installed,
    notify => Service["apache2"],
    require => File["/etc/apt/sources.list.d/repo.example.list"]
  }

  [...]

}

The require here makes sure the repo is added before we try to install anything from it.

There may be a better way to handle adding the GPG key and running and update as this command returns a positive exit code which causes puppet to think it failed, but since it works I haven't expended much effort to fix the spurious error message.

I didn't attempt to add the repo config to puppet or automate the build and add to the repo process for new packages but once you reach a certain size, these tasks will become worthwhile. Backups are always worthwhile.

The process for RPM and yum should be very similar.