Is there a streamlined way to export settings from an existing CentOS 5.10 server for local Vagrant provisioning? (Puppet, Chef, shell, or otherwise)

I've recently started the process of learning how to use Vagrant for spinning up a local development environment. (after getting frustrated with the various WAMP stack options) Vagrant is great so far... slight (ha) learning curve, but looks promising long term.

Having reached the tedious process of trying to make my VM look like my online server inside... it seems to me there is a missing piece to Vagrant. The process right now seems more or less like manual trial and error. Not ideal to get quickly set up the first time around or keep things synced from an upstream server easily.

Admittedly, I may not know exactly what I'm searching for ... hence the question.

Is there a streamlined way to export settings from an existing CentOS 5.10 server for local Vagrant provisioning? (Puppet, Chef, shell, or otherwise)

Something like this is what I would envision...

(Connect to online server...)

1. Detect repo differences and enable, disable, add on local as needed.
2. Detect packages, and sync local to match. (install or delete from local)
3. Get httpd.conf, tweak for local (if needed), and sync.
4. Get php.ini, tweak for local (if needed), and sync.
5. Get MySQL settings, tweak for local (if needed), and sync.
6. Get timezone and sync.
7. [Your suggestions for additional things that should be synced welcome here...]

Ideally, this would be run during provisioning and basically keep the local version in sync with the online version. This would remove the need to constantly be tweaking your local set up manually to keep things in sync. If something was changed online (by host or in-house)... it would propagate down automatically. (of course, ideally, you could flag settings to tweak behavior to your needs)

Alternatively, I guess if I could package up the online server without packaging the various user specific data, that could work as well. However, that doesn't seem possible from what I can tell... and it certainly wouldn't be very efficient.

Caveat

In my personal circumstance I'm on a CentOS 5.10 server with cPanel. cPanel seems to do a lot of stuff server side that isn't necessarily immediately obvious. One example is a lot of the package names start with cPanel and seem proprietary, but at the same time related to stuff I might want to sync. (such as cpanel-php53) As far as I can tell so far, those can't be synced easily... so work arounds would need to be put in place. Another example might be different paths than expected, but I'm not quite sure as I'm not familiar enough with both CentOS and cPanel default installs to be sure of any idiosyncrasies.

What I've done so far...

I did some stuff before I decided to ask if there's a better way that works more hand in hand with Vagrant. It's not awful, but not really "streamlined" or comprehensive. Here are the details...

1. I learned how to run yum repolist all on both machines and how to see repos in the filesystem using cd /etc/yum.repos.d; ll;, but not how to use this info to sync repos automatically.

2. I wrote a shell script to get the packages on local very close to those on remote. However, while it does a fair job, it's not perfect and I wonder if there's a better way out there. Issues...

   • I'm unsure about allowing removal of packages unique to local. (also, throws a few errors)
   • I haven't sorted out how to compensate for packages preprended with "cpanel" that seem to be replacements for what I really want (PHP, MySQL, etc) other than to remove each from the install list entirely.
   • This cannot be run as a Vagrant provisioner because it requires user input by its nature. On the same note running this and removing packages "unique to local" would obliterate yum installs previously run during provisions.

#!/usr/bin/env bash

# This script is a helper for syncing local packages to match a remote server.
# It is geared toward a remote CentOS server with cPanel thrown in the mix
# and a local "server" managed by Vagrant. Regardless, the concepts are the
# same for many kinds of set-ups and it's fairly tweakable.

# To run this script from the command line...
# Be root or become root by running 'sudo -i',
# then run 'source /vagrant/.vagrant/sync-packages.sh'

remote_host=1.1.1.1
destination=/vagrant/.vagrant/

echo -e '\nGetting packages from remote server...'
ssh root@${remote_host} "rpm -qa --queryformat='%{NAME}\n' | sort" > ${destination}packages-remote.txt

echo 'Getting packages from local server...'
rpm -qa --queryformat='%{NAME}\n' | sort > ${destination}packages-local.txt

echo 'Compiling package sets for comparison...'

comm -23 ${destination}packages-remote.txt ${destination}packages-local.txt                         > ${destination}packages-remote-only.txt
comm -23 ${destination}packages-local.txt ${destination}packages-remote.txt                         > ${destination}packages-local-only.txt
sed -r '/^(cpanel|newrelic)/!d' ${destination}packages-remote-only.txt                              > ${destination}packages-remote-only-proprietary.txt
comm -23 ${destination}packages-remote-only.txt ${destination}packages-remote-only-proprietary.txt  > ${destination}packages-remote-only-non-proprietary.txt

echo "Packages total  on local  $(cat ${destination}packages-local.txt | wc -l)"
echo "Packages unique to local  $(cat ${destination}packages-local-only.txt | wc -l)"
echo "Packages total  on remote $(cat ${destination}packages-remote.txt | wc -l)"
echo "Packages unique to remote $(cat ${destination}packages-remote-only.txt | wc -l)"
echo "Packages unique to remote *proprietary* $(cat ${destination}packages-remote-only-proprietary.txt | wc -l)"
echo "Packages unique to remote *non-proprietary* $(cat ${destination}packages-remote-only-non-proprietary.txt | wc -l)"

# If there are packages that are unique to local, prompt for removal

if [[ -s ${destination}packages-local-only.txt ]]; then
    read -p 'Do you want to remove the packages that are unique to local? (y/n) ' -n 1 -r; echo

    if [[ $REPLY =~ ^[Yy]$ ]]; then
        echo 'Removing packages (this runs in quiet mode and may take some time)...'
        yum -y -q remove $(cat ${destination}packages-local-only.txt)
    fi
fi

# If there are *non-proprietary* packages that are unique to remote, prompt for install

if [[ -s ${destination}packages-remote-only-non-proprietary.txt ]]; then
    read -p 'Do you want to install the *non-proprietary* packages that are unique to remote? (y/n) ' -n 1 -r; echo

    if [[ $REPLY =~ ^[Yy]$ ]]; then
        echo 'Installing packages (this runs in quiet mode and may take some time)...'
        yum -y -q install $(cat ${destination}packages-remote-only-non-proprietary.txt)
    fi
fi

# Wrap up

echo 'Ensuring all packages are up to date (this runs in quiet mode and may take some time)...'

yum -y -q update

echo -e "\nWe're all done here. If you need to see a log of changes, please run 'nano /var/log/yum.log'\n"

3. through 7. I wrote a shell script to get some remote config files and move them into place locally. It works fairly well on the surface, but I haven't deeply tested the results yet. I did run date to see that the timezone was synced as desired and checked some file contents to verify success.

Again, this cannot be run as a Vagrant provisioner because it requires user input by its nature. Also, no tweaking was done on the files to ensure they would run locally without issue. (such as http.conf to ensure Apache doesn't stumble on something or pointing MySQL to the correct data directory) Lastly, I'm sure these aren't the only files I should be porting over... these were just the most obvious ones.

#!/usr/bin/env bash

# This script is a helper for syncing local settings to match a remote server.
# It is geared toward a remote CentOS server with cPanel thrown in the mix
# and a local "server" managed by Vagrant. Regardless, the concepts are the
# same for many kinds of set-ups and it's fairly tweakable.

# To run this script from the command line...
# Be root or become root by running 'sudo -i',
# then run 'source /vagrant/.vagrant/sync-settings.sh'

remote_host=1.1.1.1
destination=/vagrant/.vagrant/

echo 'Getting config files from remote server...'

scp root@${remote_host}:"\
/usr/local/apache/conf/httpd.conf \
/usr/local/lib/php.ini \
/etc/my.cnf \
/root/.my.cnf \
/etc/localtime \
" ${destination}

echo 'Syncing files...'

mv -f ${destination}httpd.conf  /usr/local/apache/conf/httpd.conf
mv -f ${destination}php.ini     /usr/local/lib/php.ini
mv -f ${destination}my.cnf      /etc/my.cnf
mv -f ${destination}.my.cnf     /root/.my.cnf
mv -f ${destination}localtime   /etc/localtime

echo 'All done!'

Solution 1:

You may want to just use Blueprint.

Generating Puppet modules and Chef cookbooks

You'll be able to edit the resulting bundle to taste before attempting to deploy it to another server.