How to work with Git branches and Rails migrations

I am working on a rails app with quite a few git branches and many of them include db migrations. We try to be careful but occasionally some piece of code in master asks for a column that got removed/renamed in another branch.

1. What would be a nice solution to "couple" git branches with DB states?

2. What would these "states" actually be?

   We can't just duplicate a database if it's a few GBs in size.

3. And what should happen with merges?

4. Would the solution translate to noSQL databases as well?

   We currently use MySQL, mongodb and redis

EDIT: Looks like I forgot to mention a very important point, I am only interested in the development environment but with large databases (a few GBs in size).

Solution 1:

When you add a new migration in any branch, run rake db:migrate and commit both the migration and db/schema.rb

If you do this, in development, you'll be able to switch to another branch that has a different set of migrations and simply run rake db:schema:load.

Note that this will recreate the entire database, and existing data will be lost.

You'll probably only want to run production off of one branch which you're very careful with, so these steps don't apply there (just run rake db:migrate as usual there). But in development, it should be no big deal to recreate the database from the schema, which is what rake db:schema:load will do.

Solution 2:

If you have a large database that you can't readily reproduce, then I'd recommend using the normal migration tools. If you want a simple process, this is what I'd recommend:

• Before switching branches, rollback (rake db:rollback) to the state before the branch point. Then, after switching branches, run db:migrate. This is mathematically correct, and as long as you write down scripts, it will work.
• If you forget to do this before switching branches, in general you can safely switch back, rollback, and switch again, so I think as a workflow, it's feasible.
• If you have dependencies between migrations in different branches... well, you'll have to think hard.

Solution 3:

Here's a script I wrote for switching between branches that contain different migrations:

https://gist.github.com/4076864

It won't solve all the problems you mentioned, but given a branch name it will:

1. Roll back any migrations on your current branch which do not exist on the given branch
2. Discard any changes to the db/schema.rb file
3. Check out the given branch
4. Run any new migrations existing in the given branch
5. Update your test database

I find myself manually doing this all the time on our project, so I thought it'd be nice to automate the process.

Solution 4:

Separate Database for each Branch

It's the only way to fly.

Update October 16th, 2017

I returned to this after quite some time and made some improvements:

• I've added another namespace rake task to create a branch and clone the database in one fell swoop, with bundle exec rake git:branch.
• I realize now that cloning from master is not always what you want to do so I made it more explicit that the db:clone_from_branch task takes a SOURCE_BRANCH and a TARGET_BRANCH environment variable. When using git:branch it will automatically use the current branch as the SOURCE_BRANCH.
• Refactoring and simplification.

config/database.yml

And to make it easier on you, here's how you update your database.yml file to dynamically determine the database name based on the current branch.

<% 
database_prefix = 'your_app_name'
environments    = %W( development test ) 
current_branch  = `git status | head -1`.to_s.gsub('On branch ','').chomp
%>

defaults: &defaults
  pool: 5
  adapter: mysql2
  encoding: utf8
  reconnect: false
  username: root
  password:
  host: localhost

<% environments.each do |environment| %>  

<%= environment %>:
  <<: *defaults
  database: <%= [ database_prefix, current_branch, environment ].join('_') %>
<% end %>

lib/tasks/db.rake

Here's a Rake task to easily clone your database from one branch to another. This takes a SOURCE_BRANCH and a TARGET_BRANCH environment variables. Based off of @spalladino's task.

namespace :db do

  desc "Clones database from another branch as specified by `SOURCE_BRANCH` and `TARGET_BRANCH` env params."
  task :clone_from_branch do

    abort "You need to provide a SOURCE_BRANCH to clone from as an environment variable." if ENV['SOURCE_BRANCH'].blank?
    abort "You need to provide a TARGET_BRANCH to clone to as an environment variable."   if ENV['TARGET_BRANCH'].blank?

    database_configuration = Rails.configuration.database_configuration[Rails.env]
    current_database_name = database_configuration["database"]

    source_db = current_database_name.sub(CURRENT_BRANCH, ENV['SOURCE_BRANCH'])
    target_db = current_database_name.sub(CURRENT_BRANCH, ENV['TARGET_BRANCH'])

    mysql_opts =  "-u #{database_configuration['username']} "
    mysql_opts << "--password=\"#{database_configuration['password']}\" " if database_configuration['password'].presence

    `mysqlshow #{mysql_opts} | grep "#{source_db}"`
    raise "Source database #{source_db} not found" if $?.to_i != 0

    `mysqlshow #{mysql_opts} | grep "#{target_db}"`
    raise "Target database #{target_db} already exists" if $?.to_i == 0

    puts "Creating empty database #{target_db}"
    `mysql #{mysql_opts} -e "CREATE DATABASE #{target_db}"`

    puts "Copying #{source_db} into #{target_db}"
    `mysqldump #{mysql_opts} #{source_db} | mysql #{mysql_opts} #{target_db}`

  end

end

lib/tasks/git.rake

This task will create a git branch off of the current branch (master, or otherwise), check it out and clone the current branch's database into the new branch's database. It's slick AF.

namespace :git do

  desc "Create a branch off the current branch and clone the current branch's database."
  task :branch do 
    print 'New Branch Name: '
    new_branch_name = STDIN.gets.strip 

    CURRENT_BRANCH = `git status | head -1`.to_s.gsub('On branch ','').chomp

    say "Creating new branch and checking it out..."
    sh "git co -b #{new_branch_name}"

    say "Cloning database from #{CURRENT_BRANCH}..."

    ENV['SOURCE_BRANCH'] = CURRENT_BRANCH # Set source to be the current branch for clone_from_branch task.
    ENV['TARGET_BRANCH'] = new_branch_name
    Rake::Task['db:clone_from_branch'].invoke

    say "All done!"
  end

end

Now, all you need to do is run bundle exec git:branch, enter in the new branch name and start killing zombies.