Nginx automatic failover

Solution 1:

We use Corosync and Pacemaker for an active/passive nginx cluster in our environment which runs pretty good.

Here are few key points to keep in mind, based on my recent experience.

  • Join the cluster nodes using Corosync (/etc/corosync/corosync.conf) and not Pacemaker, I had frequent problems like split brain with the latter in my environment.
  • Default option is to use pcs command to configure the cluster. However, you can use crm as well which is preferred by few people. You will have to install crmsh from Suse repositories depending on the OS. This is what I use in Red Hat based distros.

    wget http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/network:ha-clustering:Stable.repo -nd -O /etc/yum.repos.d/crmsh.repo && yum install crmsh
    
  • After installation make sure to disable stonith and quorum, and set resource stickiness as well. These commands should work fine assuming you have a two node cluster.

    pcs property set stonith-enabled=false
    pcs property set no-quorum-policy=ignore
    pcs resource defaults resource-stickiness="INFINITY"
    pcs resource defaults migration-threshold="1"`
    
  • For syncing files between the active and passive node you can use tools like rysnc or unison. For block level sync (may be /etc/nginx mounted on a filesytem), you can use drbd which can be easily added as a resource in the cluster

  • Lastly, make sure to group all resources and order them using command like pcs constraint colocation and pcs constraint order

You can use Google out all the above information easily, here are few links to get you kickstarted.

  1. Active Passive Cluster to setup apache (use nginx resource instead of apache)
  2. Ensure Resources Run on the Same Host
  3. Ensure Resources Start and Stop in Order