Load balanced Drupal servers: a good way to share data among the servers?

Solution 1:

Before I moved to the CDN I had a web farm of about 20 servers. I started with NFS until I hit a bug in the linux kernel about 3 years back that caused an issue with not mounting the volume and it wasn't failing over correctly.

I then created an rsync script that would sync the dir's every 5 minutes to each image server. This wasn't ideal but it worked and I didn't have any issues with it.

The next step I switched to lsyncd which uses inotify to monitor a directory and when a file is changed it spawns a process to sync that file. By default that's rsync. That solution was solid and if not for the economy of a CDN I would have continued with that.

Lsyncd: http://code.google.com/p/lsyncd/

Hope that helps.

Solution 2:

John and Cailin's five-step tutorial on scaling Drupal will be very helpful to you.

The first two steps are as follows:

Step 1: Separate your web server from your database server.

Step 2: Set up redundant web servers and load balancing.

Bottom line: you are not going to want to scale horizontally (adding multiple servers with identical functions) without first scaling vertically (separating functions that currently reside on the same server into different tiers).

Solution 3:

I have used glusterfs (http://www.gluster.org/) to share files between load balanced web servers. I tried to also share the MySQL database files between the servers as well with glusterfs, but with no success - I recommend to separate the web from the database server as the first poster suggests.

Other solutions to share files between servers are lustre (http://wiki.lustre.org/index.php/Main_Page) and, depending on the load balancer setup, you could also use lsyncd (http://code.google.com/p/lsyncd/) to synchronize files between the servers.