mirrored filesystem across a few servers

I'm looking for a solution to mirror or replicate one directory (or one filesystem) across a few Linux servers. Ideal solution would be one, which permits all servers read-write access. I also want it to me resilient, if one of the servers goes down, rest should still work w/o loosing any data at all.

I've been looking at some solutions:

• DRBD: block level replications, seems a bit overkill;
• lsyncd: looks very simple, but I have doubts about performance;
• GlusterFS: seems like that would be a good match, haven't figured out yet how exactly the replication mode works. Will it have characteristics I require?

Any other suggestions are welcome.

The first question I would ask is do you want this replicated to two servers or more then two servers? For two servers I would go with DRDB, for three or more I would go with gluster.

If I/O latency is not a critical concern I would go with gluster. It is pretty easy to setup and could clearly do what you needed. All you need to do is make a gluster server serving the files up on all three boxes and then also make each box act as a gluster client mounting the files.

DRDB is going to to be complicated to get working in a master<->master mode with 3 or more servers. You have to configure a ring based setup and I would not recommend it. However for two servers DRDB is fantastic. Master<->Master mode is not complicated to setup and you don't have to learn any of the file system stuff.

lsycd is great for a master / slave setup, but you don't appear to want to that.

Ceph is still pretty new, last time I checked it did not even have fsck support yet. I would rather base my infrastructure on something more stable.

Lustre is a fantastic product for large scale deployments, but you need to setup heartbeating and failover for the mds server or its going to have a single point of failure. Given the limited number of servers he is talking about I suspect its overkill in this case.

How about Ceph or Lustre?

You should look into OpenAFS -- it's a mostly distributed filesystem that allows for multiple copies of data to exist distributed across the cluster and everyone can read / write to the FS at the same time.

It's got a bunch of other useful features as well (good authentication, encryption on the wire, built-in local caching on clients, native windows client, portable across lots of versions of unix, etc)

It's a bit of a heavy lift to setup, though.

Getting this to work with DRBD is going to be really difficult - the problem is not as n8whnp seems to think an issue regarding multi-way replication (you just make all the nodes stripes in a mirror set) but is of concurrency control - you'd need to run a cluster filesystem on top of the mirroring on top of DRBD.

lsyncd is even worse as there's no practical solution for concurrency control.

I'd recommend an AFS type solution (AFS, OpenAFS) as a mature, stable, open solution. I'd stay clear of lustre since Oracle shut it down. Not overly familiar with glusterfs but as it relies on distributed rather than replicated storage I'd recommend you have a long hard look at how it will behave in split-brain operation (AFS OTOH is designed to work in disconnected mode).