Need a simple High Availability file share

I am looking for the simplest way to share a single file in a high availability way between a pair of linux servers. (Version and distribution are unimportant, I'm looking for a generic solution.)

I have two servers, each with their own local disks and NFS shares and other services between them. I have a file that both servers need to access, but nothing other than those servers needs to access it.

If either server crashes, I want the most recent possible contents of that file to be available to the remaining server. (Obviously the other server should pick up changes on recovery.)

The file is a state file, and likely only one server at a time will be writing to it. The state file size is unknown, but small. Probably between 1 block and 2M. Possibly the size of the state file would grow depending on the length of downtime.

Without adding external hardware, what options are there for a high availability file share like this?

There are a lot of solutions - for one file I would probably use GlusterFS - but I think that you should have 3 servers for quorum or you will have to solve split-brains on recovery. You should be able to install it easily on every popular distro, and configure it not more than hour.

If the pair of servers are running hot-cold (i.e., only one of them accesses the file at a time), DRBD is a quick and stable way to accomplish your goal. DRBD is designed with split-brain protections in place, so it should be "good enough".

A brief blurb from the DRBD site:

The Distributed Replicated Block Device (DRBD) is a software-based, shared-nothing, replicated storage solution mirroring the content of block devices (hard disks, partitions, logical volumes etc.) between hosts.

DRBD mirrors data

• in real time. Replication occurs continuously while applications modify the data on the device.
• transparently. Applications need not be aware that the data is stored on multiple hosts.
• synchronously or asynchronously. With synchronous mirroring, applications are notified of write completions after the writes have been carried out on all hosts. With asynchronous mirroring, applications are notified of write completions when the writes have completed locally, which usually is before they have propagated to the other hosts.

As this is a block-level replication, you would require a bit of extra configuration. E.g., you'd have to create a filesystem on top of the replicated device, and you'd need to mount that filesystem. The default recommended configuration only allows one host to mount the filesystem (to avoid split-brain situations), so you can only access the data on one node at a time.

The whole process is well documented and there are also some easy guides available.

If you are more into automation, Pacemaker + DRBD is a very common combination, it is even documented in the Pacemaker guides which is also a good intro to DRBD itself.

P.S. Funny how the pacemaker guide to DRBD I linked above almost perfectly describes your question.

Even if you’re serving up static websites, having to manually synchronize the contents of that website to all the machines in the cluster is not ideal. For dynamic websites, such as a wiki, it’s not even an option. Not everyone can afford network-attached storage, but somehow the data needs to be kept in sync.

Enter DRBD, which can be thought of as network-based RAID-1.