What to use for software-based shared file storage?
Solution 1:
DRBD is great.
The good things:
- It does a magnificent job at replicating data
- DRBD has in a couple of cases prevented disaster, where it has discovered that the volume was already mounted on the other node, which the raw volumes we get from a SAN are unable to tell us.
- Heartbeat already has great support for DRBD.
The challenges:
- Remember to monitor it properly, so you discover split brains when they happen - and can deal with it.
- DRBD can't be mounted on both servers without a cluster-enabled filesystem on top - I don't have any experience with that part.
- It is easy to "DOS" the servers by configuring DRBD to use all the available bandwidth for syncing disks. Just configure for lower throughput, and you're OK.
For mounting "the same" filesystem on several nodes, we keep going back to NFS, even though we keep testing various solutions for it. A setup I have no problem with having in production is NFS on top of EXT4 on top of DRBD. I wouldn't dare to do this with the database filesystems, but it's OK for the wwwroot.
Solution 2:
DRBD mirrors data in real time, transparently. Please be noted some points as under:
- You need a cluster file system for the dual-primary mode. OCFS2 is only supported CentOS 5. If you're using CentOS 6, you should use GFS2 instead.
- Although all the server are on the same local network, I still recommend connecting with a crossover cable.
- You should have a monitoring mechanism to detect split brain, for e.g: Nagios's check_drbd plugin.
- Don't forget to measure and optimize DRBD performance
If you want to setup 3 or 4 nodes, take a look at:
- http://www.drbd.org/users-guide/s-three-nodes.html
- http://www.drbd.org/users-guide/s-pacemaker-stacked-resources.html