Redundant Linux Servers in Seperate Datacenters

MySQL has replication capabilities built in - no need for DRBD. See here.

This replication occurs over the normal MySQL protocol, on TCP port 3306. The native protocol supports TLS encryption, but a VPN might not be a bad idea, either, in light of recent vulnerabilities. Up to you!

From there, you'll just need for your application that's utilizing MySQL to be aware of both servers in some way, or work out some other kind of failover mechanism depending on your application - sounds like you have a local instance of your web application in each location, so just pointing each to its local MySQL instance may do the trick.

Be careful, though - if you're replicating in both directions, you likely don't want both MySQL servers being written to at the same time; conflicting changes could be made to the different servers.

DRDB only really works as a failover solution. As Shane says, MySQL replication is much more sensible approach. While you could implement this as master/slave you've then got the complications of promoting the slave when you detect an outage. A better solution is to use master-master replication.

But there are limits to the effectiveness of this - replicated writes are implemented by a single thread - so the 'slave' has to work harder than the master to apply them. There is inevitably a delay - although usually this is small enough not to be an issue.

Althouh mysql supports SSL encryption, it doesn't provide a great deal of functionality for tuning access control. It's trivial to set up stud or stunnel to wrap a specific port in SSL. And the replication connection(s) are held open, so the bandwidth overhead is very low. Or even just restrict access to the MySQL port to the remote IP address using iptables.

But you've still got to address the problem of detecting failures and fencing the failed node. Presumably you're doing round-robin balancing across the HTTP servers - in which case clients will tend to stay on the same servers for the duration of the session. The webservers will get the best performance from the local DBMS instance - so should only try to access the remote system when you detect an outage on the local system. Don't worry about trying to automate the recovery - as long as you can script it.

Do read up carefully on MySQL replication - if you do the groundwork then conflicting writes should be very, VERY rare.