Replicating a linux web server
Solution 1:
You don't need paid TLS certificates for your own private communication. You can set up your own CA (with very long lasting certs, in case of compromise you just throw away the entire CA) and make your servers trust it, then you can issue as many certs as you want for different services. Paid certificates are only needed when you can't reliably make the remote hosts trust your CA, like your website's visitors for example.
If you just need to use a single service and it supports TLS (like MySQL does), go with that and add an additional layer of security by only allowing connections from your server's IPs at the firewall level.
If you need more than one service, you're better off with a VPN solution. Don't waste your time with OpenVPN, your kernel has built-in IPSec support and you can use that. Plus, it's supported out of the box on Windows so if you ever deploy such servers it'll be easy to set up.
That was the easy part. The real hard part is to keep the files of your app in sync, it's easy if your app only uses a database, but if it's a general-purpose CMS there's a good chance it also modifies its own files for whatever reason (plugin updates for example) or creates new ones (user-uploaded content, etc) and I don't know of any reliable way of keeping them in sync. The only solution that comes to mind is either NFS (and only having a single server which hosts the files, but that's against your HA requirement) or GlusterFS, both of which will perform quite poorly with this kind of latency.
Solution 2:
rsync
is a great tool for keeping files in sync. I would use it in combination with SSH (and public keys), like this:
rsync -az -e ssh --delete /var/www otherserver:/var/www
For multiple servers multiple uses of rsync
might be the best option. Another option is pdcp -r
, but that requires copying all the files every time instead of doing delta-transfers. In other words, it's better for small amounts of data and many servers.
How to best do database replication depends greatly on what your application does. There's a lot of good advice in the MariaDB docs and other questions around here.