Migrate live server

I'm in trouble, I have a running server with the usual services (httpd, mail, sql), and looks like I must be really fast to prevent total data loss (my RAID array failed, so I'm depending on a single hdd now).

The whole system built on HowtoForge's howto's and as I read the one on Squeeze, looks like I can easily? migrate my (virtual) users onto the new box.

I really know this is very specific problem but could you tell me some important point of the whole migrating process. Which things can be tricky where should I need to focus at most? How would you start the process?

Some background infos:

• The two server is 200kms far from each other, so the migrating should be happen over internet
• Both server will be Debian, the old one is Etch, the new will be Squeeze
• The critical service is mail, websites can be suspended for some time
• RAID means two disks in RAID 1 (both machine)
• The old box is the DNS1 for all hosted domains

My (really) basic outline:

• Building a similar system like the old one (similar virtual mail structure, passwords stored with the same encryption type, because i cannot tell the old ones)
• create one domain where i can check everything is fine (dns, email, www are okay)
• Migrate domain after domain until the old one is doing nothing

My blind spots:

• Can't find a brief description about how to do a RAID-1 on Squeeze (a Lenny howto will be enough?)
• How to copy specific folders (e.g. users' mail folders) with specific user rights when needed? (rsync will do this?)
• How to setup DNS on the new machine (old one is DNS1) (new machine should be DNS too and migrated domains should use itself as DNS1?)
• How can i prevent e-mail loss (because of DNS refresh time) (i need to set up the new machine as MX?)

Solution 1:

Your basic outline is more or less correct.

The squeeze installer will do a software RAID just fine. One point of difference is that squeeze uses grub2 by default; lenny used grub-legacy (0.9 I think). This makes some grub administration somewhat different, particularly around ensuring grub will boot from both disks in case of failure. You can always revert to grub-legacy, which is what I ended up doing last time I was confronted with this problem. It'd pay to test whatever you're doing though, to make sure you can reboot if either disk fails.

To copy data across, use rsync for sure. I'd make sure the right accounts are set up first, with the same uids/gids, then rsync, but you can always fix this up afterwards. rsync -avPHAX should get everything (-a gets most things, except for -H[ard links] -A[CLs] and -X[tended attributes], so this is useful.

Given that you're doing an effective upgrade at the same time, prepare for services to break in subtle ways. Configurations will change slightly. Having done something similar recently, I think I'd prefer to do a like->like migration, then upgrade the final host sometime after the fact, as migrating everything and managing upgrades at the same time made it a mammoth task.

Updated for your additional pain points:

For DNS, I'd move all DNS hosting to the new machine in one step. Leave the records as they are to start with, but set the refresh right down (10 minutes or so). This gives you the ability to move DNS records over to the new machine as you complete moving each domain. Moving your DNS may require updating the whois records for any domains you host, so bear in this in mind. Set the refresh down for all domains right now in fact, before you start anything else - nothing worse than having a badly cached result causing problems.

Regarding email loss: As you move mail to the new server, make sure that DNS records have been updated to point the primary MX to the new server, and make sure that the old server is still configured to allow relaying for your domains. As long as both your old and new servers have up to date views of DNS, your mail service on the old server will work out it's no longer the primary MX and will forward any email on to the new primary. Mail will also deal with short delays in delivery - so you could just shut down or firewall all email access to both machines, move everything over at once, test it on the new machine, then move MX records and open up mail on the new machine. SMTP is designed to deal with outages, and any reasonably short outage (less than 4 hours I think) won't even generate temporary error notifications.

Updated to add:

Another possibility is to create a VM on your new server, and to rsync the entire old server over into this new VM's disk image. You can then fire up the VM and update IP addresses etc, and have a fully working replica of the original server inside this VM. Potentially a lot less effort, but if you're not familiar with VM stacks like KVM or Virtualbox, then it might not be worth it.