Server migration, minimize wrong DNS lookups
I run a large website (500k uniques a day) and am about to move to another hoster.
My new machine is setup, and tested, all files are copied over, so I am almost ready to change the my domain's IP address at my registrar.
Now I would like to know if there is a away to minimize the amount of people hitting the old server, because their DNS information isn't updated yet.
Sometimes it can take ages to update, and people hitting the old server would bring my website out of sync.
Is there a way to force forward people to the new machine from my old machine?
No, well yes, but in reality no.
Set your DNS TTL really low before the migration (like 5 minutes), this tells clients to only cache the DNS for 5 minutes then refresh. In theory, after you change the IP over in your DNS it should only take 5 minutes for the clients to start hitting the new server IP.
Unfortunately, the theory isn't reality. Some ISPs and DNS providers cache records longer than the TTL set (I've seen some ISPs cache TTLs of 5 minutes for 48 hours) and, in short, there's absolutely nothing you can do from a technical perspective to stop them doing this, even though they shouldn't. And alas persuading all your users to move to OpenDNS might not be the best idea.
When I've moved larger sites before this is generally the process I follow;
Set up synchronization between the two (new and old) database servers.
If the database your using supports master-master replication (I.E. writes to ether node will be propagated to the other), run both the old server and the new server along side each other until all the clients have updated. This means clients can hit ether server and the site will be fully functional.
If the database only supports master-slave/log shipping etc. your only real option to keep the site up is to have the old server running a "read only" copy of the database, it would still have the latest data but only to read, not write/update. Depending on your site this might not be too much of an issue.
Another option, and probably the easiest to achieve, is put a proxy on the old server that forwards any requests to the new server. Users on the old server will experience some latency due to the extra hops in the proxy but with a clever caching configuration you can probably minimize that.
With any of the above options, monitor the old server and when all/most of the clients have gone decommission it as you would normally.
Of course, all of this could be avoided if everyone followed the standards they were supposed to.
Thought of two tiny addition to sam's: If database access can tolerate latency between the two servers, you can also set then both to use the database on the new machine. (If you need security you can use SSH tunnel or VPN).
Another option would be to set old machine to answer to all queries with temporary HTTP redirects (307) to the new IP (or you can set-up some temporary domain name like www1.yoursite.com with the new IP, and use that in redirects).
Use a Reverse Proxy
If you have root access to both system, you can proxy traffic from the old server to the new one using Apache's mod_proxy.
With mod_proxy, you can setup a reverse proxy from the old sever to the new one. This way client's activites all happen on the same server. When scheduled carefully, you can have minimal downtime (e.g. the time it takes to restart apache).
I like this approach as it lets you test things before changing DNS. A good last minute sanity check.
You may also have to use a module called mod_rpaf to get the visitors true IP address if you have tools that require that information.
Some sites use canonical URLs which can cause an issue. One trick is to set the new server's IP in your /etc/hosts file on the old server. You can then put something like this in your proxy setup:
ProxyPass / http://www.domain.com/
ProxyPassReverse / http://www.domain.com/
That's it. You can do this for HTTPS as well.
Also note that many ISPs ignore TTL values. Comcast, ATT and others will refresh their DNS caches only a few times a day.