Making hosted applications resilient to BGP failures

You're unlikely to be able to work around this, unless your address space is large enough for you to be able to run your own BGP. Even then, you're vulnerable to BGP failures by your peers.

If you're using multiple DNS servers in separate ASes, you may be able to have some kind of work around by setting a low TTL and failing over to a separate web server in a different netblock/data centre by changing DNS once problems are noted. Even this will take several minutes however, at the least.

EDIT: as pointed out by Chris, if you're running BGP, you need all of your peers to fail before you become unreachable.

You're unlikely to be able to run BGP unless you have at least a /23 of Provider Independant address space and have an ASN number. As such, you need to trust your hosting company. Router changes tend to be fairly rare, so the likelihood of this problem happening again is slim. You could investigate any SLA you have with them, but this is probably just going to involve getting a refund on your hosting fees.

As far as monitoring is concerned, we have a dedicated server outside our network, which we use as an external Nagios server. You could buy a cheap VPS server and use that to monitor things from the PoV of an external user. For example, we check SMTP and HTTP work, rather than checking that exim and apache are running, which we do on our internal monitoring.

For the record, it exists several gratis BGP monitor and alarm systems. None provide a resolution of 15 mn as you want. And, since you can have many other causes of outage, monitoring the IP connectivity from the outside is the only real solution.

• Cyclops
• RIPE IS
• BGPmon, my favorite

A general article about BGP monitoring, in french.