Azure Virtual Machines - what fault tolerance do they provide?

Virtual machine vhd's are backed by Windows Azure Storage. To be more precise, page blobs. A blob is triple-replicated within the data center itself, then georeplicated to a neighboring data center (unless you opt out of georeplication). In the US, those pairs are East<-->West and North Central <--> South Central. In Europe, it's Dublin <--> Amsterdam, and in Asia it's Hong Kong <-->Singapore.

Storage SLA (detailed here): 99.9% availability.

Regarding backups: It's trivial to create a copy of a blob. Up until recently, the API only supported blob-copy within the same storage account (ergo same data center). With the Spring 2012 release, the API was updated, and now supports cross-account blob copy (so you can very easily make periodic backups anywhere you want). Announcement and full description here.

I'm not saying you need to back up your vhd's. It's up to you. Given the relatively inexpensive cost of storage (full details here): Assuming a 50GB vhd, that would run you under $3 monthly (or less than $2 if you turned off georeplication). Having a backup somewhere else would just add another few bucks to your monthly storage cost.

Regarding your 2nd question: First let me point out an article that Michael Washam posted, with a detailed overview of Virtual Machines. This article hopefully answers some of your questions. Let me call out a few items.

All hardware and related infrastructure support is taken care of: Front-end load balancer, hardware health, Host OS updates, availability sets (when load-balancing across multiple VMs), etc. Note that, in that article, there's mention of a new Single-instance SLA of 99.9%. So, while you will have a short period of downtime if only running one VM, the Windows Azure fabric will bring your VM back up fairly quickly if, say, the hardware running your Guest VM fails. To avoid single-instance downtime, consider an Availability Set, where you can load-balance an endpoint across multiple Virtual Machines.