Exchange Server vs. IMAP

If you are the only user, and you don't plan on that changing anytime soon, then I think you would be well served with an imap account.

Hosted Exchange is great, I use in for our company of 18 people.

You might want to change out Google's offering. For free you can setup an account with up to 50 users, each with their own 7 GB (or 8 - I can't remember) mailbox. They offer full imap support and great spam filtering. You would still need to point your MX records to another location, but that really should not be a big hasle on a ongoing basis.

The big thing you lose is the ability to use the full functionality of Outlook.

An IMAP server can't host Contact, Task, Note, or Calendar items for Outlook. You'll have to have a PST file for those items and then you'll need to worry about keeping that backed-up.

If you're doing using any of the shared-calendaring functions of Outlook you'll be losing them, too.