Are there downsides to saving my XBOX Live profile to USB?

I have my profile stored on a USB flash drive and I have not noticed any slowdowns due to this. I also use it to move my profile easily between my xbox and a friend's xbox without needing an expensive memory unit. I recommend it.

Without knowing for an absolute fact, I don't think there would be any slowdown. The Xbox should only need to access your profile saved on the drive when you sign in or to save an achievement you just unlocked. And even then, it wouldn't necessarily be any slower than a hard drive.

If the USB drive becomes damaged or corrupt, you could always re-recover your profile from Xbox Live to another hard drive or usb device. The only thing you would theoretically lose is any achievements you earned while offline without having connected to Live after your earned them but before your profile was damaged. Achievements earned online would still be saved; those earned offline get recorded on Xbox Live once you connect again.

Game saves are another issue, if they're only on the USB drive then they'd be gone forever. Most (all?) games ask you what device you want to save to the first time, you could always just keep game saves on each respective Xbox's hard drive if you're concerned.