Will a restore to default factory settings remove the firmware updates from my Iphone?

Restoring to factory setting will simply wipe out every bit of customization you did to it (every setting you changed from the default) but it will not downgrade the firmware.

In other words, if you bought your iPhone with iOS 3 and now you're on iOS 4, then a factory setting reset will still leave you on iOS 4.