Is there a way to replace the web browser in steam?

Solution 1:

Xfire and PlayXpert offer web browsers as part of their in-game overlays. There is also the Rogue in-game browser. Xfire offer a preview video of their web browser. All look to have tabbed browsing and offer more features than Steam's, though it's hard for me to say whether they meet your expectations.

Both Xfire and PlayXpert also ship with chat clients.

Raptr's in-game overlay supports AIM and I believe XChat makes available an addon to that end also. Personally I'd try Raptr first. Any of the web browsers may also support AIM Express, an official browser-based AIM client.

Solution 2:

Steam uses a webkit-based browser, the same engine that is behind Chrome and Safari. It was chosen above Trident (IE technology) because of crossplatform compatibility (= it runs mostly unchanged on Mac OS X). At best of my knowledge, this cannot be changed.

Overlay chat is already available through Steam's social features. Although it's not very convenient, you can get access to many chat networks through webapps; for example, x.aim.com.