How does choosing Imperials or Stormcloaks affect me later in the game?

The choice between Stormcloaks and the Imperials is supposed to provide a little more immersion in the story: you choose a side, and while both sides have comparable quests, they take on a different flavor and will have different actors.

To this end, both the Imperials and the Stormcloaks have and 11-part quest chain concerning the Civil War that ravages Skyrim. The first few quests are almost identical, but then they branch off as each side wins (or loses) major fronts in the war.

And in general, choosing one side or the other makes things easier for you if the side you chose is in power.

For example, if your side controls a major city, you generally don't have to do any special favors for the Jarl in order to get permission to buy a house: it'll be given as a gift for siding the cause.

But the Elder Scrolls series is all about choice, and you're not going to be limited much by what side you choose. Choosing the side you most identify with will generally work out just fine.

But fear not, you didn't make a permanent choice when you went with Ralof (the rebel). You'll be able to play both sides until the last possible moment. You don't seal the deal until you swear an oath to either side by going to their stronghold city.

Additionally, there's one more chance to switch sides during the civil war:

During The Jagged Crown, bringing the crown to the opposing side's leader will let you turncoat and join the other side.

If you're choosing the stormcloaks to ally with be sure to do the main quest "alduins wall" before fully committing to them, otherwise Delphine of the blades will become your enemy thus making the "alduins wall" quest impossible to complete. I learned this the hard way.