What happens when choosing Magicka, Health or Stamina when leveling up in Skyrim?

Almost nothing. That choice basically is the new replacement for the old attributes in the previous games.

The developers said that the old attributes were basically just a fancy way to raise or lower Health, Magicka or Stamina (besides also influencing skills, obviously), so they went for this "choose between the three" plus "perks" system.

The only thing that changes is the amount you can carry. As Arkvive pointed out, choosing stamina will increase your maximum carry capacity by 5.

Choosing Stamina increases not only your availiable Stamina pool, but also your carrying capacity.

Additionally, whichever attribute you choose will be refilled to full. This means that you can 'save' a level up and apply the points in combat as a sort of poor mans potion.