Surviving and thriving as a low level thief?

Core Skills

If you start with pure stealth and neglect your weapons/armor, you're not going to survive long - your stealth-hit damage is controlled by your base weapon damage, and without a high enough base weapon damage and some armor for survival, you're likely to be toast quickly. At low levels stealth is not terribly effective at hiding you, either.

Instead, you want to start by getting your 1-handed, archery, and light armor skills up. This isn't as challenging as it seems, since at early levels enemies aren't that difficult to dispatch. Draugr crypts are good for this sort of thing - once you know how to spot "active" Draugr, you can hit them before they wake to get a very high damage guaranteed stealth hit, which levels your Stealth and weapon skills very quickly.

Complimentary Skills

I also highly suggest that every thief prioritizes learning Alchemy, as being able to heal yourself cheaply is very important. Poisons and other alchemy results (ie, invisibility) are also critical. There are abundant ingredients in the fields around Whiterun that should get you started along this path. You can also rob the alchemists in towns to build up your potion and ingredient stocks, which will offset your low health in battle.

As I mentioned, your damage is based on your weapon damage, so being able to up that cheaply with blacksmithing is also a clear plus. Any time you can level this skill, you should think about doing so.

Illusion magic can make you more stealthy by muffling your footsteps and eventually making you completely invisible. Conjuration magic is also a good one to level, as you can summon Atronachs to tank/distract for you. The college at Winterhold is your one-stop shop for everything magical.

Early-Game Strategy

You can join the Companions in Whiterun early, which will give you the ability to take warrior "tanks" along for the ride in many early missions. Their sneak skill is poor, so they will tend to blow your cover if you try to do these as a thief at higher levels. At low level, they're probably around your skill level, so it's less of a disadvantage. Since you're not likely to care about the quest rewards, you can start this early and use it as a stepping stone to higher levels and better skills.

Honestly, your first dragon fight isn't that hard - you can mostly hide in the shadows and plink away the dragon's health with arrows and so forth while the Whiterun soldiers do most of the tanking. I'd almost suggest going and getting it over with, and even going to High Hrothgar early - dragons are much easier once you have the shout you learn by following the main quest.

A few levels in...

Once you've gotten a few levels under your belt, head over to Riften and begin the Thieves' Guild questline. I don't recommend completing it until you're pretty significantly leveled (the quest rewards are level dependent), but you can at least start it to get some base gear for efficient thieving. The vendors and quest givers in the guild hideout are also quite useful for low-level thieves.

From here you've got a lot of options, and what you tackle next mainly depends on what strikes your fancy.

You can start The Companions questline to gain the ability to become a werewolf. When you need to fight, just use the power and transform; kind of like having two game plays in one character.