Where is the first available house in Skyrim?

As I grabbed everything that wasn't nailed down so far and as I don't want to throw away any ingredients I could use later I though it would be time to look for a house to store all my stuff. My inventory is getting fuller and I'd like to deposit all my crafting ingredients in a safe place.

How can I get a house to store all my stuff the easiest way at the beginning of the game?

Following the main quest a bit further, I stumbled upon an opportunity to buy a house in Whiterun. You have to complete the Dragonstone quest (Bleak Falls Barrow), then you'll be granted the right to buy a house in the city.

To buy the house you have to speak to Proventus Avenicci, he's near the Jarl in the keep. Unfortunately the house is pretty expensive for a starting character, it costs 5000 gold.

The house itself is called Breezehome, it is the second house on the right when you enter Whiterun.

The first house available in the game is in White run. It's a great house to have, cheapest and actually probably the most useful.

White run is by far the best town for selling, crafting, etc, in other words a place to quickly pop over to in between dungeons, quests, or anytime you get a full load of loot.

The house is literally like 15 steps from the fast travel location. And next door on either side are the blacksmith and general goods store, as well as the potion shop.

This blacksmith (war maidens) is fully stocked with smelter, tanning rack, workbench, grindstone, and of course the forge. Shopkeeper restocks with more than a thousand gold each day.

It's perfect, you travel in, stop at shops very quickly to offload loot and your house for anything you wish to keep. Then whenever you need to you can do all your crafting very conveniently.

Also travelling to dragons reach keep allows you to conveniently use an enchanter, and the wizard there is stocked with all sorts of spells and soul gems as well.

I'm pretty far in the game, having completed many major quest lines and visited all the cities, and I can say that this house is by far the most useful.

That being said, it's nothing to look at and for aesthetic reasons it's probably the worst! But as a storage location it's the best.

Although I do wish there were more chests to store things in, to keep them sorted better.

If you're referring to the house in Whiterun, it's called Breezehome, and it's right next to the smithy when you enter town. (Zoning in from the main Skyrim map, it's the second building on the right.)

You can get a free house with more storage available than any house you can purchase in the game(not sure how it compares to the amount of storage you can create in certain configurations by building with the Hearthfire DLC).

If you start the quest House of Horrors( just start it, you don't have to finish the quest right away or at all) you gain access to the Abandoned House in Markarth. Just talk to Vigilant Tyranus outside the house to start the quest.

Like Breezehome, you have access to all smithing stations(except a wood chopping block if you have Dawnguard and want to make arrows or crossbow bolts), 2 pre existing alchemy labs(apothecary and court mage) and an arcane enchanter(again next to the court mage). It even has a second smith in town like Whiterun( Moth Gro-Bogol in Understone Keep), but you cannot invest in either him or his sister Ghorza Gra-Bagol who runs the forge in town. There is also no fletcher in Markarth like there is in Whiterun(Drunken Huntsman) but the house is free.

If you have the Dragonborn DLC, you can earn a free house(Severin Manor) by completing a couple quests. It has all crafting stations inside(save a wood chopping block of which there is at least one in town next to the smelter by Raven Rock Mine). It is however somewhat lacking in the storage department.

If you intend to make full use of all you crafting skills(including the frequent changing of powers from the black book Sallow Regent and wearing the Relics of Ahzidal while enchanting) it can be handy to have this house as a place to store crafting supplies(mostly alchemy and enchanting for me, as I keep Seeker of Might active most of the time and do most of my tempering at Lakeview Manor on a hill overlooking Lake Illinalta in Skyrim).