How do I become a Thane?

If you want to buy a house in Skyrim, you need to become a Thane.

Whiterun's Thanehood comes with the first parts of the main quest and is relatively straight forward to get, but I've been having trouble attaining Thanehood in the other cities.

How do I become a Thane in Solitude, Windhelm, Riften, and Markarth? Are there specific quests that need to be done to earn it? (Like in Whiterun?)

In general, getting the title thane bestowed upon you follows a set strategy:

1. Do an introductory quest-line that winds up getting you noticed by the Jarl of the hold.
2. Complete a number of miscellaneous objectives within the hold (5 if a house is available, 3 otherwise)
3. Purchase a house in the hold's capital (if available)

The Eastmarch (Windhelm)

1. Capture Whiterun and Falkreath (if siding with the Stormcloaks) or capture Windhelm and complete "Kill the Bandit Leader" (if siding with the Imperials) to get into the Jarl's good graces
2. Complete 5 miscellaneous objectives within the hold
3. Purchase Hjerim, the house in Windhelm

Falkreath

1. Complete "Rare Gifts" (if given) and "Kill the Bandit Leader" to get into the Jarl's good graces
2. Complete 3 miscellaneous objectives within the hold

Haafingar (Solitude)

1. Complete "The Wolf Queen Awakened" (possibly optional) and "Elisif's Tribute", which gets you into the Jarl's good graces
2. Complete 5 miscellaneous objectives within the hold
3. Purchase Proudspire Manor, the house in Solitude

Hjaalmarch (Morthal)

1. Complete "Laid to Rest" to get into the Jarl's good graces
2. Complete 5 miscellaneous objectives within the hold

The Pale (Dawnstar)

1. Complete "Waking Nightmare" to get into the Jarl's good graces
2. Complete 3 miscellaneous objectives within the hold

The Reach (Markarth)

1. Complete "Dungeon Delving" (different from the Winterhold quest of the same name) to get into the Jarl's good graces
2. Complete 5 miscellanous objectives within the hold
3. Purchase Vlindrel Hall, the house in Markarth

The Rift (Riften)

My complete walkthrough for this is contained in another answer about how to get the house in Riften, but since that's the final requirement for obtaining the title, it's exactly the same. In short:

1. Handle Riften's drug problem by completing the quest chain that starts with Wujeeta
2. Complete 5 miscellaneous objectives in the Riften (can overlap with step 1)
3. Purchase Honeyside, the house in Riften

Whiterun

You've already mentioned how to become thane here, but for the sake of completeness:

1. Travel to Whiterun as part of the main questline
2. Complete the main questline up to and including "Dragon Rising"

Winterhold

1. Complete "Dungeon Delving" (different from the Reach quest of the same name) to get into the Jarl's good graces
2. Complete 3 miscellaneous objectives within the hold

Yes, there is a quest (or series of quests) in each major hold that you perform to becoming a thane.

Morthal

From Wikia's article on Becoming Thane:

Becoming Thane of Morthal is fairly straightforward. First, you investigate the house that burned down and complete the subsequent quests. The grateful Jarl will tell you that she'd be happy to make you Thane as long as you are "known through her Hold." This translates into helping three different people, all of which can be found in the limited square mileage of Morthal.

Markarth

From IGN's article on Becoming Thane:

The politics of Markarth are quite complicated, and thus becoming Thane is as well.

One way is to complete the Forsworn Conspiracy by killing Mad King Madanach - the Jarl will thank you for handling him, and give you a quest to retrieve Hrolfdir's Shield. Once you've done that, the Jarl rewards you by allowing you to purchase property. He then adds a followup quest to become Thane of Markarth -- own a house and help 5 citizens of Markarth. After all of this, he will finally make you a Thane, and give you the Blade of the Reach.

Solitude

From IGN:

Talk to Falk Firebeard in the Blue Palace, and he will give you a quest summoning you to investigate lights and sounds coming out of Wolf Skull Cave which is located in the mountains west of Solitude. After completing the quest, talk to Falk again and he will reward you.

Sometime soon you will receive a letter from a courier asking you to speak to Falk once more. (It is unknown if you can go talk to Falk immediately after and simply force him to give you the next part of the quest).

Falk will give you another quest. Once this quest is done, you may speak directly to the Jarl herself, and she will thank you and give you one more task.

From Wikia:

After you have completed this task go back and talk to the Jarl. She will invite you to be a thane of hers but in order to be a Thane, you must first own property in Solitude. This property can be purchased from Falk for 25000 gold pieces. Once you buy the property, go and talk to her, and she will appoint you to be a Thane of Solitude.

Falkreath

From Wikia:

This Hold is as straightforward as Morthal: obtain a drink for the Jarl, then kill the assigned Bandit Leader. Complete three quests for locals and you're set. This is the standard formula for the smaller Holds (that is, Holds like Morthal, Windhelm, and Falkreath).

Riften

From Wikia:

Connected to the Theives Guild Quests.

Windhelm

From Wikia:

The objectives you need to complete here are as simple as is Morthal. Obtain the Helm for the Jarl, and then help out three people in his Hold, and he'll see fit to make you Thane.

Dawnstar

From IGN:

Connected to the Dark Brotherhood Quests.

Winterhold

Complete all the College of Winterhold quests and you will be made into the archmage of the college.