How do I find an ore mine?

I have an urge to practice some smithing, but ore seems to be in short supply. How do I locate a mine? Will people be able to tell me where one is if I talk to the right ones? Are there certain terrain features that I should look for? Or is it just a matter of wandering the world map long enough to randomly locate one?

Like Cloudancer said, one of the first mines in the game is to the east of the Guardian Stones. If you fast travel there, you can look for the "Mine" icon in your compass:

Follow that, and it should lead you directly to the Embershard Mine. Once inside, you'll need to look for veins of ore. Look for any sort of discoloration in the walls, then walk up and see if the tooltip says that you're looking at ore. Here's what an Iron Ore vein can look like:

The easiest mine to find very early in the game is Embershard mine. If you head northwest for Helgen Keep you will find the Guardian Stones by the road (they are 3 tall stones with holes in them). If you head a little east from there you will hit Embershard mine. There is a bandit outside that should help make it obvious. There are also pick's inside the mine you can grab to mine with (after you kill the first bandits).

Beyond the first mine (above), here are the other locations for ore (or for dwarves, scrap) that I know about:

Iron: Iron Breaker Mine (Dawnstar), Left Hand Mine (Markarth), Redbelly Mine (Shor's Stone), Rockwallow Mine (Stonehills), Whistling Mine, Halted Stream Camp (Whiterun Hold, north east of town)

Orichalcum: Dushnik Mine (Dushnik Yal), Bilegulch Mine, Mar Khazgur Silver Cidna Mine (Markorth), Fenn's Gulch Mine (Karthwasten), Sanuaroch Mine (Karthwasten)

Corundum: Darkwater Crossing, Goldenrock Mine (Darkwater crossing, EastMarch)

Ebony: Gloombound Mine

Gold: Kolskeggr Mine

Moonstone: Soljund's Sinkhole

Malachite: Steamscorch Mine (Just north of Kynesgrove, safe)

Quicksilver: Quicksilver (Dawnstar)

Dwarven Scrap: Mzultf (EastMarch)

I will update as I find more locations. Feel free to edit your own in and/or comment and I can pull them in.

[Source: Game exploration, Official Strategy Guide (book), assorted]