What makes a "legendary" legendary?

Legendaries represent a unique character, other cards represent a spell or a class/title. This is also why you can only have one of each legendary in your deck, but two of each of the rest of the cards.

There's only one Lord Jaraxxus and one Lorewalker Cho, their names are unique. The other cards have general names only describing them and/or their job, Bloodfen Raptor, Armorsmith and Angry Chicken for example.

3ventic's answer is the best one, but it's also worth noting that Legendaries are almost always better than an equivalent card that isn't legendary. By "equivalent" I mean class specific vs class specific, neutral vs neutral, of the same or similar mana. (Assuming there is a comparable card.)

For example: Nat Pangle may not be as effective as a Mana Tide Totem (50% vs 100%, draw at the start of your turn instead of the end), but he's better than any other neutral card-draw card. So there's less reason for a Shaman to take him, but for non-Shaman he can be a great card.

While 3ventic most definitely has the "correct" answer, there's a whole different side to this (at least, in my mind). A legendary is "legendary" because Blizzard said it was. By Blizzard telling us a card is legendary, that card now carries a certain amount of "clout" which brings the whole notion of psychology into the mix and that's immensely powerful.

For example, Nat Pagle may not be the best Legendary out there but Blizzard said he was legendary and therefore he scares your opponent. Your opponent is now going to blow something to get rid of Nat Pagle. This effectively gives Nat Pagle a pretty strong "soft" taunt.. and, again, that's super powerful. Even if your opponent ignores Nat, you still win because you will start to have the card advantage.

In short, a legendary is a legendary because they represent a unique character in the Hearthstone world. They are also legendary because Blizzard told us they were, and being legendary carries "clout" which makes them targets for spells, clears, etc which means one less spell/clear/etc for your bigger and better cards.