Why can't some skins be purchased in the shop?

I saw that many skins can't be found or bought in League of Legends' shop. Why?

Some skins were for events, such as the Winter Olympic skins that were only purchasable during the 2010 Winter Olympics. Others were given for helping Riot out, such as "King Rammus," which was given to beta testers. I don't have a complete list, but I believe that covers most of the unbuyable skins.

Some skins were available only in Collector's Edition:

• Black Alistar
• Human Ryze

Other skins were available only in some Special Events:

• Riot Squad Singed
• PAX Jax
• PAX Twisted Fate
• King Rammus
• UFO Corki
• Victorious Jarvan
• Judgement Kayle

Finally some skins are now known as Legacy, they were available in store but are now not purchasable anymore.

The list is quite long and is maintained here on League of Legends Wikia.