Gay (homosexual) and gay (happy)

When did the main meaning of the word 'gay' shift from happy to homosexual? How did the meaning evolve, if there is a relation between the two?

The development is actually quite straightforward once it has been pointed out.

As your question indicates that you already know, 'gay' originally meant 'carefree' and 'happy'.

Over time, the liberalness of 'carefree' increased and by the 1600s, it was used to describe someone of promiscuous sexual proclivities. Womanisers, prostitutes and other such sexually liberal people were termed 'gay'.

The idea of sexuality within the word continued to become more liberal and sexual promiscuity came to be sexual obscurity and eventually, by C20 (yes, that recent) it was used to refer to homosexuals.

Now, that meaning has developed even further and the word 'gay' has taken on a negative connotation (since many oppose homosexuality) and can now be used almost interchangeably with the word 'bad', in certain youth cultures. i.e.: "I don't want that one; it's gay"

The New Oxford American Dictionary has:

USAGE.   Gay meaning ‘homosexual,’ dating back to the 1930s (if not earlier), became established in the 1960s as the term preferred by homosexual men to describe themselves. It is now the standard accepted term throughout the English-speaking world. As a result, the centuries-old other senses of gay meaning either ‘carefree’ or ‘bright and showy,’ once common in speech and literature, are much less frequent. The word gay cannot be readily used unselfconsciously today in these older senses without sounding old-fashioned or arousing a sense of double entendre, despite concerted attempts by some to keep them alive. Gay in its modern sense typically refers to men ( lesbian being the standard term for homosexual women), but in some contexts it can be used of both men and women.

Etymonline has more to say:

The word gay by the 1890s had an overall tinge of promiscuity — a gay house was a brothel. The suggestion of immorality in the word can be traced back at least to the 1630s, if not to Chaucer: “But in oure bed he was so fressh and gay / Whan that he wolde han my bele chose.”

Slang meaning “homosexual” (adj.) begins to appear in psychological writing late 1940s, evidently picked up from gay slang and not always easily distinguished from the older sense. The association with (male) homosexuality likely got a boost from the term gay cat, used as far back as 1893 in Amer.Eng. for "young hobo," one who is new on the road, also one who sometimes does jobs.

The "Dictionary of American Slang" reports that gay (adj.) was used by homosexuals, among themselves, in this sense since at least 1920.

Gay as a noun meaning "a (usually male) homosexual" is attested from 1971; in M.E. it meant "excellent person, noble lady, gallant knight," also "something gay or bright; an ornament or badge" (c. 1400)

^{(many citations omitted; see the link above for the full text)}

For some personal research, Google ngram gives two interesting graphs:

Both graphs show a reversal in the trend of usage of gay in the late 1970s. Of course, those graphs are established from books, so written usage in other media and oral usage probably predate that in books.

Tom Robinson a British musician and singer, one of the first, if not the very first pop artist who did not disguise or hide his homosexuality from the general public. Until then homosexual actors, musicians, athletes or anyone in the public eye had two choices in life: to keep their homosexuality a secret, especially from the media, or adopt the mannerisms and a code of dress which were very camp but at the same time (ironically) never admitting their sexual preferences publicly.

What's the name of that amazingly camp actor with the high voice and a funny walk?

Tom Robinson released a very successful song entitled (Sing if you're) Glad to be Gay, it was the mid-70s, and I was still at a Catholic primary school when I remember quite distinctly hearing gay being used for the first time to counteract the BrEng derogatory terms such as: poofter, poof, queer, not normal, fairy and queen that were rife at the time.

Glad to be Gay

The song was originally written by Tom Robinson for a London gay pride parade in 1976, inspired by the directness and confrontational style of the Sex Pistols. An out gay singer, he subsequently formed the Tom Robinson Band with three straight musicians. [...]

"Glad to Be Gay" is built around four verses criticizing British society's attitudes towards gay people. In the first verse, it criticizes the British police for raiding gay pubs for no reason at all, once homosexuality had been decriminalized since the 1967 Sexual Offences Act. In the second verse, it points to the hypocrisy of Gay News being prosecuted for obscenity instead of porn magazines like magazines Playboy or the tabloid The Sun which publishes photographs of topless girls on Page 3. It also criticizes the way homosexual people are portrayed in other parts of the press, especially in conservative newspapers News of the World and Sunday Express. On the third verse, it points out the extreme consequences of homophobia, such as violence against LGBT people.

For me, a young child living in London at the time, the term gay (meaning homosexual and not "happy") was used much more frequently by the media and the general public after the release of "Glad to be Gay". It also helped that Tom Robinson looked straight and was not in the least camp, he did not fit the stereotypical idea of how homosexuals looked and behaved. The song was also incredibly catchy, upbeat, joyful and anyone with an iota of sensitivity in their body could understand that this song was saying to the world: Don't be ashamed of who you are, stand tall and be proud.