Why "lemon" for a faulty or defective item?
Solution 1:
Evan Morris over at The Word Detective, answering a similar query, has some helpful musings.
He argues that despite all the good lemons have done, they've suffered from an image problem since the dawn of their cultivation—due primarily to their stinging acidity and tough skins.
He continues,
The word “lemon” comes to us from the Old French “limon,” which was derived from Arabic roots and served as a generic term for citrus fruit in general (which explains how the same root could also give us “lime”). The use of “lemon” to mean “disappointing result” or “something unwanted” is very old, reflecting the fact that, while useful in cooking, a lemon standing alone is just a lump of sourness with a tough skin to boot. In Shakespeare’s play Love’s Labours Lost (1598), for instance, one character proclaims, “The armipotent Mars, of lances the almighty, Gave Hector a gift …,” to which another puckishly suggests, “A lemon.”
And, clearly drawing from some of the OED citations mentioned by @Barrie, he concludes,
In the mid-19th century, “lemon” was used as a colloquial term for a person of a “tart” disposition, as well as, more significantly for our purposes, slang for a “sucker” or “loser,” a dim person easily taken advantage of. It has been suggested that this latter use stems from the idea that it is easy to “suck or squeeze the juice out of” such a person (“I don’t know why it is, rich men’s sons are always the worst lemons in creation,” P.G. Wodehouse, 1931). By 1909, “lemon” was also firmly established in American slang as a term for “something worthless,” especially a broken or useless item fobbed off on an unsuspecting customer.
It’s likely that the current use of “lemon” to mean “something that doesn’t live up to its billing” or “a disappointing purchase” comes from a combination of “lemon” in the “sucker” sense (i.e., the buyer got “taken”) and the much older sense of “lemon” meaning “something undesirable.”
Also of note, I found occasional use of the phrase (at least as early as 1918), "to give someone a lemon and pass it off as a nugget (of gold)." If this was the original saying, later shortened to "handing someone a lemon," then the implication of trickery is confirmed and the metaphorical use of lemon further explained.
Solution 2:
I note the reference to Shakespeare's Love's Labour's Lost, originally flagged by Evan Morris over at The Word Detective and referenced by Callithumpian here. We are given to understand that Hector is given a gift, which is then described disparagingly as a 'lemon'. I think Morris is on to something, but it's more subtle than just a matter of 'sourness' as suggested above. Take the full exchange from Love's Labour's Lost (According to Pauline Kiernan in her 'Filthy Shakespeare', a play notorious for its word play):
*Arm. The armipotent Mars, of lances the mighty, Gave Hector a gift, -
Dum. A gift nutmeg
Biron. A lemon
Long. Stuck with cloves.
Dum. No cloven.*
Nutmegs in Shakespeare's time were astronomically high value items, and given as gifts. Sometimes this is written as a 'gilt' nutmeg. This doesn't change the essential context of Lord Dumaine's remark, he's referring to a high value item. Biron counters by suggesting that the gift is not a nutmeg, but a lemon. Shakespeare at this point is - I'd argue - not using 'lemon' as a metaphor for 'a sour surprise', but rather as a 'cheaper' substitute for the nutmeg. A lemon stuck with cloves (the sharp cloves 'stabbed' into the lemon skin) was a common New Year's gift, something that smelled pleasant and was said to ward off disease. It may also be a reference to the fruit of the nutmeg, which is not unlike a lemon. The rejoinder from Lord Dumaine, 'No, cloven' might be a coded reference to the tendency of the nutmeg fruit to split and reveal a bright red interior, a visual metaphor that I'll leave alone.
I am inclined to the view that Shakespeare's relatively obscure reference has worked it's influence on the English usage of the term 'lemon' as a 'poorer substitute for the real thing', and that this usage has been reinforced over time by the association with the inherent sourness and bad taste of lemons.
Solution 3:
The earliest figurative use of the word, in 1863, was, in the OED’s words, as
a person with a tart or snappy disposition. More usually (slang), a simpleton, a loser; a person easily deluded or taken advantage of.
The OED’s earliest citation showing it used as
something which is bad or undesirable or which fails to meet one's expectations
is dated 1909, and in this use it is of US origin. There is also a citation dated three years earlier showing its use in the expression 'to hand (someone) a lemon’ meaning
to pass off a sub-standard article as good; to swindle (a person), to do (someone) down.’
I suppose it’s not too unlikely a progression from being used to describe a person who is in some way unsatisfactory to being used to describe a thing that is is in some way unsatisfactory.
Solution 4:
Lemon is the prototype of a sour fruit in the USA. The facial grimace associated with an unexpected strong sour taste is also associated with strong negative emotion, like grief or sorrow.
- After she told him, he looked like he was sucking on a lemon.
So, metaphorically, lemon gets to be the mythological symbol of bad fortune, at least in the USA.
Solution 5:
J.E. Lighter, The Random House Historical Dictionary of American Slang, volume 2 (1997), gives a first occurrence date of 1906 for lemon in the relevant sense:
lemon n. ... 2.a. A disappointment; (hence) anything considered as worthless, unworkable, fraudulent, defective, etc.; (esp.) a defective automobile; an unprofitable prospect.—in early use usu. in phr. hand (someone) a lemon to cheat someone.—occ[asionally] used attrib[utively]. Now S[tandard] E[nglish].
[First cited occurrence:] 1906 Nat[ional] Police Gaz[ette] (June 2) 10: {Boxer} Marvin Hart is till trying to square himself for the lemon he handed to the Madison Square Garden [New York City] promoters when he met Mike Schreck.
[Second cited occurrence:] 1906 H. Green Actors' Boarding House 36: Him gettin' handed a lemon in that English act.
A search of the Library of Congress's Chronicling America database of historical newspapers yields three instances that are slightly earlier than the first one in Lighter. From Marvin Green, "The Man Higher Up," in the [New York] Evening World (November 18, 1905):
"Do you think that J. Pierpont Morgan ever said 'what's the use' when he thought somebody had handed him a lemon? If he had cultivated the habit regarding imposition as humorous play in life he would probably be a curb broker to-day, getting his name in the papers once a year by betting stage money on the election."
From Left Cross, "More on the Passing of That Grand Old Man—Fitz," a story datelined "New York," in the Richmond [Virginia] Times Dispatch (December 31, 1905):
Fitz [boxer Bob Fitzsimmons] was always suspicious in a small way, and this made him an easy mark for the really clever. That's why he isn't a rich man now, as he ought to be. He has earned enough. But he has had the lemon handed him pretty often. All he got out of a $40,000 purse for licking Jim Hall was a phoney check. Ed Dunkhurst was put up at the Hercules Club in Brooklyn as a punching bag for Fitz. Bob disposed of the ton of lard at a given signal in the second round, and Dunkhurst got $200 more out of the affair than did the real fighter.
And from "Roy L. McCardell's 'Chorus Girl'," in the [New York] Evening World (January 6, 1906):
"I don't see why Mr. Maginnis, of Marietta, wasn't appointed police commissioner," said the Chorus Girl. "He isn't hep to anything, lives far away from New York and has always been anti-Tammany. And yet they handed him the lemon on his offer to take the job. Say, wouldn't he made the fine commish—him with that When-the-Robins-Nest-Again bunch on his face?"
The remainder of 1906 sees about a dozen additional instances reported in newspapers from New York (where Martin Green seems to have launched a regular Evening World column called "The Order of the Lemon") to Washington, D.C. to Fairmont, West Virginia to Minneapolis, Minnesota to Roswell, New Mexico, to Salt Lake City, Utah to Los Angeles to Walla Walla, Washington.
This jokey item appears in the Minneapolis Journal (November 25, 1906):
Henry James, the novelist, says the expression "handing a lemon" conveys no meaning. If this is really true, the inquiry naturally and inevitably arises, when did Mr. James originate it?
By 1907, the expression was so familiar nationally that William Jennings Bryan's newspaper, The Commoner, published in Lincoln, Nebraska, uses the saying to set up a punchline about Republican tariff policies. From The Commoner (February 1, 1907):
The Boston Herald quotes a fruit dealer as saying that the popular slang expression "handed him a lemon," is hurting the lemon trade. "People who formerly had no hesitancy in asking for a lemon," says the Herald, "now pass on and buy some other variety." In view of this the Herald asks for something new—something that will relieve the lemon trade and incidentally provide something new in the way of slang. Because of our sympathy for the lemon dealers, and also because of our desire to be of service to the esteemed Boston Herald, we suggest the phrase "handed him some republican tariff revision." It conveys the same idea as "handed him a lemon," and at the same time is very expressive of a true state of affairs.
The newspaper evidence indicates that the expression "handed [someone] a lemon" caught on in a narrow window of time between late 1905 and early 1907. The earliest instances of the expression are connected to New York City, and several of the early instances involve boxing. Beyond that, it's difficult to say why being "handed a lemon" became synonymous (in slang) with being cheated.
The Etymonline assertion (cited in the OP's question) that "to hand someone a lemon was British slang (1906) for 'to pass off a sub-standard article as a good one'" is simply not supported by contemporaneous evidence. Consider, for example, this excerpt from Ford Madox Ford, Ring for Nancy: A Sheer Comedy (1913), which is set in England and involves an American woman (Miss Olympia Peabody) and an Irishman (Major Edward Foster):
[Miss Peabody speaking:] "... You are fond, for instance of saying that some one handed you a lemon ..."
"Oh, but hang it all," the major interrupted, "that's a Americanism; I only use Americanisms now and then to make you feel comfortable and home-like. Personally I detest them. To hand any one a lemon means ..."
"I am perfectly well aware what the phrase means," Miss Peabody said. "It signifies what in English we should term a rebuff or a slap in the face. But you must consider, that to a person not in the least knowing what the phrase may signify, and casting about in his or her mind for an allegorical meaning—to such a person—supposing you should use the phrase, 'She handed him a lemon and he quit,' as you are fond of doing when you desire to be amusing—to such a person the words might seem to connote a reference to the fall of man when Eve handed Adam an apple—which, however, was a fruit more exactly resembling a lemon and not an apple at all—and our first parents were forced to leave the Garden of Eden; and as you are aware, to many old-fashioned people, any reference to Holy Writ is apt to be considered not only blasphemous, but even on this particular case possibly obscene."
"But I never said anything about handing anyone a lemon," the major said. "I shouldn't among English people. They don't like your American slang. ..."
As an English author, Ford was well positioned in 1913 to know whether "hand someone a lemon" was American English slang or British English slang.
Conclusion
The search results in Google Books and in the Library of Congress's newspaper database tend to confirm the view of J.E. Lighter that lemon in the sense of disappointment originated in the United States, perhaps in the expression "hand [someone] a lemon." Instances of this usage begin in U.S. newspapers in late 1905 and it seems to have become familiar across the nation within 18 months.
As other answerers have observed, the sourness of lemons naturally suggests an association with experiences that leave a sour taste in one's mouth, but I can't find any instances before 1905 where being handed a lemon is treated as metaphorically equivalent to being cheated or given a raw deal.
Update (10/31/2017): 'hand [someone] a lemon' in boxing parlance
In a comment beneath this example, Brian Drummond asks, "was it the custom at the end of a round [of a boxing match] to hand some refreshing drink or fruit to the boxer?" I don't have a clear answer to this question—but here is an interesting item involving what may be such a practice—with a twist—from "News of the Day," in the Alexandria [Virginia] Gazette (My 23, 1878):
Barney Trainor and Dip Kelley fought Tuesday for $200 a side, near Philadelphia. After five rounds of terrible fighting, and just before the call of time for the sixth round, a friend of Trainer's handed him a lemon. He filled his mouth with tho juice and toed the scratch. When Kelley approached him he squirted the juice in his eyes and then struck him savagely in the mouth. At this the ropes were torn down, and the crowd entering the ring, separated the bruisers. Kelley was given the stakes and all hands made haste back to town.
The report doesn't indicate whether Trainor's receiving a lemon between rounds of a prize fight doesn't struck anyone in attendance as odd, so perhaps swallowing lemon juice between rounds was a common practice. But if so, it is still unclear whether the point of it was to refresh the fighter, sharpen his senses after having been pummeled for a round, or serve some other purpose.