How did "biscuit" come to have a distinct meaning in North American English?
The Oxford Living Dictionary makes a clear distinction between the usage of biscuit in Britain and North America:
British: A small baked unleavened cake, typically crisp, flat, and sweet. ‘a chocolate biscuit’
North American: A small, soft round cake like a scone.
When I hear biscuit being used in a British context, it usually refers to a confection (what Americans call cookies). In a U.S. context, it never refers to a confection. [See also: Biscuits & Gravy]
Why did this distinction come into being?
Solution 1:
A look at early (pre-1800) English dictionaries points to a possible source of confusion early in the career of biscuit. Two dictionaries—Edward Phillips & John Kersey, The New World of Words: Or, Universal English Dictionary (1706) and John Kersey, Dictionarium Anglo-Britannicum: or, a General English Dictionary (1708)—have identical definitions for biscotin:
Biscotin, (F.) a sort of Confection made of fine Flower, the Whites of Eggs, Powder-Sugar, &c.
But John Kersey, A New English Dictionary; or, A Compleat Collection of the Most Proper and Significant Words, and Terms of Art commonly used in the English Language (1713) has this very different definition for bisket:
A Bisket, a sort of Bread.
At least superficially, the definition of biscotin might describe a type of modern English biscuit, while the definition of bisket might serve as a rough description of a modern American biscuit.
Thomas Dyche & William Pardon, A New General English Dictionary, third edition (1749) is the earliest dictionary I've been able to find that lists the spelling biscuit:
BISKET, BISCUIT, or BISQUET (S.) commonly understood of small cakes made by the confectioners, of fine flower, eggs, sugar, &c. also the bread carried to sea, is called sea biscuit.
This same definition appears in editions of Dyche & Pardon as late as the eighteenth edition (1781).
Next to weigh in is Samuel Johnson, A Dictionary Of The English Language (1755), with entries for biscotin and biscuit:
BISCOTIN, n. s. {French} A confection made of flour, sugar, marmalade, eggs, &c.
BISCUIT, n. s. {from bis, twice, Lat. and cuit, baked, Fr.} 1. A kind of hard dry bread, made to be carried to sea; it is baked for long voyages four times. [Cited examples:] The biscuit also in the ships, especially in the Spanish gallies, was grown hoary, and unwholesome. Knolles's History of the Turks. Many have been cured by abstinence from drink, eating dry biscuit, which creates no thirst, and strong frictions four or five times a day. Arbuthnot on Diet. 2. A composition of fine flour, almonds, and sugar, made by the confectioners.
From Johnson's remark that sea biscuits are baked for long voyages four times, we might conclude that a more suitable name for these items would be quadriscuits or tetriscuits. Richard Knolles, The Generall Historie of the Turkes (1603) does indeed contain the passage that Johnson quotes, but he spells the word bisket:
for all the water thereabout was most vnpleasant and exceeding brackish, so that both sicke and whole were glad when they could get a crab to quench the extremitie of their thirst : although the emperour did what he might to remedie these extremities, and much releese was in good time sent both from SICILIA and NAPLES : the bisket also in the ships, especially in the Spanish gallies, was growne hoarie and vnwholesome.
In fact, Knolles uses bisket five times in the course of his Historie—including this instance:
there was such a dearth in the Turks armie, that they were enforced to giue their camels bisket and rice, and when that failed, they gaue them their pack-saddles to eat, and after that pieces of wood beaten into pouder, and at the last the very earth : which dearth endured vntill they arriued at VAN.
It seems clear from this instance that Knolles used the term bisket for what much later in the United States came to be called hardtack. In pioneer days, land travelers as well as sea travelers commonly carried hardtack with them on long trips. Here is the entry for hardtack in The American Heritage Dictionary of the English Language, fifth edition (2010):
hardtack n. A hard biscuit or bead made with only flour and water. Also called sea biscuit, sea bread, ship biscuit.
Next, chronologically, in my dictionary survey comes Nathan Bailey, The New Universal English Dictionary, fifth edition (1760), which covers biscotin and bisket/bisquet:
BISCOTIN (Confect.) a confection made of fine flower, powder'd sugar, marmalade, the white of eggs, &c.
...
BISKET, BISQUET (with Confectioners) a composition of fine flower, eggs, sugar, &c.
The seventh edition of Bailey's New Universal English Dictionary (1776) further complicates the picture by adding a separate entry for bisquet:
BISCOTIN (Confect.) a confection made of fine flower, powder'd sugar, marmalade, the white of eggs, &c.
...
BISQUET (probably of bis, twice, and coctus, baked) a sort of hard baked-brad or cake.
BISQUET, BISKET (with Confectioners) a composition of fine flower, eggs, sugar, &c.
Conclusion
The history of bisket, biscotin, biscuit, and bisquet is quite convoluted. At various times before 1800, dictionaries have used all of these words to identify a confection made with flour, eggs, sand sugar (among other ingredients). But at other times before 1800, dictionaries have applied the words bisket, biscuit*, and bisquet to tiny rounds of hard-baked bread.
Under the circumstances—especially in view of the equivocal treatment of the word in Samuel Johnson's tremendously influential 1755 dictionary, it is hardly surprising that British English went one way with the word biscuit and North American English went the other.
Solution 2:
An interesting anecdote appears in Benjamin Franklin's Autobiography.
Then I walked up the street, gazing about till near the market-house I met a boy with bread. I had made many a meal on bread, and, inquiring where he got it, I went immediately to the baker's he directed me to, in Second-street, and ask'd for bisket, intending such as we had in Boston; but they, it seems, were not made in Philadelphia. Then I asked for a three-penny loaf, and was told they had none such. So not considering or knowing the difference of money, and the greater cheapness nor the names of his bread, I bade him give me three-penny worth of any sort. He gave me, accordingly, three great puffy rolls
The story is described in the context of the English language in Speaking American: A History of English in the United States by Richard W. Bailey:
Biskit in New England was a collective noun like salt: "Please give me some of the bisket."
[...]
Bisket, a staple food in Franklin's native Boston, was an unleavened bread made from flower [sic], salt, and water, a food known later in American English as hardtack. The contrast between the dry bread of Boston and the yeasty confection of Philadelphia is plain enough, but Franklin had more in mind with this anecdote. Boston and Philadelphia spoke different kinds of English, and they do so to this day.
All of this goes to show that confusion about the meaning of "biscuit" in American English dates back quite far, and in fact other sources seem to suggest that the distinction dates back to early cooking habits of European migrants colonizing North America, where yeast was a rare commodity and unique methods of preparing dough for baking biscuits allowed for a kind of bread textured well for mixing with gravy, resulting in the common dish "biscuits and gravy," which became a popular dish in the U.S. after the American Revolutionary War.
In essence, it seems that this method of baking was cheap, efficient, and provided a hearty breakfast to agricultural workers in the American South, where these fluffy biscuits were a common breakfast food. Judging from the sort of confusion illustrated in Franklin's story about ordering a "bisket," it seems unsurprising that a term for one kind of baked good could be appropriated to refer to another.
For what it's worth, the OED points out with uncharacteristically judgmental language that the modern spelling "biscuit" was only adopted in the 18th century. Previously both the English and American pastries were commonly spelled bisket:
The regular form in English from 16th to 18th cents. was bisket, as still pronounced; the current biscuit is a senseless adoption of the modern French spelling, without the French pronunciation.
Solution 3:
A minor addition to other answers:
If we accept a common meaning including something we might unambiguously call hardtack, i.e. a hard unleavened bread, we can consider from a culinary point of view what may have happened since then. The quotes from and referring to Franklin in RaceYouAnytime's answer dates such a meaning to the 18th century.
Modern meanings include both unleavened and leavened products. Recipes calling for yeast are rare and mainly modern, so most of the leavened interpretations rely on chemical raising agents (baking powder or baking soda). According to Wikipedia , these date from the early 19th century. The addition of baking powder would make a softer, more porous dough that wouldn't keep as well as a hard biscuit, but this wouldn't matter except when travelling, and would be more pleasant to eat in quantity. So someone making a sort of raised biscuit for sale as a staple food might expect to do quite well, and this would them become popular.
Mrs Beeton, writing in mid-19th-century London, has several recipes in her famous book fitting the British definition of biscuits, that is unleavened, hard and made mainly from flour and butter. She also includes some with beaten eggs that are clearly lighter, and some recipes which use the terms biscuit and cake almost interchangeably. Most contain sugar except "SIMPLE HARD BISCUITS...These biscuits are very nice for the cheese course." One recipe stands out though:
SODA BISCUITS.
- INGREDIENTS.—1 lb. of flour, 1/2 lb. of pounded loaf sugar, 1/4 lb. of fresh butter, 2 eggs, 1 small teaspoonful of carbonate of soda.
Mode.—Put the flour (which should be perfectly dry) into a basin; rub in the butter, add the sugar, and mix these ingredients well together. Whisk the eggs, stir them into the mixture, and beat it well, until everything is well incorporated. Quickly stir in the soda, roll the paste out until it is about 1/2 inch thick, cut it into small round cakes with a tin cutter, and bake them from 12 to 18 minutes in rather a brisk oven. After the soda is added, great expedition is necessary in rolling and cutting out the paste, and in putting the biscuits immediately into the oven, or they will be heavy.
Time.—12 to 18 minutes. Average cost, 1s.
Sufficient to make about 3 dozen cakes. Seasonable at any time.
[italics mine]
The warning or they will be heavy clearly implies they're meant to be light, and the recipe is similar to scones or (US) biscuits (though Mrs Beeton's dough is drier and uses less baking soda).
Having made the recipe (pictured below) these are indeed softer than a shortbread or similar plain biscuit, like a denser (and rather sweet) scone, or a dense, sweet version of the American biscuit. This is a very different softness to a soft cookie. These soda biscuits can be regarded as intermediate between the two current meanings, giving us an idea of when the meanings were more fluid.
There are parallels with the history of Irish soda bread, and Wikipedia's article on biscuit (bread) is also worth a read. Both of these sources discuss the suitability of certain American wheats for quick breads. There's a long history of the same word ending up with different meanings either side of the Atlantic; flapjack is a related example, but we can also consider elk or robin.