Origin of the phrase "free, white, and twenty-one"?

The earliest citation I can find is Marion Harland's Alone (1856), but I don't suppose it's an "origin".

Free, white, and twenty-one" sang Emma, cheerily. "Twenty-one! In four years, I shall be a spinster of a quarter of a century!"

The fact that it was well-established long before OP's 1930s movies is attested by this sentence in the Transactions of the Annual Meeting from the South Carolina Bar Association, 1886

And to-day, “free white and twenty-one,” that slang phrase, is no longer broad enough to include the voters in this country.

There were still black slaves in some states in the mid 1800s, so obviously being free and white was a meaningful part of "I can do what I want and no one can stop me". But unless it refers to the "freedom" to vote, I don't know what the significance of reaching 21 would have been at the time.

There was a time, sadly, when not being free, white, and 21 was a significant legal disability. Even by the 1930's, fortunately, that phrase was mostly a joke.

My father used to use the expression to indicate that a woman was a suitable romantic candidate: "She's free, white, and 21."

Although the phrase became something of a Hollywood cliché in the 1930's, it was around long before that and didn't die out until the civil rights movement of the 1960's.

According to a couple of sources the phrase appeared around 1828 as a description of who should be allowed to vote. Prior to 1790, one had to own land and/or meet other additional qualifications in order to vote (actual qualifications were set on a state-by-state basis), but those restrictions were gradually being dropped. With the election of Andrew Jackson as President of the United States in 1828, Jackson's position that property ownership restrictions be abolished everywhere apparently gave rise to the phrase that all you should need to be in order to vote is "free, white, and 21". (Being male was still required, but so obvious it did not need to be stated.) While some states had gotten there before 1828, it was not until 1850 that these other restrictions were finally eliminated across the entire country. This also appears to be the point of the quote from South Carolina Bar Association that FumbleFingers dug up: "free, white, and 21" used to (1850-1870) cover everyone who was allowed to vote, but with black men being allowed to vote, this was not the case anymore.

In this context, "free" referred not to not being a slave but primarily to not being in jail or prison for committing a crime. Even today, most people incarcerated for crimes cannot vote. Prior to the 15th amendment to the constitution in 1870, black men could not vote in most states whether they were free or not.

Of course "free, white, and 21" eventually came simply to mean unencumbered (by law or custom), and women would use it well before they would be allowed to vote to assert their right to make other choices in their lives, such as if and when and to whom to get married (or otherwise get sexually involved with). This was usually how it showed up in the Hollywood movies of the 1930's.