"Here's looking at you, kid" meaning?

I'm sure many will know Rick's famous line from the film Casablanca:

Here's looking at you, kid.

While I can guess at it, I was never fully confident about the meaning of this phrase. I am not a native speaker. It's clear that it's a toast. Would someone please explain in detail?

In more detail,

• Here's [to] because it's a toast
• Was this phrase a common American expression at the time?
• Why looking? Is it simply part of a common phrase or does it refer to looking at her as looking at a woman?
• Can you give me examples of similar (or the same) phrase, in context?
• Is the meaning unambiguous to native speakers or is there room for interpretation?

The toast goes back more than half a century before the scriptwriters of Casablanca used it in 1942.

From Anonymous, A Holiday Skip to the Far West (1884), we have this scene set in the Southern Hotel in St. Louis, Missouri:

"Any friend of Mr. O'Sullivan welcome. Will you nominate your poisons, gentlemen?" Since poisoning was apparently a necessity, I nominated whisky and water, in preference to a native mixture. It would be more satisfactory to know the cause of death, thought I, should this uncalled-for fatality occur. "Wall, —— here's looking at you," says my newly-acquired friend of the white apron, winking over the bottle-nose, and emptying his glass. "Here's how," responds the porter, and the curious bacchanalian ceremony came to a close. I am not proud, and enjoyed the hospitality, as the porter paid the drinks, be it said to his credit.

From "Things Pleasant and Otherwise," in Ballou's Monthly Mahgazine (May 1884):

"Ah," said the colonel, as he threw out a chew of tobacco, and took the bottle. "Here's looking at you."

"Drink hearty," the young man replied, and taking the bottle he held it up, and added, "Here's to the hair of your head."

From "Editor's Table" in The Yale Literary Magazine (April 1895):

Are your glasses filled, gentlemen? Then, here's to the rose of the girlhood—Dorothy, and one more in which that jolly devil, Saint Elihu, shall join for all his shadowy background and his austere countenance. And so fellow Collegians—"here's looking at you."

From Edith Chase & W.E.P. French, Waes Hael: A Collection of Toasts Crisp and Well Buttered (1903):

New Mexico

Cactus and prickly pear and sagebrush,/“Men with the bark on” and hearts brave and strong,/New Mexico, here's luck, here's looking at you;/You're a deep note in freedom's world-wide song!

Early extensions of the toast include this one from The Harrow (1889) [result not shown in snippet window]:

French returns in a few minutes and finds the Sophomore has cultivated it up. Waldo, at Hillsdale— "Here's looking at you." George — "Yes, through a glass darkly."

And this one from Wilbur Nesbit, The Loving Cup: Original Toasts by Original Folks (1909):

To you, and you, good fellows of the quill,/Belongs the Homage of the Wine I spill;/Here's looking at you from the Rosy Side,/And may you never lose a Dollar Bill.

I imagine that in a heavy-drinking, toast-oriented social environment, few dedications would come to the befogged mind more readily than "Here's looking at you," since the person addressing the toast is in all likelihood looking at the person thus addressed.

An early filmed instance of the expression occurs in the movie Three on a Match (1932), as Michael Loftus (Lyle Talbot's character) attempts to woo/seduce Vivian Kirkwood (Ann Dvorak):

Michael: Here you are, darling. [He hands Vivian a martini.] Well, here's looking at you.

Vivian: At me?

Michael: Yeah! And liking it, too!

Evidently, the expression was widely viewed as being quintessentially American, which would make sense in Casablanca as another marker of Rick's Americanness. At any rate, in one of J.P. Marquand's Mr. Moto stories (by 1938), we have this bit of dialogue [combined snippets]:

"Boy!" called Uncle William. "Whisky soda. ... Here's to you, Mr. Moto." Mr. Moto laughed again.

"Here is looking at you, gentlemen," he said. "That is the American expression, is it not? What beautiful weather we are having!"

"Yes," said Uncle William.

UPDATE: 'Here's Xing' as an American form

Because so many readers have echoed the comment/question (from andy256 below) "Yes, but ... what does it mean? Or if it is quintessentially American, what does that mean about America?" I decided to look into the idiomatic use of "Here's Xing" in U.S. usage.

Since Americans are (as Fowler says of people who grew up with the distinction between will and shall) "to the manner born," the form "Here's looking at you" doesn't strike us as particularly odd. Indeed, the similarly formed expression "Here's hoping" is used not just as the start of a toast, but as an equivalent (in a multitude of settings) to "I hope so."

One of the earliest examples of "here's hoping" in a Google Books search appears in "Metaphors of the People," in The Galaxy (May 1870), which quotes Mark Twain's description of toasts given in various locales in the U.S. West:

The reaching of the popular mind after odd fancies is illustrated in Mark Twain's humorous explanation of the etiquette of the bar-room on the Pacific coast. He says:

In a mining camp in California, when a man tenders you a "smile," or invites you to take a "blister," it is etiquette to say, "Here's hoping your dirt will pan out gay." In Washoe [Nevada], where you are requested to "put in a blast," or invited to take your "regular poison," etiquette admonishes you to touch glasses and say, "Here's hoping you'll strike it rich in the lower level." And in Honolulu, when your friend the whaler asks you to take "fid" with him, it is simply etiquette to say, "Here's eighteen hundred barrels, old salt." But "drink hearty" is universal. ...

The sentimental method of asking a person to drink is in the formula, "Suppose we shed a tear." The operation, strange as it may seem, is identical with "taking a smile." There is a frequent toast in some places which seems to contain considerable truth, viz.: "Well, here's another nail in my coffin."

Of all these expressions, only the "Here's another nail" and the "Here's eighteen hundred barrels" follow (arguably) grammatical form, but it appears that the "Here's Xing" wording arose fairly early and has shown impressive cultural persistence. In fact, it appears in James Fenimore Cooper's The Pioneers (1823):

Toasts were uniformly drunk ; and occasionally, some one, who conceived himself peculiarly endowed by nature to shine in the way of wit, would attempt some such sentiment as "hoping that he" who treated "might make a better man than his father; or "live till all his friends wished him dead;" while the more humble pot-companion contented himself by saying, with a most imposing gravity in his air, "come, here's luck," or by expressing some other equally comprehensive desire. In every instance, the veteran landlord was requested to imitate the custom of the cupbearers to kings, and taste the liquor he presented, by the invitation of "after you is manners;" with which request he ordinarily complied, by wetting his lips, first expressing the wish of "here's hoping," leaving it to the imagination of the hearers to fill the vacuum by whatever good each thought most desirable.

Given decades of "Here's hoping..." in the context of toasts and in the ordinary affairs of life outside barrooms, the emergence of "Here's looking at you" can hardly have struck listeners as a strange innovation. That toast, in turn, may have prompted the toast, "Here's mud in your eye," which a Google Books search finds as far back as 1912 in the form, "Here's mud in your eye and a quick trip back to Texas!" There are numerous competing interpretations of that expression, however.

Both Medica's and Sven Yargs' answers throughly address the origin and etymology of the quote, but for the meaning behind it, we'll have to dissect the quote itself.

"Here's to" Is a common toast - a wishing of good will and declaration of admiration towards an idea. That's easy enough to understand.

"Looking at you" is the idea that Rick is toasting - the idea of looking at Ilsa. If we take the two together, we get the meaning of Rick's Toast. He's saying that looking at her is something to be happy about. And given the first time he says it he's being quite flirtatious with her, the intent is almost certainly to flirt with Ilsa.

"Kid" in this case is just an affectionate way of referring to her as young - which re-affirms the idea that he's calling her good-looking and being flirtatious with her.

So, "Here's looking at you, kid" simply means that he's glad she's there, that she looks beautiful, and is a very unique way of flirting with her.

Here's to is a common toast (or even expression of a wish, good luck, or other sentiment, and has long been so. Romeo it in Romeo and Juliet (Act V Scene 3) as he dies at Juliette's side.

Eyes, look your last!
Arms, take your last embrace! and, lips, O you
The doors of breath, seal with a righteous kiss
A dateless bargain to engrossing death!
Come, bitter conduct, come, unsavoury guide!
Thou desperate pilot, now at once run on
The dashing rocks thy sea-sick weary bark!
Here's to my love!
[Drinks] O true apothecary! Thy drugs are quick. Thus with a kiss I die.

Romeo is looking at Juliet when he toasts he with the poison.

Here's to is still very common in English. The shortened version is "To". It is unambiguous. In Casablanca, Richard says it to Ilsa, the woman he loves as he says goodbye to her. It is very much like Romeo used it.

"Here's to a long and happy life together" is a common wedding toast. It can be used with anything.

• Here's to looking at the world through a child's eyes!
• Here's to seeing big changes in the upcoming weeks!
• Here's to seeing. Here's to recognizing what we're seeing. And certainly, here's to Friday and the ending of a chaotic week.
• Here's to imagining great things and living in bliss.

Here's looking at you was used before Casablanca. There is a 1938 book of that title by an author named Houghton.

I'm surprised nobody mentioned that — in Europe particularly — there is a tradition that says that one must look directly into the other's eyes when toasting, or bad luck will befall one ("Seven years of bad sex"). So by saying, "Here's looking at you", the speaker is calling specific attention to the fact that s/he is honoring this tradition and expects that no bad luck will ensue. This may in fact be the origin of the phrase.