What's the origin of "with flying colo(u)rs"?

The OED's first quotation for successfully "[passing] with flying colours" is:

1622 W. Ames Reply Dr. Mortons Gen. Def. v. 83 He is as it seemeth, a great adventurer: For hee commeth forth upon this peece of service with flying colours.

Partridge's A Dictionary of Cliches (1978) says it is "Originally of a man-of-war."

Colours has been used for a flag or ensign flown by a military ship since at least Shakespeare's time. The first three quotations in the OED:

1590 J. Smythe Certain Disc. Weapons 2 b, Their Ensignes they will not call by that name, but by the name of Colours.
1595 Shakespeare Henry VI, Pt. 3 ii. ii. 173 Sound trumpets, let our bloudie colours waue.
1598 R. Barret Theorike & Pract. Mod. Warres ii. 20 We English-men do call them [ensigns] of late Colours, by reason of the variety of colours they be made of.

And the OED has "with flying colours" for a regiment or ship flying a flag, ensign or standard:

1612 J. Speed Theatre of Empire of Great Brit. ix. xxiv. 855/2 Philips Army with flying colours sent lately into Ireland vpon gift made vnto him by the Pope..bewraied their intents.

The phrase is sometimes also "to come off with flying colours", and the OED's 6th sense of the verb come, first recorded by Shakespeare, is:

6. To leave the field of combat; to retire or extricate oneself from any engagement; usually with reference to the manner, as to come off with flying colours , to come off second best , to come off badly , to come off safely , to come off victorious , to come off a loser , etc.

1600 Shakespeare Merchant of Venice i. i. 128 But my cheefe care Is to come fairely of from the great debts.
a1616 Shakespeare Coriolanus (1623) i. vii. 1 We are come off, Like Romans, neither foolish in our stands, Nor Cowardly in retyre.

So to come off with flying colours is from a battleship being able to successfully leave the battle, undefeated, with its flags and ensigns still flying proudly.

Antedatings?

The OED has the literal "with flying colours" from 1612 (adj. n.). I found some possible antedatings from 1609, but they're "with their colours flying" (n. v.) in A Generall Historie of the Netherlands by Edward Grimeston. Page 120:

... and drew in his men with their colours flying, making great and fearfull cries.

Page 501:

...there was also a part of the armie in battaile, with their colours flying, and the rest remained readie in their trenches...

Page 790:

That garrison should depart with their colours flying, armes, bag and baggage, whither they are pleased.

Page 1316:

... depart freely unto Dam, with their baggage, armes and colours flying, bullets in the mouthes, drummes sounding, and matches light ...

I think you're right there. "Colors" refers to naval flags of all kinds. When the "colors are flying", they're hoisted, flying from the masts. In a boat-race, the winner would be sailing in, with flags flying.

You're correct! It refers to the flags on a naval craft. Flying the "colors" expressed commitment during a battle. Thus, if you pass a test with "flying colors", you're expressing dedication and commitment to your area of study!