Why "well-behaved" instead of "well-behaving"
Solution 1:
Short answer: This is a consequence of the original (15th century) behaviour of the verb behave.
CAVEAT: "Why?" questions can often be unanswerable in historical linguistics. One can only discuss factors that might have had an influence, and describe how they interact through the passage of time.
The verb behave was formed in the late 15th century as a transparent compound of be- and have. But it was generally used reflexively, i.e. with an object pronoun -self or one of its forms. For example, from William Caxton in 1474:
Ony man that wylle truly behaue hym self.
From the King James Version of the Bible, printed in 1611:
1 Chronicles xix.13: Let vs behaue our selues valiantly for our people.
Its use as an independent adjective, beheft or behaved, is attested in Shakespeare. From Hamlet, Act III:
And gather by him as he is behau'd, Ift be th' affliction of his loue or no.
Its use with well- as well-behaved is attested by the OED from the late 16th century. From 1577, in a "Praise of Solitarinesse" by Roger Baynes:
Musicall birdes..maye rightly be sayde, to followe a wished and well behaued kinde of life.
Only later did the verb behave become an intransitive verb, without a reflexive pronoun. The OED dates it back to 1721 - from Edward Young's "Revenge: A Tragedy":
As you behave, Your father's kindness stabs me to the heart.
Even later on, from the 19th century onwards, did behave start being applied as an intransitive verb to inanimate noun subjects such as words, chemical elements, and mathematical functions.
Hence, the use of the passive past participle behaved as an adjective came first, when one still behaved oneself. Only in the 18th century did behave become intransitive, but by that time well-behaved had been established in the language for several hundred years.
The OED draws parallels between behaved, learned and well-read, with similar semantics and similar use of the past participle as the adjective. I would also draw parallels between behave oneself and the idea of carry oneself, still visible in the French reflexive verb se comporter, which is the basis of the French equivalent for the noun, comportement meaning behaviour. This is also paralleled in dress oneself, which formed its adjective well dressed in the late 15th century as well.