Why "broke" and not "broken" in "If it ain't broke, don't fix it"?

Solution 1:

If it ain't broke, don't fix it.

The saying uses "broke" because it's deliberately going for a folksy, non-grammatical feel of homespun wisdom. The implication is that simple people (those least likely to adhere to strict grammar rules) have an innate common sense that the more refined among us do not share. Such people tend to use "ain't" for "isn't" and "broke" for "broken" in that case.

In other words, it simply wouldn't be as convincing to say

If it isn't broken, don't fix it.

Addendum

I'll further note that the OP's construction, which mixes ain't with broken, violates the register's integrity by mixing a grammatical syntactical element ("broken") with an "ungrammatical" one ("ain't"). So it gets the tone wrong as well. It would be a little like showing up in formal wear to a barbecue.

Point of clarification

I do not mean to imply that the construction is ungrammatical, only that it is perceived as such by habitues of a more formal register. That's why I was careful to qualify my references to grammaticality.

Solution 2:

Because the phrase is not in standard English, it is in any of several dialects in which ain't broke is grammatical.

It is true that the phrase is often used by people who do not otherwise use such dialects: in that sense, its almost a foreign phrase, like je ne sais quoi.