Why won't Minsc join my party?

From what I've been able to find your character's charisma is to blame. Baldur's Gate uses reaction rolls which are based on charisma, if you rolled terrible + bad charisma then he won't have anything to do with you.

A couple of people got him to join by increasing their character's charisma (with a character editor) and then removing Dynaheir from the party. After asking Dynaheir to re-join Minsc apparently came along for the ride.

If it isn't the charisma, then it could have something to do with having sub 10 reputation.

Sources:Bioware Forums & Gamefaqs

It's not just charisma. If you have the magician Edwin join you before Minsc, he won't like you very much. Minsc is guided by his hamster Boo, and I think Boo doesn't really like that magician.