Once I replace a party member, can I ever get him back?

This is coming entirely from my memory, so please forgive me for any mistakes:

Most party members will stay where you put them (or if you say 'No' to them when they want to join). There are a number of exceptions though:

• If you remove Kivan, he stays gone for good.
• If you remove Shar-Teel, she stays gone for good.
• If you say 'No' to Jaheira and Khalid the first time, you can never get them again.
• Furthermore, if you kick out one of them, the other goes as well.
• If you say 'No' to Edwin with Dynaheir in your party, you can never get him again.
• If you say 'No' to Viconia the first time, you can never get her again.
• The same goes for Branwen and Yeslick
• If you make Xan leave by being rude, he's also gone forever.

Basically, whenever a party member leaves beacuse of arguments or disagreements they are gone for good.

In addition to the accepted answer, Garrick will also be gone for good if you refuse him the first time.

Khalid and Jaheira go back to the Friendly-Arm-Inn if you kick out one of them.