Force Colloquy not to use built-in Growl notifications
I lead myself to the solution from a posting on Google Groups. Turns out you have to force Colloquy to run in 32-bit mode in order to get it to show up in the Growl .prefpane
:
Then, you can customize the Colloquy notifications however you choose: