OS X 10.6.4 System VPN failing to connect - IKE Packet: transmit failed

I had the very same issue:

It works on one system with 10.6.5, and with the same credentials, on a second 10.6.5 system I get this error.

I then discovered that on one system, another racoon daemon was running (in this case IPSecuritas) Turning this off made the internal client work.

This occasionally occurs for me as well, e.g. when I switch between two different Cisco VPNs. It seems to be a minor crash in racoon somewhere*. Manually restarting the process seems to do the trick for me:

~ :➜ ps -ef | grep racoon
0  5861     1   0   0:00.09 ??         0:00.15 /usr/sbin/racoon
~ :➜ sudo kill 5861
~ :➜ ps ax | grep racoon
<wait until it shuts down. sometimes you may need to SIGKILL it (kill -9 <pid>)>
~ :➜ sudo /usr/sbin/racoon
~ :➜ ps -ef | grep racoon
0  6786     1   0   0:00.08 ??         0:00.15 /usr/sbin/racoon -x

(Don't know what the '-x' param does, it's not mentioned in the man page)

(*) Haven't investigated