How to choose keys in keychain access?
Put your per-server SSH-configuration into ~/.ssh/config like this:
Host useKeyOnly.net
user sshuser
Identityfile ~/.ssh/mykey1
IdentitiesOnly yes
Host usePasswordOnly.org
PubkeyAuthentication no
Port 53221
user root