How to configure ssh client to use private keys automatically

Yes, you want to create a ~/.ssh/config file. That lets you define a shortcut name for a host, the username you want to connect as, and which key to use. Here's part of mine, with hostnames obfuscated:

Host tabs
     HostName tabs.com
     User     me
     IdentityFile       ~/.ssh/new_rsa

Host scm.company.com
     User       cap
     IdentityFile       ~/.ssh/git_rsa

Host project-staging
     HostName 50.56.101.167
     User     me
     IdentityFile       ~/.ssh/new_rsa

With this I can say, ssh tabs and get connected to host tabs.com as user me, with key new_rsa, as though I'd used ssh [email protected] -i ~/.ssh/new_rsa.

SSH clients will typically use ~/.ssh/identity (ssh v1) or one of~/.ssh/id_rsa or ~/.ssh/id_dsa (v2) as the default private key. You can change this in ~/.ssh/config (the IdentityFile parameter - the -i option to SSH actually overrides this. See man ssh_config for details).

If you have multiple private keys to deal with using ssh-agent is probably a better choice.
See man ssh-agent for more details.