Please use the ssh-agent.

  • First add your key to the agent: ssh-add ~/.ssh/$keyfile.
  • Then verify that the key is loaded successfully: ssh-add -l
  • Then try to ssh to your remote box. (The agent should do the authentication.)

If there really is a problem with your key you should notice once you try to add it.