SSH: prevent from asking for passphrase when ssh-agent auth fails

I am writing a script to cycle through a list a known hostnames so as to find a working SSH server on which I can auth with my SSH key.

The key is already loaded with ssh-agent and ssh-add, so if I connect to a working remote host that knows my key, no passphrase is prompted and I am successfully connected without any interaction.

Thing is, when the remote host does not know my key, ssh prompts me with the passphrase (it has no chance to succeed because, if it could, it would have already logged me in thanks to ssh-agent). I would like to prevent this behavior and make ssh abort when the ssh-agent auth fails.

I am currently using the following command to bypass most of the interaction but I can not prevent the one I just described:

$ ssh -i ~/.ssh/id_rsa                      \
      -o UserKnownHostsFile=/dev/null       \
      -o KbdInteractiveAuthentication=no    \
      -o StrictHostKeyChecking=no           \
      -o PreferredAuthentications=publickey \
      -o ConnectTimeout=1                   \
      $host -n "whoami"

Thanks for your answers.

man ssh_config:

 BatchMode
         If set to ``yes'', passphrase/password querying will be disabled.
         This option is useful in scripts and other batch jobs where no
         user is present to supply the password.  The argument must be
         ``yes'' or ``no''.  The default is ``no''.

While the BatchMode answer is certainly the right one, I want to also mention that you can also do: -o PasswordAuthentication=no ; same man page.