How to properly remove an old ssh key [duplicate]

I have a script that dynamically creates Amazon EC2 instances and adds their ssh key to my ~/.ssh/known_hosts. However, whenever I need to refresh the instance by terminating and recreating it, I get obnoxious warning messages like:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
<fingerprint>.
Please contact your system administrator.
Add correct host key in ~/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in ~/.ssh/known_hosts:94
  remove with: ssh-keygen -f "~/.ssh/known_hosts" -R <hostname>
Password authentication is disabled to avoid man-in-the-middle attacks.
Keyboard-interactive authentication is disabled to avoid man-in-the-middle attacks.

The warning is understandable, but even if I run that command to remove the old key, logging in still gives me the warning:

Warning: Permanently added '<hostname>' (ECDSA) to the list of known hosts.
Warning: the ECDSA host key for '<hostname>' differs from the key for the IP address '<ip>'
Offending key for IP in ~/.ssh/known_hosts:96

The solution is to manually remove that line in my known_hosts, but is there a way to automate this by running a single command beforehand?

From man ssh-keygen (SSH-KEYGEN(1))

 -R hostname
         Removes all keys belonging to hostname from a known_hosts file.  This option is useful to
         delete hashed hosts (see the -H option above).

Try this:

ssh-keygen -R hostname [-f known_hosts_file]