Why am I able to SSH a remote machine even with the wrong keys?

The command cat ~/.ssh/fede_windows.pub >> ~/.ssh/authorized_keys appends the key to authorized_keys, it doesn't replace what's already there.

To replace all old keys, run cat foobar.pub > ~/.ssh/authorized_keys. This will truncate the file and then add the new information.

You can add as many keys as you want. To check, simply open the file with a text editor and have a look at the content.

The ability to have many keys is actually a security feature. It means you can use different keys from different computers, and if a computer is lost, you only have to remove the compromised keys, not all keys.