Where should AWS EC2 Key Pair be stored?

The best way to protect your EC2 keypairs, which are just SSH keys, is to encrypt them with a passphrase (and follow your normal password management process for that passphrase). Assuming you are using linux, you can use ssh-keygen -p -f $file to encrypt the key. You should keep a backup, preferably physically secured (i.e. a thumb drive in a safety deposit box or something). I am assuming you are talking about the private half of the key, since the public key is obviously public.

Theoretically, it would be better to store the key on a TPM on your workstation, or on a smartcard, but there are usually practical problems with this solution when dealing with SSH keys.

Whether it is bad to store the key on your home PC depends on whether this is a violation of policy. If it's not, honestly, there is little reason to view this as being any worse than storing it on a laptop you use for work.

You can certainly keep a backup of the key in S3 (instead of a physical backup). The threat model is such that you are already having a very bad day (in terms of data leaks and interruption of service, among other things) if someone is able to access your AWS account. But, unless there is some security principal who might have access to the S3 bucket with your key but not be allowed to log into the machines, you would need to find another way. If you store a copy in S3, at least make sure it is encrypted with a passphrase.

Well, the public key can be stored wherever you'd like. Tattoo it on your forehead, for instance. :)

The private key is what you need to protect, as it is truly your identity. Anyone that gets their hands on your (unencrypted) key can have their way with your servers. So, protect it and back it up as you would any other important but confidential file. Encrypt it and save it on a flash drive, print it out on paper and store it in a safe as a last resort, etc. If you want to store it in S3, that's probably fine, but I'd only do that in its encrypted form.