How to rsync with a private key?

The command fails not because of wrong keys, but because you're telling rsync to run my/file instead of ssh (by using the -e option, which picks up the word following it). Remove the -e option first.

Since rsync normally uses ssh to connect, you can configure both to always use a particular key for connecting to cloudapp. For example, put this at the top of your ~/.ssh/config file:

Host me.cloudapp.net
    Username me
    IdentityFile ~/my-cloudapp-key.key
    IdentitiesOnly yes

The Username me part will also let you skip adding me@ when using ssh or rsync. Plain rsync -avz my/file me.cloudapp.net:/my/path will work.

Note: SSH keys are not X.509 certificates; they're just plain RSA or ECDSA keypairs without any additional information.