Private/Public Key Authentication for Windows Remote desktop
Solution 1:
Remote Desktop supports X.509 client certificates, under the "smart-card authentication" name. Despite the name, it should work with locally-installed certs/keys (i.e. without an actual smart-card). Though it does however require an Active Directory domain, as far as I know.
So, sort of but not really in a way that's useful to you.