Cannot find user certificates in Windows registry

Public and private keys are not stored in the same place. Public certificates are stored in the registry, but their associated private keys are stored in the file system.

My public keys are in the registry at:
HKEY_CURRENT_USER\SOFTWARE\Microsoft\SystemCertificates.

I find my private keys on the disk at:
C:\Users\USER-NAME\AppData\Roaming\Microsoft\SystemCertificates\My\Certificates

The PowerShell command ls Cert:\CurrentUser\My\ returns installed certificates with public keys.

For more information see the article System Store Locations.