AWS: how to install AWS-issued SSL certificate on my Windows Server EC2?

Are you talking about the AWS Certificate Manager? If so you can't use them like that.

From the FAQ:

Q: What can I do with AWS Certificate Manager?

A: You can request and provision SSL/TLS certificates and deploy them for sites and applications that use Elastic Load Balancing, Amazon CloudFront, or Amazon API Gateway. Once you validate ownership of the requested domain and the certificate is issued, you can select the SSL/TLS certificate from a drop-down list in the AWS Management Console to deploy it. Alternatively, you can deploy certificates provided by ACM to AWS resources using AWS Command Line Interface (CLI) commands or API calls. ACM manages certificate renewals and certificate deployment for you.

If you're talking about another kind of certificate, it's probably not supported by AWS-issued certificates — for example, you can't export the certificate to import into IIS or Apache.

Alternatively, please edit your question to make it more precise.

You can only use those certificates (AWS Certificate Manager) on services like Elastic load balancer, Elastic Beanstalk, Cloudfront and I think API Gateway as well. This means you don't see the certificate itself, you can only use it on their services (unfortunately not EC2)