Can I use windows inside a docker container?

is it possible to configure a docker container to run Windows

Yes, you can Run native Windows Server Containers

connect to it via RDP

Nope.

According to an answer, Windows Containers does not support RDP. To quote an article from 2019:

But notice you still cannot RDP into such Windows containers.

See What's new for Docker on Windows Server 2019?