What port should I open to allow remote desktop?

Remote Desktop requires TCP port 3389 to be open. Also, opening UDP port 3389 enables acceleration since RDP 8.0.

It is possible to change the port used by the terminal server (or PC which is accessed), see this Microsoft support article: How to change the listening port for Remote Desktop. The UDP port for accelerated connection uses the same port number as the TCP setting and cannot be changed separately.

UDP acceleration is available since RDP 8.0 (shipped with Windows 8 and Windows Server 2012, available via an update on Windows 7 / Windows Server 2008 R2).

In addition to opening port 3389 for UDP and TCP, I had to go edit the windows firewall rule and set Edge traversal to allow. Like this: