Connect to websocket server without specifying port

I'm trying to create a secure node.js server without a websocket server on the same port. The port is 8080.

I can access the url in the browser and I can connect to websockets when I specify the port.

https://ws.site.com // Works
wss://ws.site.com // Don't work
wss://ws.site.com:8080 // Works

Why is this? What am I doing wrong? This is the nginx config

upstream ws.site.com {
    server 127.0.0.1:8080;
}

server {
    listen 443;
    server_name ws.site.com;

    ssl on;
    ssl_certificate /path/ssl-bundle.crt;
    ssl_certificate_key /path/myserver.key;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

    location / {
            access_log off;

            proxy_pass https://ws.site.com;
            proxy_redirect off;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_read_timeout 86400;

            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
    }
}

The wss:// URL handler uses the default port of WebSockets, which is 443 if you don't add a port suffix by your own. If you run it on a non-default one (8080) you need the port suffix.