How to fix PR_END_OF_FILE_ERROR when using nginx with ssl?

For anyone else with this issue.. it can also happen if you have forgotten to add ssl to the listen directives. Chrome shows ERR_SSL_PROTOCOL_ERROR whilst Firefox shows PR_END_OF_FILE_ERROR.

server {
     listen 443 ssl http2;
     listen [::]:443 ssl http2;
     ...
}