Context

I have a personal server that I use for the web. I sometimes need to SSH/SFTP to it.

Disclamer: I have very little experience with nginx internals.

Problem

This morning, I figured out that the free wifi in a well-know cafe chain was blocking SSH (actually, they are blocking anything that's not on 80/443). But when I need SSH, I need it, so I looked for ways to share SSH and HTTPS on the same port.

What I looked at

I have looked at a few possible solutions that can run on port 443:

• SSHL: a SSH/OpenVPN/HTTPS multiplexer;
• OpenVPN: a VPN solution has a built-in multiplexer for OpenVPN and HTTPS;
• HAProxy: a webserver/load balancer can also multiplex everything.

All of these seem pretty straight-forward but I don't really like the fact of adding layers and complexity and possibly slowing things down just in the unlikely event that I need to SSH on 443.

Putting nginx into the mix

I know that nginx already supports raw TCP streams handling. So I was wondering if I could use that on port 443 too directly in nginx. The idea being that nginx could choose to use the http module if it recognizes HTTP(S) or stream for everything else.

Questions

In that context, I have two questions:

• Is nginx even capable of doing such a distinction? (I am not even sure I would be able to listen on port 443 in both the http and the stream block at the same time.)
• If so, would there be any blatant performance issue with that setup? (I am thinking about transfer speed with SFTP for instance, not really SSH per se.)

Since nginx version 1.15.2 added new variable $ssl_preread_protocol. And in official blog added post about how to use this variable for multiplexing HTTPS and SSH on the same port https://www.nginx.com/blog/running-non-ssl-protocols-over-ssl-port-nginx-1-15-2/

Example of configuring SSH(by default) and HTTPS:

stream {
    upstream ssh {
        server 192.0.2.1:22;
    }

    upstream web {
        server 192.0.2.2:443;
    }

    map $ssl_preread_protocol $upstream {
        default ssh;
        "TLSv1.2" web;
    }

    # SSH and SSL on the same port
    server {
        listen 443;

        proxy_pass $upstream;
        ssl_preread on;
    }
}

I have a working configuration tunneling ssh over tls on port 443, using the nginx stream module. I also do XMPP over TLS and normal HTTP on the same port. I do the mutliplexing via ALPN.

(You need nginx > 1.13.10 to use the ssl_preread module with alpn http://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html#ssl_preread)

My configuration uses the docker version of nginx, but it should also work without docker.

stream {
    # check ALPN for xmpp client or server and redirect to local ssl termination endpoints
    map $ssl_preread_alpn_protocols $ssl_multiplexer {
        "xmpp-client"     127.0.0.1:5422;
        "xmpp-server"     127.0.0.1:5469;
        "identifyssh"     127.0.0.1:8822;
        default           127.0.0.1:8443;
    }

    server {
        listen 443;
        ssl_preread on;
        proxy_pass $ssl_multiplexer;
        proxy_protocol on;
        set_real_ip_from  172.18.0.0/32;
    }

    # ssl termination for c2s connections
    server {
        listen 5422 ssl proxy_protocol;
        # ... <- tls keys and options here
        proxy_ssl off;
        proxy_pass ejabberd:5222;
    }

    # ssl termination for s2s connections
    server {
        listen 5469 ssl proxy_protocol;
        # ... <- tls keys and options here
        proxy_ssl off;
        proxy_pass ejabberd:5269;
    }

    # ssl termination for ssh connections
    server {
        listen 8822 ssl proxy_protocol;
        # ... <- tls keys and options here
        proxy_ssl off;
        proxy_pass yourserver:22;
    }
}

If you want to use the XMPP stuff you have to add some SRV records to point to your servers 443 port, see https://xmpp.org/extensions/xep-0368.html

If you want to connect to your ssh server you have to wrap your ssh session in a ssl session that sends the ALPN string you defined in your stream config. I used "identifyssh" in the example above. You can use anything, but try to not collide with the official defined names: https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids

To start the ssh session from your client to your prepared server use:

ssh you@yourserver -o "ProxyCommand openssl s_client -alpn identifyssh -ign_eof -connect yourserver:443"

And you should be connected.

I should also note, that I use the proxy_protocol to keep the clients headers and IP-address while passing to my backends.

Your normal http server configured in the http {} section should take care of this:

server {
  listen 8443 ssl proxy_protocol;
  # ...
}

The best thing is, that you don't need tools like sslh, stunnel, proxytunnel or others to make this work. You only need a newer nginx and openssl. Hope this helps somebody. It would helped me digging into that stuff.

Or, you could proxytunnel ssh through your nginx with HTTP/S connection

Take a look: proxytunnel