SSH X11 forwarding is extremely slow over VPN

I'm using an VPN (with OpenVPN) to keep access between my home and work computers, and today I tried to ssh-forward an app which is GUI-only, and I found this to be terribly slow. I've used SSH X11 forwarding before, and it indeed has latency, but between this two hosts its really big. It takes around 20 seconds between the click of a button and the output being shown on the local machine.

I have rtt min/avg/max/mdev = 84.393/86.858/91.297/3.163 ms latency between this two hosts, and the SSH connection gives me around 1.2MiB/s, which I think it should be more than enough :\

I'm using -YCX, and I've experimentally tried with and without Y and C (openvpn already compresses stuff with lzo), as well as diferent ciphers, with similar results.

I'm starting to think it might be the GTK theme, which could be really heavy or something.

Does anybody know if this is normal, and what could I do to get less latency? (3-5s could be bearable, but 20 is way too much)

The problem with forwarding contemporary X (not that "old" X when its network transparency was invented) is with font smoothing: to properly smooth each glyph of text rendered on some surface, the X server has to get the bitmap which is located under the bounding box of that glyph from the client which wishes to render that glyph. (This is neede for the smoothing algorythm to work properly as it takes the context on which the glyph is rendered into account.)

Hence with contemporary GUI toolkits the amount of traffic shoveled between the X server and its clients is huge: you can see this by enabling TCP in your local X server (these days they are typically started with -nolisten tcp) and force some GTK- or Qt-based X client to talk to the server via TCP:

$ DISPLAY=localhost:x11 /usr/bin/that/x-app

(see grep x11 </etc/services for the X server's standard ports). You will immediately notice how sluggishly that client behaves even though the X traffic is not leaving the local host: that's simply because normally the X traffic is carried over a Unix-domain socket which basically just copies bytes between buffers in memory, thus having quite low overhead, and now it traverses the full TCP/IP stack with all its queues and complicated logic. Now consider what happens when this traffic is sent in your case—wrapped in three layers of data transfer protocols: SSH tunnel carried by VPN tunnel carried by TCP/IP carried by the wire.

As to what to do about this, I'm not so sure.

With mosh being out of the game, I'd try playing with the IPQoS option of the OpenSSH client.

Another approach is to attack the problem from another angle: try VNC-based access to your application. Options vary here:

• You can start simply by exporting the whole display via x11nvc or something like this.
• Use software packages which are able to "export" specific application or window—Xpra and winswitch.
• Try a more heavy-weight though complete solution such as X2Go.

I'm facing a problem that really looks like yours : some gtk applications (e.g. meld) are slow to start over ssh, some others not (e.g. synaptic). I actually may be more precise on how slow it is :

$ echo "$TIMEFORMAT"
        %R
$ time meld --version
meld 3.20.0
        25.293

This is the exact same time as the other slow applications I've found (nemo, gedit).

Inspecting meld with strace, it appears that it is waiting for some event that never comes, and exists on a timeout that is exactly of 25s.

It is very likely that this problem is the same as the one that was reported here: https://bbs.archlinux.org/viewtopic.php?id=230036

This is a dbus problem - kind of session environment that is not set over ssh. Unfortunately, I don't know how to fix this.

The only workaround I've found with meld is to launch one in background before I can use it the normal way.

Edit

Found it ! Launch dbus simply and export reported variables:

$ dbus-launch 
DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-2EzslkNeji,guid=c9dec1622d6575f468559f8b5d9ee0e0
DBUS_SESSION_BUS_PID=4745

Set the above and export, then:

$ time meld --version
meld 3.20.0
        0.268

I'll put this in my startup script on remote and report here.

Follow-up

This is rather verbose and largely out of scope, but as I do like easy-to-try copy/paste solutions in the posts I see, I may except that you too do. I've tested the following as a session script to feed ssh with (e.g. ssh -X my@dark-side ~/bin/session):

#!/bin/bash

LAUNCH=(
    #   xload
    #   thunderbird
      dbus
      konsole
)

Tmp=$(mktemp -d)
mkdir -p $Tmp

KILLS=()
WAITS=()

echo "info: starting session" >&2

app_dbus() { # # Launch dbus and remember to kill
  # redirect error because of "create ~/.dbus/session-bus/" 
  # deal with failure by checking DBUS_SESSION_BUS_PID afterwards
  dbus-launch > $Tmp/dbus.sh 2>/dev/null
  . $Tmp/dbus.sh

  if [ "$DBUS_SESSION_BUS_PID" ] && 
       ps --no-header -o pid -p "$DBUS_SESSION_BUS_PID" \
          > /dev/null 2>&1
  then
    export DBUS_SESSION_BUS_ADDRESS DBUS_SESSION_BUS_PID
    KILLS+=( $DBUS_SESSION_BUS_PID )
  else
    echo "warning: failed dbus" >&2
  fi
}
app_konsole() { # # Launch konsole and remember to wait for
  konsole &
  WAITS+=( $! )
}

for APP in "${LAUNCH[@]}"
do
  app_$APP
done

rm -r $Tmp  # not needed anymore

wait "${WAITS[@]}"

echo "info: ending session" >&2

for ID in "${KILLS[@]}"
do
  kill $ID
done