X11 - Forwarding and efficiency

I was under the impression that the whole screen is forwarded, no matter what's going on. Then X11 forwarding should be application-agnostic.

No, it's actually the opposite. The reason X11 forwarding is called "X11 forwarding" is because it transports the actual X protocol messages used by applications to render their windows on the "X server" (typically Xorg). Those messages are commands for creating/moving windows, drawing text and graphical primitives (lines/rectangles), drawing bitmaps, etc.

You could say it's conceptually the opposite of "full screen image" protocols such as VNC/RFB. I think it is somewhat comparable to Windows' RDP, which was also made for transporting GDI drawing commands.

So the reasons you see differences between programs are:

1. To quote the post that you've referenced, originally most X-based programs worked like this:

   Basically X11 doesn't send the screen to your computer, but it sends the display-instructions so the X-server on your local computer can re-create the screen on your local system.

   So when a program wanted to show a button, it just sent a few short commands – "draw a rectangle", "draw text", and perhaps some lines to make it look 3D.

2. Over time this changed, programs started doing the rendering by themselves, and many of those instructions became just "here's a bitmap which I already rendered, put this on screen" – very fast locally, but very slow over the network due to X11 lacking any image compression.

   This means programs built using modern toolkits are much slower over networked X11, even if it's something as basic as antialiased fonts.

   (In contrast, RDP has adapted over time and includes various forms of image compression such as JPEG and even H.264; you can often notice the compression artifacts while the full image is loading.)

3. Fortunately, most UI toolkits such as GTK implement damage tracking so only updated regions are re-sent. However, a few programs (such as several Firefox/Thunderbird versions), don't support this and request a complete re-render of the entire window, even if it hasn't really updated.

   That's another difference between programs – well-behaving ones are still quite usable over the network, but buggy ones can saturate a 100 Mbps link doing absolutely nothing useful.