Ubuntu 18.04 LTS x11vnc no longer works

I have recently changed from Ubuntu 16.04 LTS to 18.04 LTS, as I needed a newer version of kvm/qemu.

With 16.04 I was able to easily get the x11vnc service (daemon) running by following the instructions from "Community Help Wiki" : https://help.ubuntu.com/community/VNC/Servers

This has a script required to set up the x11vnc daemon service, which lets you log into a machine remotely without having to first be logged in locally. I require this as I have my machine running as server and located in the basement and I always access it via VNC, not locally.

Initially I did nothing with my clean 18.04 install other than adding the systemctl script exactly as per "Community Help Wiki" instructions: "Have x11vnc start automatically via systemd in any environment (Vivid+)".

On finding that this no longer worked I then did the following based on some searching:

1. Disabled "Wayland" display manager by editing /etc/gdm3/custom.conf and setting WaylandEnable=false in this script:

   [daemon]
   # Uncoment the line below to force the login screen to use Xorg
   #WaylandEnable=false
   WaylandEnable=false  <--- HERE
   

2. Changed the /lib/systemd/system/x11vnc.service ExecStart command to use a different xauth directive as in 18.04 it appears that there is no automatically generated $HOME/.Xauthority file which can be found via the -xauth guest directive:

   From:

   ExecStart=/usr/bin/x11vnc -auth guess -forever -loop -noxdamage -repeat -rfbauth /home/USERNAME/.vnc/passwd -rfbport 5900 -shared
   

   To:

    ExecStart=/usr/bin/x11vnc -auth /run/user/120/gdm/Xauthority -forever -loop -noxdamage -repeat -rfbauth /home/USERNAME/.vnc/passwd -rfbport 5920 -shared
   

I did this based on some reading and testing on Xauthority, which indicated that the location of the .Xauthority token is now given via $XAUTHORITY environment variable.

To find the value of this I run the following "find" command to find which processes has an XAUTHORITY environment variable defined.

NOTE: this uses the linux /proc/<procid>/environ file structure to search through the process environment variables,

cd /proc
sudo find . -maxdepth 1 -type d -exec sh -c "(test -f '{}'/environ && cat '{}'/environ | tr '\0' '\n' | grep XAUTHORITY= )" \;

This returned two different results:

• XAUTHORITY=/run/user/120/gdm/Xauthority
• XAUTHORITY=/run/user/1000/gdm/Xauthority

I then used the following to find the corresponding process ids:

sudo find . -maxdepth 1 -type d -exec sh -c "(test -f '{}'/environ && grep -aH XAUTHORITY= '{}'/environ )" \;

The corresponding processes for these are:

240 tty1     Sl+    0:00 /usr/lib/gnome-session/gnome-session-binary --autostart /usr/share/gdm/greeter/autostart
14923 tty2     Sl+    0:00 /usr/lib/gnome-session/gnome-session-binary --session=ubuntu

The first of these appears to be associated with the login greeter screen, while the second is the user desktop.

A further check of the environment variables shows that one has USER=gdm and the other has USER=<ME>

The problems is that if I use the "greeter" auth location, then I get prompted to provide a password which is then followed by black/blank screen. If I use the user auth location, then I do not get any client connection at all as the status returns an error that it is unable to open the Display:

13/05/2018 16:19:14 *** XOpenDisplay failed.

So it seems that you get caught by the change in xauth mechanism.

Can someone please provide some guidance on this?

Ubuntu 18.04 x11vnc users.

Here is a "hack answer" which allows you to get VNC access without being logged in.

I say hack as it involves having 2 x11vnc services running.

The first is to allow login via DISPLAY=:0 and the second is to use VNC to get access to desktop via DISPLAY=:1

To achieve this I used the following 2 daemon scripts:

The first is: x11vnc-login.service just for login greeting

[Unit]
Description=Start x11vnc-login at startup.
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/bin/x11vnc -auth /run/user/120/gdm/Xauthority -forever -loop -noxdamage -repeat -rfbauth /home/<ID>/.vnc/password -rfbport 5922 -shared -display :0
[Install]
WantedBy=multi-user.target

The second is: x11vnc.service for desktop:

[Unit]
Description=Start x11vnc at startup.
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/bin/x11vnc -auth /run/user/1000/gdm/Xauthority -forever -loop -noxdamage -repeat -rfbauth /home/<ID>/.vnc/password -rfbport 5920 -shared -display :0
[Install]
WantedBy=multi-user.target

The installation and enablement of scripts is as per documentation on community help site.

This is a horrible hack but gets me running for the time being until a proper solution is found.

In use I first open up VNC session to port 5922 and do login. Once logged in you end up with a black screen. So you then open up VNC session on port 5920 and voila, there is your desktop. Still easier for me than having to go to where the server machine is running...

Obviously what is needed to have some script which does a preliminary search of running processes to see if a user is logged in and if so then just use the XAUTHORITY/DISPLAY info from the existing user section (as extracted from /proc/PROCID/environ, otherwise connect socket up to greeter screen with greeter XAUTHORITY/DISPLAY and then somehow move socket connection over to another x11vnc session using users desktop XAUTHOURITY/DISPLAY values.

A bit of complex fork/socket/file descriptor programming I suspect.

Other possibility is to figure out whether there is some way to get 18.04 display manager to behave as per prior 16.04 one.

I had the same issue and after some mucking around with x11vnc and gdm, I decided to simply switch back to lightdm:

apt install lightdm

That should bring up the display manager configuration. If not run:

dpkg-reconfigure lightdm

I now run my x11vnc server via supervisor with the following config:

$ cat /etc/supervisor/conf.d/x11vnc.conf
[program:x11vnc]
command=/usr/bin/x11vnc -xkb -safer -nopw -once -geometry 1024x768 -auth /var/run/lightdm/root/\:0 -display :0
user=root
autorestart=true

In addition, I run noVNC (also via supervisor) on top so I can access my desktop remotely just via a browser. Just in case you're interested, the configuration files look like this:

$ cat /etc/supervisor/conf.d/novnc.conf
[program:noVNC]
command=/opt/noVNC/utils/launch.sh --vnc localhost:5900
user=root

$ cat /etc/nginx/sites-enabled/novnc
upstream vnc_proxy {
    server 127.0.0.1:6080;
}

server {
    listen 443 ssl default_server;
    listen [::]:443 ssl default_server;
    include snippets/snakeoil.conf;

    root /var/www/html;

    index index.html index.htm index.nginx-debian.html;

    server_name _;

    location / {
            auth_pam               "Secure Zone";
            auth_pam_service_name  "nginx";
            proxy_pass http://vnc_proxy/;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            keepalive_requests 10000;

            proxy_read_timeout 61s;

            proxy_buffering off;
    }
}

You may want to wrap some additional security around it.