Do I need a desktop to run a GUI?

I am setting up a light weight VM, following the guide in the Ubuntu Documentation

What I want to be able to do is launch a GUI application. I don't need / want anything on the desktop.

First I installed xorg. But I can't launch a gui. The next step is to install a "Window Manager". So I installed xfce4, and then startx launches a desktop.

What do I need to launch a gui but not launch a desktop?

If you want to control exactly what runs when you run startx you should create a .xinitrc file in your home directory. If you make .xinitrc something very simple like:

xfwm4 &
xterm

Then startx will run with only a window manager and a terminal.

From gdm you can select xterm (recovery console) and it will give you a simple terminal runing on X. From it, you can start any gui app without the full desktop, like:

nautilus $HOME &

Since gnome is not running, the themes won't work so you'll see the application running in simple gtk (ugly but functional).

Remember to add & at the end so it returns to prompt in case you want to start other applications.

Also, if you install evilwm

sudo aptitude install evilwm

and run it fromt that xterm, also with the &, it will allow you to move the windows with Alt+click

evilwm &
nautilus &
whatever_program &

If you start with a clean minimal installation, install openbox, then run startx on startup. After this, you just need to right click the desktop to access the Openbox menu which should allow you to launch a xterm or any installed applications.

Openbox is very light and has little dependencies.

I asked this question over at unix.SE, and got the following answer:

If you build the VM and use it as a server, then ssh with x11forwarding (e.g. ssh -X), then the system does not need a desktop or window manager, although the host will.