Is there a variant of Ubuntu that comes with none of the bundled software?

Ubuntu minimal install CD or Ubuntu server. If you then run this script (source):

sudo apt-get -y install gnome-core gdm network-manager-gnome \
  fast-user-switch-applet  human-theme x11-xserver-utils \
  tangerine-icon-theme gnome-themes-ubuntu ubuntu-artwork \
  jockey-gtk gnome-screensaver gnome-utils    

you get the most minimalistic gnome you can get.

From that link

... memory usage from around 250-350mb to about 80-90mb. All of the themes are there, the network-manager, gedit, restricted hardware tool, etc. ...

The closest that you can get is Ubuntu Core. It is currently in development and will be shipped with Oneiric in October 2011. As per Canonical Engineering Services, Ubuntu Core is "ideally suited for the next generation of embedded devices where Internet connectivity is key". You can use Ubuntu Core to create a Ubuntu Server, Desktop or a customized OS. Ubuntu Core has apt-get, so you can use it to install only the software that you need and nothing else.

You could take a look at Minimal CD. But then you would have to sit and get all the packages including Gnome and stuff from the command-line.