How can I strip down Ubuntu?

Start from Ubuntu Server and build up. It uses the same repos as the desktop distribution and you can install various desktop configurations.

I just suggest you stay away from the big metapackages like ubuntu-desktop or you'll end up will the full blown distro.

If you only want to strip down (rather than building up) you're going to have to remove ubuntu-desktop (which depends on loads of things) and convert all the automatically met dependencies to manually installed. If you don't aptitude will nuke them as it (amongst other tools) will automatically clean up "obsolete" packages -- those are packages that fulfil no dependency or user choice.

Use the Ubuntu Server media, and choose the option to build the minimal installation. People call this "JeOS", although I think the term "JeOS" may have been deprecated. The Ubuntu Server Guide says:

While installing from the Server Edition ISO (pressing F4 on the first screen will allow you to pick "Minimal installation", which is the package selection equivalent to JeOS).

Ubuntu has a mini.iso which is a totally stripped down version of ubuntu that you can use to build up yourself. It's about 28MB or so.

As of this reply the latest version is 12.04 Precise, so you can just replace the distro release name in the url to get the mini.iso for that release.

For 64bit: http://archive.ubuntu.com/ubuntu/dists/precise/main/installer-i386/current/images/netboot/mini.iso

For 32bit: http://archive.ubuntu.com/ubuntu/dists/precise/main/installer-i386/current/images/netboot/

You can then install your GUI/Desktop environment of choice using apt-get and so on with whatever packages you wish to install.