How to remove Kernel/LTS Enablement Stack?

this question regarding the LTS Enablement Stack has actually two parts:

1. I installed Linux 3.5.0-x on Precise 64bit via sudo apt-get install linux-generic-lts-quantal xserver-xorg-lts-quantal . As it turns out, VirtualBox does not work yet on the new kernel (EDIT: This is fixed now.). Therefore I want to remove it from the system. As expected removing the meta-packages via sudo apt-get install linux-generic-lts-quantal xserver-xorg-lts-quantal is not enough. What packages do I need to remove manually? For the kernel I would do (as of now): sudo apt-get remove linux-image-3.5.0-25-generic . However, regarding xserver-org I am lost. How can I switch back to the original xserver?
2. I installed a system with the 12.04.2 installation media. There I have also the 3.5.0-x kernel, because that's the default for this point release. How can I downgrade to the kernel (and xserver?) to the 3.2.0-x series?

Thank you!

Important: The below instructions only work if you have one of the pre-enablement stack kernels installed still, e.g. 3.2.0-38.

With the Enablement Stack installed run 'uname -r' to check the current kernel version. Substitute it into the command below:

sudo apt-get purge linux-generic-lts-quantal xserver-xorg-lts-quantal linux-headers-generic-lts-quantal linux-image-generic-lts-quantal linux-image-3.5.0-25-generic linux-headers-3.5.0-25

Re-install important components (including X itself!) that are removed due to dependencies by the above commands:

sudo apt-get install xserver-xorg xserver-xorg-input-synaptics

Fix the steam client:

sudo apt-get install libgl1-mesa-glx:i386

Finally, reboot your computer.

IMHO the enablement stack isn't ready for release. In my experience Virtualbox, Samba, and the Steam client all break with the Enablement Stack installed. There's a workaround for Virtualbox but none for Samba or Steam so far (March 2013).

In Trusty/14.04 there are a few more packages that need to be removed, or apt-get will complain.

sudo apt-get remove linux-generic-lts-utopic xserver-xorg-lts-utopic libgl1-mesa-glx-lts-utopic libegl1-mesa-drivers-lts-utopic

Then install xserver-xorg if not installed, reboot to old kernel version, remove 3.16 kernel(s) and you are good to go.

If you use Steam, you need to install two packages:

sudo apt-get install libgl1-mesa-dri:i386 libgl1-mesa-glx:i386