Ubuntu Server - How to Remove Desktop
Solution 1:
ubuntu-unity-desktop is a metapackage that has dependencies on other packages. Simply removing the metapackage doesn't remove the dependencies.
Run sudo apt autoremove --purge after you remove the metapackage, because the other dependencies weren't manually defined for installation but were instead included because ubuntu-unity-desktop needed them, you need to autoremove the 'now not needed' dependencies pulled in.