XFCE4 Network Manager
XFCE hasn't it owns network manager.Use wicd instead (pygtk-based)
sudo apt-get install wicd
Xubuntu 10.04 comes with some GNOME applications such as the Network Manager, just tried on a fresh install:
Don't know what is used in Xubuntu 10.10, but I doubt it's much different. If you're not happy with Network Manager you can always switch to WICD as karthick87 suggested.