Running X/Unity inside LXC container

I have already created a number of LXC containers in my 12.04 machine. I need to run GUI applications inside them (mostly Firefox).

Is it correct to presume that it will work out of the box? If not, what do I need to do to make it work?

Your lxc guest will need a desktop but whether or not a full ubuntu-desktop or not depends on your use. Then you might wanta means to access a desktop in the container (VNC, xfreerdp/xrdp, x2go). In my lxc containers I install x2go server then use an x2go client (mac, pc, linux) to log into the lxc desktop. x2go clients support unity, gnome others so you can choose.
Also, you may need/want to modify ssh default port etc for the container (/etc/ssh/sshd_config) to something other than default but that may mean you may need to make a change on your router firewall also.