Is it possible to start LXC container inside LXC container?

Solution 1:

I'm going to dispel a few myths here.

This is just a bad idea. I'm sorry. – Jacob Mar 5 at 20:30

I don't see how this is a bad idea. It's really just a chroot inside a chroot. On one hand, it could possibly decrease performance in some negligible manner (nothing compared to running a VM inside a VM). On the other hand, it's likely to be more secure (e.g. more isolated from the root host system and it's constituents).

Do you actually have a real reason to do this? Please remember that questions here should be about actual problems that you face. – Zoredache Mar 5 at 21:52

I agree 100% with the poster's following comment. Furthermore, I think it's safe to assume that everybody who posts a question on here likely thinks that they have a real reason to do [ it ]..

I think, that lxc should be able to simplify VM migration(and backup+recovery too). But I'm not sure about cases, when there is no access to host OS(cheap vps for example). – Mikhail Mar 6 at 11:17

I actually came across this question back in June when I was first diving into LXC for PaaS/IaaS projects, and I was particularly interested in the ability to allow users to emulate cloud environments for development purposes.

LXCeption. We're too deep. – Tom O'Connor Mar 6 at 22:46

I laughed a little bit when I read this one, but that's not, at all, the case :)

Anyway, I eventually set up a VirtualBox environment with a stock install of Ubuntu 12.04 LTS Server Edition after reading all this, thinking that this was 100% possible. After installing LXC, I created a new container, and installed LXC inside the container with apt-get. Most of the installation progressed well, but resulted in error eventually due to a problem with the cgroup-lite package, whose upstart job failed to start after the package had been installed.

After a bit of searching, I came across this fine article at stgraber.org (the goodies are hiding under the "Container Nesting" section):

sudo apt-get install lxc
sudo lxc-create -t ubuntu -n my-host-container -t ubuntu
sudo wget https://www.stgraber.org/download/lxc-with-nesting -O /etc/apparmor.d/lxc/lxc-with-nesting
sudo /etc/init.d/apparmor reload
sudo sed -i "s/#lxc.aa_profile = unconfined/lxc.aa_profile = lxc-container-with-nesting/" /var/lib/lxc/my-host-container/config
sudo lxc-start -n my-host-container
(in my-host-container) sudo apt-get install lxc
(in my-host-container) sudo stop lxc
(in my-host-container) sudo sed -i "s/10.0.3/10.0.4/g" /etc/default/lxc
(in my-host-container) sudo start lxc
(in my-host-container) sudo lxc-create -n my-sub-container -t ubuntu
(in my-host-container) sudo lxc-start -n my-sub-container

Installing that AppArmor policy and restarting the daemon did the trick (don't forget to change the network ranges, though!). In fact, I thought that particular snippet was so important that I mirrored it @ http://pastebin.com/JDFp6cTB just in case the article ever goes offline.

After that, sudo /etc/init.d/cgroup-lite start succeeded and it was smooth sailing.

So, yes, it is possible to start an LXC container inside of another LXC container :)

Solution 2:

With Ubuntu 14.04 (trusty) you can simply add the following in the parent container config:

lxc.mount.auto = cgroup
lxc.aa_profile = lxc-container-default-with-nesting

reference: https://ubuntu.com/server/docs/containers-lxc (search for "nesting)

Make sure that you have pre-configured network before booting to avoid an long pause before the login screen appears!

HTH