Symbolic Links and Multiple Mount Points

I am sure this may seem like a simple question, but I cannot seem to ask Google the correct question for the life of me. In short, I am having trouble understanding how symbolic links function when a given filesystem is mounted at multiple points.

I have been following the FreeBSD Handbook in setting up application jails on my little home server. Everything has been going along smoothly, until I encountered the second to last step in setting up the master template. More specifically, when I am to issue the following commands:

# cd /home/j/mroot
# mkdir s
# ln -s s/etc etc
# ln -s s/home home
# ln -s s/root root
# ln -s ../s/usr-local usr/local
# ln -s ../s/usr-X11R6 usr/X11R6
# ln -s ../../s/distfiles usr/ports/distfiles
# ln -s s/tmp tmp
# ln -s s/var var

Now, these commands will execute just fine. What I am concerned with is the why. I know that the master template will be mounted as read-only so mounting one file set for multiple jails is no problem. My confusion is how the operating system knows which service's data directory (/s directory, mounted read-write for each service) to use given that the master template only maintains one set of links? Say I had two services, www and ftp. Each will use the master template at /home/j/mroot (mounted read-only at /home/j/) and have their own data directories (found at /home/js/; mounted read-write at /home/j/). Based on the links, how is the filesystem able to know that it should write to the www data directory within one jail and to the ftp data directory from another?

Also, why are the "../" pathing operators required from some of the links? It is my understanding that those would back you right out of the current directory and to elsewhere in the filesystem, whish is likely an invalid path?

I apologize if these are simple questions. My searching has not turned up anything, but I certainly suspect that I may be simply asking the wrong questions. For reference, here is the link to the particular FreeBSD handbook page.

http://www.freebsd.org/doc/en/books/handbook/jails-application.html

Thank you in advance for your assistance.

Solution 1:

Alright... Let's attempt an answer to this question.

The setup and master/skeleton directories

The "root" directory of the jail system is /home/j. It contains 3+ subdirectories:

/home/j/
   |- mroot/       # The master (read-only) root directory
   |- skel/        # The master (read-write) directory
   |- js/          # Each jail will have a subdirectory here
                   # where its writable directories point to
   |- <jail>/      # Each jail will have its own directory

Now the mroot/ directory contains the base system as you've done a make installword to it. It will keep the read-only directories such as bin/ and dev/. All writable directories (e.g. etc/) will be moved to the skel/ directory. This is done in step 3 of section 14.5.1.

As of now, there is no /home/j/mroot/etc/ folder (since you moved it), so we'll create a symlink for it. First we'll create a new subdirectory s/ (short names can be too short and thus become confusing too easily). This is done in step 5. The setup of the mroot/ folder will now look a bit like this.

/home/j/mroot/
   |- s/             # Empty directory for writing to
   |- etc            # Symlink to s/etc/
   |- home           # Symlink to s/home/
   |- ...            # More symlinks are created

These symlinks will now be broken as those directories do not exist. They will be created when a jail is created.

Creating a jail

Now the third folder (/home/j/js/) will come into play. In here we will create a subdirectory for each jail to contain the writable content.

/home/j/js/
   |- mail/
   |- ns/
   |- www/

The skeleton directory (skel/) is copied to each of these subdirectories.

Using nullfs mounts we'll now tie everything together. Each jail gets another directory. These are nullfs (read-only) mounts to /home/j/mroot (step 1 of 15.5.2).

/home/j/
   |- mail/
   |- ns/
   |- www/

As they are mounts, going into such a directory will bring you to /home/j/mroot/ instead, though your path won't change. Each jail will have a writable /etc/ folder which seems to be /home/j/mail/etc/ but due to the above created mount, it will actually be /home/j/mroot/etc/.

However, remember from the very beginning that this folder doesn't exist anymore. It was moved and instead a symlink was created. This symlink pointed to a not-yet-existing subfolder of /home/j/mroot/s/. The second nullfs mount will fix this.

When creating a mount for /home/j/ns/s to point to /home/js/ns. This destination contains the copy of the skeleton directory containing all writable files. The source, /home/j/ns/s, will actually be /home/j/mroot/s due to the first mount.

Example

When a jail accesses a file in /etc/ (which should be writable), it is actually accessing /home/j/<jail>/etc/ on the host. However, this is a mount to /home/j/mroot/etc/ due to the first mount. But this "directory" is not a directory at all, it's a symlink to /home/j/mroot/s/etc which is, due to the second mount, in fact /home/js/ns/etc/.

Conclusion

I hope I succeeded in clarifying the setup a bit. I will reread my own answer once I've cleared my mind again because my head is still spinning from trying to figure this one out myself!

Solution 2:

This is not exactly what you asked for but it might make your life easier with ezjail. Ezjail auztomates the process of creating jails and and the templating of them.