mount overlayfs on “/” before systemd starts

I'm working on an embedded device with an arm cortex A9. The kernel version is 4.16.0 and is generated by buildroot 2018.05. The storage memory is an mmc with several partition. U-boot reads the uImage in one partition and starts the kernel. The filesystem is included in the kernel image and is loaded at startup in an initramfs (config : BR2_TARGET_ROOTFS_INITRAMFS).

I want to use an overlayfs to use a folder (in the mmc) as a 'user' filesystem, mounted onto /.

I already tried succesfully to overlay /etc with a directory named /data/etc by adding these lines to the /etc/fstab :

/dev/mmcblk0p1 /data auto defaults 0 1
overlay /etc overlay x-systemd.requires=/data,lowerdir=/etc,upperdir=/data/etc,workdir=/data/work/etc 0 1

This is working, but the overlay is mounted after systemd has already launch its sevices. So the network configuration for example is the one in the uImage and not the one in my user filesystem. I'd like to mount the overlay before lauching systemd, just after / is mounted.

I have understood that at startup, the system creates a first memory space in ram and then extracts the cpio image in it. Then / is mounted there and the system launches the first program : systemd:/sbin/init with the PID 1. Is my understanding right ?

I have read lots of article and Q&A website but I still do not understand where I have to make the changes to perform my overlay at boot time, before systemd startup. What are the files that contains the mount / operation ?

Solution 1:

I have understood that at startup, the system creates a first memory space in ram and then extracts the cpio image in it. Then / is mounted there and the system launches the first program : systemd:/sbin/init with the PID 1. Is my understanding right?

Close but not entirely correct, you're missing a few steps in the middle. The actual sequence – if you use an initramfs – is:

1. Linux mounts a memory space ("tmpfs" filesystem) on / and extracts the cpio image in it.

2. Linux runs the /init binary that was present in the cpio image. This becomes PID 1.

3. The initramfs init process mounts your main rootfs somewhere like /new or /real. Usually it doesn't use the fstab, only the information supplied in kernel command line.

4. The initramfs init process uses pivot_root() and chroot() functions so that the rootfs becomes mounted at /. (The original in-memory / becomes inaccessible.)

5. The initramfs init process executes the "real" init system (i.e. systemd) from /sbin/init. Since it uses exec() to do this, it remains the same PID 1 before and after.

6. Systemd starts launching services and mounting /etc/fstab entries.

So in your case, you want to edit the initramfs so that it will directly pivot into the overlayfs in steps 3–4, instead of using the 'raw' root partition. This shouldn't be difficult as the initramfs /init file is commonly a basic shell script.