How do I make Ubuntu "power loss proof"?

I like to run a system unattended for every day 'kiosk' use.

However, the system may be powered off anytime without someone doing a proper shutdown before. By default, Ubuntu is not expected to be shut down by power loss. After rebooting after a power loss, fsck may be run to check the file system, and fsck may tell the system to reboot after it has finished. So even if no configuration data gets corrupted, the machine may not boot well after power loss. So I tested some workarounds:

1. change the root mount fsck priority to 0 (last field in fstab root entry) that prevents running fsck after every power loss. However, the the system may boot with root mounted read-only then, which is not expected by many services and results in a console login prompt instead the graphical login.

2. Replace errrors=remount-ro by errors=continue. This gives a bad feeling that further data loss may occur by an incosistent filesystem. However, with fsck enabled again, it should increase the system's boot propability after power loss. So I dropped 1).

3. Reconfigure grub2 to use a normal default option timeout after a failed boot. For that, i've added GRUB_RECORDFAIL_TIMEOUT=0 to /etc/default/grub.

However, it is hard to know if these hacks make the system power loss proof. Any ideas? Any more one can do?

Solution 1:

For kiosk application, the problem is solved by using a readonly root partition. In addition, any changes users make or save are undone at the next reboot.

To provide a writable root for most applications needs, overlayfs can be used to overlay the readonly partition with a writable ram tempfs.

There is a script that helps creating such a setup at the bottom of https://help.ubuntu.com/community/aufsRootFileSystemOnUsbFlash . The script contains the instructions to set this up:

• put the file in /etc/initramfs-tools/scripts/init-bottom/root-ro
• sudo chmod 0755 root-ro
• sudo update-initramfs -u

I suggest to add GRUB_RECORDFAIL_TIMEOUT=0 to /etc/default/grub and run sudo update-grub too, otherwise the boot menu may appear without timeout.

After that, reboot. The machine then starts in readonly mode, which can be checked by mount. Any change applied will vanish on next reboot. To make any changes, install software and updates etc. you just need to enter the GRUB menu, press e to change the boot command lines, and append disable-root-ro=true to the line starting with kernel. Press F10 to continue booting. You can then use mount to confirm root is mounted writable as usual. Make your changes and reboot, the system is readonly again.

Solution 2:

Some times after an unclean reboot (say power loss or reset button pressed or even a kernel panic), the system will not boot asking you you to press "y" for an fsck to repair the partition.

If you would like to avoid this, edit /etc/default/rcS and change:

FSCKFIX=no

To:

FSCKFIX=yes

This will ensure this repair is ran automatically without prompting you.

The downside may be that you can lose data and you may want to take the hard drive out and clone it first if there is anything critical on it that isn't backed up.

For example if your hard drive controller has a fault and fsck falsely identified the partition as broken and tries to repair it, that can lead to data loss that can otherwise be avoided. I have never experienced this myself and have dealt with close to a thousand servers over the past 7 years or so - but still this is something to keep in mind.

Solution 3:

As of today, for a kiosk read-only solution one can also install the package overlayroot by

sudo apt-get install overlayroot

which will easily provide a complete solution like the one of the approved answer. It also allows for the beneficial command

sudo overlay-chroot

which will log into an shell having the former read-only base disk mounted on /. It is then possible to make any changes to the protected system one like and for example use apt-get to install packages to the former readonly disk. However, after exiting the shell a reboot is strongly recommended as temporarly overwritten files on RAM may obscure newly installed ones.