Migrating from unencrypted Ubuntu 16.04 to LUKS Encrypted install of 16.04

My company is working on implementing some Ubuntu Systems in a few of their new testing efforts and I have been tasked with an encryption solution for them. Having been a primarily Windows user with just a working knowledge of Ubuntu and Linux/Unix in general, this has been a bit challenging but intriguing. I know many in place encryption services work w/ Windows but have noticed a lack of these for Ubuntu and after reading 20+ posts on here about it, decided fresh install w/ full disk encryption is the only realistic option. That being said, some of the systems have already been in use, and would not be able to just be wiped and fresh installed w/ encryption. I would need ALL settings and changes as well as existing files copied over/backed up. Therefor, it's not as simple as copying over /home. I was curious what would be the best course of action for me, I have a full copy of the Partition (used gparted to make a quick copy to external) for my test machine and a fresh install of Ubuntu w/ LUKS. I am familiar with the basics of clonezilla, but I don't believe that will help me here from what I gather.

tldr: I need to move a full install of Ubuntu 16.04 to an encrypted disk. Some machines will be TPM enable and some will not have them, I need to know the best course of action for each. Thank you for helping a new Admin out.

BONUS if you can describe a way that doesn't require outside network access, but not required. I.e. if it needs a package, I can download it on a test PC to USB/CD/DVD and install the package offline.

Solution 1:

Described below is a procedure I have successfully used multiple times. It works for Ubuntu 16.04 and 18.04. Unlike the solution proposed by @Arno in their answer, it doesn't require manual editing of kernel boot options. Instead, it relies on update-grub generating proper config based on /etc/crypttab - a more canonical (pun unintended) solution which is also used by Ubuntu when installing with its default, LVM-based encryption. (this solution doesn't use LVM nor encrypted /boot)

I am not responsible for any data loss or other potential unpleasant consequences of following this guide. Make sure you have reliable backups before doing anything.

I'm not testing this guide while I'm typing this. It's based on a (tested) blog post by myself on somewhat similar case and some details I can remember.

This guide assumes that:

Ubuntu is already installed
The system boots with (U)EFI, not BIOS/CSM
/boot is on a separate partition¹
You can boot a live media of Ubuntu 16.04/18.04 or Pop!_OS 18.04 (from USB, PXE, DVD, whatever) ²

If your /boot is not on a separate partition, extracting it is pretty straightforward: create a 200-500 MB partition anywhere, format it to ext4, copy current /boot content, add /etc/fstab entry, update-grub, reboot.

1. Backup

You have to back up current system partition(s). The easiest way to do this is to use Clonezilla. It's user-proof, no manual needed - just follow the instructions.

2. Shrinking partitions

LUKS headers take some of the space on a partition. Clonezilla/partclone can't restore images to devices smaller than source ones, so you wouldn't be able to restore image of unencrypted partition to encrypted container, because it's a tad smaller.

We then have to shrink partition(s) to be encrypted a little bit. Shrink them by 10 MB or more, just to be safe. The easiest way to do this is to use GParted. It's available on Ubuntu live media.

3. Making images of shrunk partitions

Either do this with Clonezilla again, or already familiarize yourself with partclone, which Clonezilla is using by default. We'll be manually restoring images with partclone later.

Cloning to an image with partclone:

sudo partclone.ext4 -c -s /dev/sda2 | pigz -0 > /mnt/backup/sda2.ext4.ptcl.gz

partclone.ext4 is one of partclone's binaries dedicated to working with ext4. Each supported filesystem gets its own partclone binary.
/dev/sda2 is obviously the partition you want to clone. I prefer to refer to partitions by their partlabel, eg. /dev/disk/by-partlabel/os - much cleaner IMO. /dev/sda2 is more recognizable, though.
pigz is multicore gzip. -0 tells it to favor speed over effective compression.
/mnt/backup here represents some external location where you want to store the image. If you've just used Clonezilla before, it can still be mounted under /home/partimag. If you want to mount an SMB share: sudo mount -t cifs -o username=gronostaj //192.168.1.90/Backup /mnt/backup (asks for password interactively)

4. Enlarge partition(s) to original size

We want to have that extra space for LUKS headers back, don't we? Resize partitions back to their original sizes.

5. Formatting to LUKS

This is the moment where you loose your original data. Make sure your backups are okay.

Format system partition(s) (except for /boot) to LUKS:

sudo cryptsetup luksFormat --type luks2 /dev/sda2

Open created container(s):

sudo cryptsetup open /dev/sda2 os

Make sure that entire encrypted container looks like random garbage and your old data isn't still readable:

sudo dd if=/dev/zero of=/dev/mapper/os bs=1M

(this will overwrite decrypted content of container with zeros, but encrypted content will look like random garbage)

6. Restoring images

Manually run partclone to restore images:

cat /mnt/backup/sda2.ext4.ptcl.gz | pigz -d | sudo partclone.ext4 -r -o /dev/mapper/os

If you took the easy path and made "shrunk" backups with Clonezilla, look in its output files, you'll easily figure out which are the partclone sources. Unless you set huge chunk size they will be fragmented, you have to cat them together before piping to pigz.

You should also adjust filesystem geometry to fit entire partition:

sudo resize2fs /dev/mapper/os

7. Enabling encryption support

First, chroot into the just-restored OS:

mkdir /mnt/os
sudo mount /dev/mapper/os /mnt/os
cd /mnt/os
mount --bind /etc/resolv.conf etc/resolv.conf
mount --bind /dev dev
mount -t tmpfs tmpfs tmp
mount -t sysfs sys sys
mount -t proc proc proc
sudo chroot .
mount -a

This terminal is now working on your installed instance of Ubuntu, not the live one.

Install cryptsetup:

apt update
apt install cryptsetup -y

It should create a file /etc/crypttab. If it didn't, don't worry, create it manually. Edit this file and add entries for partition(s):

os /dev/sda2 none luks

Save and quit editor. Rebuild initramfs:

update-initramfs -u -k all

Update GRUB entries:

update-grub

8. Removing extra password prompts

If you have more than one encrypted partition, then you have to type in the password for every single one on boot. LUKS however lets you add additional keyfiles that can be used to unlock partitions instead of password. You can store these keyfiles on encrypted / partition and use it to unlock subsequent ones.

Create a random keyfile:

dd if=/dev/urandom of=/luks.key bs=4096 count=1

Add it to non-root partition(s):

cryptsetup luksAddKey /dev/sda3 /luks.key

Add crypttab entries:

home /dev/sda3 /luks.key luks

Rebuild initramfs:

update-initramfs -u -k all

¹ Why I use separate /boot:

It's easier this way ;)
GRUB doesn't support LUKS2 yet, so /boot must not be on LUKS2 encrypted partition
If you want to LUKS-encrypt /boot and LUKS2-encrypt /, then you have to enter password twice or embed keyfile in initramfs - too much of a hassle IMO, because...
Encrypted /boot alone doesn't make you any less vulnerable, because even if your /boot is encrypted, your EFI System Partition cannot be, so attacker can tamper with it and eg. use their custom malicious kernel/initramfs instead of your one from /boot. (to fix this, you have to build a self-container GRUB binary and sign it with your private key, then use UEFI Secure Boot to verify it)

Define your own threat model and decide what level of security (and against what) you need.

² You can try your luck with other Ubuntu versions or rescue CDs such as GRML, but your mileage may vary. I've experienced this procedure failing just because live media didn't match the system.

Solution 2:

You could image the root partition, format it with LUKS, mount it and overwrite it with the image. Then a few changes to GRUB would be needed so it would decrypt it at boot. I haven't tested the steps below, they are just a lead.

I assumed that /boot is on sdb1, the root partition is on sdb2 and /media/hdd is an external drive. We are on a Live USB in an admin prompt.

Encrypt the root partition

dd if=/dev/sdb2 of=/media/hdd/diskimg bs=1M
cryptsetup luksFormat /dev/sdb2
cryptsetup luksOpen /dev/sdb2 enc_root
dd if=/media/hdd/diskimg of=/dev/mapper/enc_root

Mount the partitions

mount /dev/sdb2 /mnt
mount /dev/sdb1 /mnt/boot

Configure GRUB

Note the partition's UUID by running:

blkid /dev/sdb2

Edit /mnt/etc/default/grub, find the line that says GRUB_CMDLINE_LINUX_DEFAULT="quiet splash". Change it to the following, replacing device-UUID by the UUID you noted in the previous step.

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash cryptdevice=UUID=device-UUID:enc_root root=/dev/mapper/enc_root"

Then update the GRUB config:

mount --bind /dev /mnt/dev
mount --bind /sys /mnt/sys
mount --bind /proc /mnt/proc
chroot /mnt
update-grub
exit

And unmount everything:

umount /mnt/dev
umount /mnt/sys
umount /mnt/proc
umount /mnt/boot
umount /mnt

And reboot. A few pitfalls:

I haven't tested any of this and haven't played with Linux partitions for a while, so I'm almost sure I forgot or messed up something. Wait for people to proofread and fix it.
This assumes that /boot is on a separate partition, it wouldn't work if everything was on the same partition.
On a UEFI system, don't forget to also mount the EFI partition where it belongs.
dd will copy everything including free space, a quicker method would be to copy only the files in the partition.

Good luck!