How do you make a USB drive that is bootable for most PCs?

I apologize for the title. I had no idea how to put my situation into a single legitimate question for the title. I will gladly change it if someone can give me a good suggestion.

My goal:

I want my custom Ubuntu USB drive (NOT a Linux live copy but an actual installation to the USB drive) to be able to boot on multiple machines the same way that a Linux live USB does. The Ubuntu USB boots on all of my PCs without issues regardless of UEFI settings.

I have done this before years ago (I didn't have to deal with UEFI back then) and I have it working now to an extent.

What I have:

• I have an Ubuntu live USB that contains the Ubuntu installer
• I have a USB drive with a GPT partition table and the following layout:
  • A data partition (NTFS) that allows the drive to act as a normal USB drive when inserted into a running system
  • An ESP partition (fat32, 500MB) with the boot and esp flags that is used for UEFI boot (or maybe it isn't in my case?)
  • An OS partition (ext4, 30GB) that I have Ubuntu 20 installed to
  • A swap partition
• A 2 year old Dell Precision
• A 3 year old Dell Precision
• A 10 year old Dell Precision

What I've done:

1. I attempted to install Ubuntu to my USB using my 2yr PC but was unable to due to the use of RST. Since this was my work PC and I have others on hand I moved on.
2. I used my 10yr PC and successfully installed Ubuntu to my USB. Unfortunately the USB would not boot on my newer PCs without putting them into legacy boot mode.
3. I added an ESP partition to my USB and used my 3yr PC to Install Ubuntu. I ran into two bugs in the Ubuntu installer. The installer will ask where to place the bootloader in UEFI mode despite what Ubuntu's documentation says. Apparently this is a known bug and yes I double checked that I was in UEFI mode. The next issue is that if there are multiple ESP partitions (on multiple drives) the installer will use the first one it sees even if you tell it not to and to use the other one.
4. I removed the hard drive from my 3yr PC and installed Ubuntu to my USB drive as if it were the actual hard drive.

Now my USB drive will boot on all of my PCs but there are some issues:

• When it boots on my 2yr and 3yr PCs it complains that the USB does not have an entry on the ESP partition on the PCs hard drive. Not a huge deal, it will add one for you. Its annoying though and it doesn't look good.
• It will only boot on my 10yr PC but only if I manually select it from the boot menu (yes I've set the boot order). It also doesn't seem to be stable (there are screen blackouts). (I understand the blackouts could be unrelated)

Summary: The Ubuntu live USB has a single fat32 partition with the boot and lba flags set. It will boot (and run) on all of my PCs without issue and without having to change any settings. How can I make my USB able to boot this way? I feel like this should be possible since the Ubuntu live USB does it. However my knowledge of how Linux live works and how UEFI works novice at best.

Solution 1:

Generally speaking, you should use a MBR/MSDOS partition table for maximum (backward) compatibility. It does NOT prevent you from creating an EFI system partition.

You can then install both the i386-pc and the x86_64-efi variant (well, perhaps even i386-efi for those rubbish machine with 32-bit UEFI) of grub to the drive by specifiying --target explicitly when grub-install.

Solution 2:

Why not put Ubuntu Live with persistence on the USB (or better, external SSD, because most flash USB drives are abysmally slow when running the OS)? Just set boot-order on each machine to boot from USB first.

A persistent live device is a full Ubuntu installation which can be modified as needed and it will retain any changes. For example, if you install dconf-editor while using the USB drive on machine A, it would also be available on B and C. If you edit dconf on B, those changes persist for A and B. A LibreOffice document saved on C should be editable on A and B.

Persistence is also explained at wiki.Ubuntu and at Linuxhint. This seems to be what you want to achieve, if I understand your question properly. Let me know if that works.