Unable to boot Ubuntu Live USB Flash Drive with casper-rw persistent partition
So, I've spent the last day going through similar questions, and can confirm first that this is not a duplicate of:
- Ubuntu Live-USB Using a "casper-rw" Partition
- Persistent Ubuntu 14.04 USB fails to boot after creation of ext4 casper-rw partition
With that out of the way: I've created an Ubuntu 14.04.03 Live USB Flash Drive (32GB) using UNetbootin, as well as the Ubuntu Startup Disk Creator. In both cases, I was able to successfully deploy my Ubuntu ISO (which I've confirmed is intact via md5sum analysis). In both cases, persistence worked successfully (I had to manually enable it when using Ubuntu Startup Disk Creator, but UNetBootin adds the -persistent kernel argument automatically to syslinux.cfg), no problems.
I need to have a persistence file larger than 4GB, as I'm creating a bunch of these flash drives for Field Application Engineers (FAEs), and they need to be able to use them on the road to diagnose installations for custom systems my company currently supports. It's fine if they burn out due to excessive write cycles, and each FAE gets a dozen of them if needed for each trip.
Also, we need to keep the first partition on the disk as a FAT partition in case the FAE needs to drop files onto the disk from Windows, Mac, Linux, etc, as Windows machines refuse to mount FAT partitions on a flash drive unless they are the first and only FAT partition on the disk.
I've tried the steps in the following articles:
- How to make a persistent live Ubuntu USB with more than 4GB
- How do I get a live-USB to use a partition for persistence?
The instructions were clear:
- Create a Live USB flash drive with Ubuntu, leaving space for an extra partition.
- Delete the
casper-rwloopback file on the first FAT partition on the flash drive. - Use
gpartedto create anext2/ext3/ext4partition after the FAT partition, and give it a volume lable ofcasper-rw.
I followed all of the advised steps, but booting fails. When I attempt to boot from my newly-created flash drive, I am directed to a prompt instead of Ubuntu properly booting, though I briefly see the graphical Ubuntu startup screen animation.
dmar: IOMMU: failed to map dmar0
ACPI PCC probe failed.
BusyBox v1.12.1 (Ubuntu 1:21.0-1ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.
(initramfs) _
I've gone through the Ubuntu bug tracker, and it seems that this issue has been observed by numerous others:
- Bug #1241589: ubuntu 13.10 unable to boot on live usb (busy box - initramfs)
And that it has been solved, so I should expect the fix to already be in my Ubuntu 14.04.03 image:
- bug #1293811: casper no longer wants to boot off HDD devices
I've also tried updating my kernel boot parameters, by inserting LIVE-MEDIA=/dev/sdd1 (in my case) at boot time by hitting TAB, ie:
kernel /casper/vmlinuz.efi initrd=/casper/initrd.lz file=/cdrom/preseed/ubuntu.seed boot=casper LIVE-MEDIA=/dev/sdd1 -- persistent
This time it at least finds the boot partition and starts attempting to boot the Live USB disk, but seems to fail along the way, with a bunch of "casper" related warnings. I can't seem to save those logs though, so I'm unable to provide them at this time. My current default kernel arguments are:
label ubnentry1
menu label ^Try Ubuntu without installing
kernel /casper/vmlinuz.efi
append initrd=/casper/initrd.lz file=/cdrom/preseed/ubuntu.seed boot=casper quiet splash -- persistent
How can I go about resolving this issue?
Thank you.
As mentioned in the previous comment, this is related to the bug https://bugs.launchpad.net/ubuntu/+source/casper/+bug/1489855 "the sequence of mounting changed from root partition then persistent partition (15.04) to 1st persistent then root partition (16.04) for whatever reason."
There is a work around recently updated in the thread for the bug, which I will elaborate on below as the other thread doesn't seem to have as much visibility (comes up lower in the google search when I was trying to work it out).
Create partitions casper-rw and OS
Use universal USB installer, etc to set up the OS partition with casper-rw file large enough to be able to make changes - needed for later step to workaround bug (I did it with 2gb, but surely it requires less)
Boot up with the new USB. Open
/usr/share/initramfs-tools/scripts/casperwith root to edit (sudo). Change the function setup_unionfs() to the following : https://launchpadlibrarian.net/258626969/casper%20function.txtsudo update-initramfs -u(Had to uninstall cryptsetup to do this)Copy the generated initrd.img file from
/bootto another location.From another OS, delete the casper-rw file in the USB. Copy the initrd.img back to the live partition and change menu item to use the newly built initrd.img.
The next time you boot up from USB, it should now bootup from the casper-rw partition.
This worked for me for 16.04 after a long frustrating time searching for the solution!
Updates to Yu Jia's answer... Thanks Yu Jia for pointing to the right direction.
1) Use casper-rw file first to avoid the casper-rw partition issue. 500MB or less is good enough.
2) Boot up with the new USB. Open /usr/share/initramfs-tools/scripts/casper with root to edit (sudo). Change the function setup_unionfs() to the following : https://launchpadlibrarian.net/258626969/casper%20function.txt
3) sudo update-initramfs -u (warning from cryptsetup, but can be safely ignored.)
4) Copy the generated "initrd.img.4.4....generic" file from /boot to another location. Rename it to initrd.img.
5) From another OS, delete the casper-rw file in the USB.
6) Copy the initrd.img back to the live partition and change menu item to use the newly built initrd.img. e.g.
label ubnentry1
menu label ^Ubuntu 16.04
kernel /casper/vmlinuz.efi
append initrd=/newly_generated_initrd.img file=/cdrom/preseed/ubuntu.seed boot=casper quiet splash -- persistent
mkusb version 11 classic and version 12 alias mkusb-dus can create a persistent live drive automatically with all current Ubuntu versions and flavours, as well as with Debian Jessie.
mkusb uses a casper-rw partition
mkusb works with some linux distros derived from Ubuntu too. I have not tested with the newest Linux Mint versions, but it works with previous versions because the boot structure is/was similar enough to Ubuntu.
Install mkusb with the following command lines in Ubuntu and flavours of Ubuntu (Kubuntu, Lubuntu ... Xubuntu).
If you run standard Ubuntu, you need an extra instruction to get the repository Universe. (Kubuntu, Lubuntu ... Xubuntu have the repository Universe activated automatically.)
sudo add-apt-repository universe # only for standard Ubuntu
sudo add-apt-repository ppa:mkusb/ppa # and press Enter
sudo apt-get update
sudo apt-get install mkusb mkusb-nox usb-pack-efi
See these links,
mkUSB-quick-start-manual.pdf
help.ubuntu.com/community/mkusb
help.ubuntu.com/community/mkusb/persistent
help.ubuntu.com/community/mkusb/install-to-debian
Downloading and using mkusb-installer works with several other linux distros. Use one of the following links (and methods),
help.ubuntu.com/community/mkusb/gui#from_phillw.net
help.ubuntu.com/community/mkusb/gui/tarball
I just managed to achieve this with a LOT of help from this very helpful set of answers.
For reference, I was setting up a 16 GB Kingston Data Traveller with a custom ISO image that I had previously built using SystemBack.
The OS I am working on is Lubuntu, and my custom ISO had a lot of software package changes (I removed a lot of gumpf and installed some necessary utilities).
I started out by using araghuteja's very useful set of instructions and set up my device with three partitions:
- 4 GB for the Windows accessible FAT32 partition (
/dev/sdc1) - 2 GB for the OS (
/dev/sdc2) - The rest (under 8 GB, but approximately that) as the casper-rw partition (
/dev/sdc3)
This unfortunately didn't quite work, as just removing the casper-rw file from the OS partition isn't quite enough.
The real help was from Yu Jia Cheong, and StarBloom where they explain how to reset the GRUB bootloader to 'see' the casper-rw partition.
I did have to make some details that may make Yu Jia's instructions a bit clearer. (Please note, these instructions may not be fully appropriate for default Ubuntu ISO image as I think some of the file locations may be altered by the Systemback configuration).
Specifically - on Part 5, the file generated by
sudo update-initramfs -u
was named initrd.img-4.4.0-47-generic.
So - I copied this file to my 'casper-rw' partition, but then I realised that my GRUB boot menu looked was referencing /casper/initrd.gz (in four places) which resides within /boot/grub on the UUI partition.
So - I renamed my initrd.img-4.4.0-47-generic to initrd.gz, and replaced the file in UUI/boot/grub/casper/ (obviously retaining a copy of the old file just in case!).
Booted up with that particular USB stick and using df -h in the 'machine' root folder I get the following:
$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 16G 4.0K 16G 1% /dev
tmpfs 3.2G 1.6M 3.2G 1% /run
/dev/sdb1 2.0G 1.3G 729M 65% /cdrom
/dev/loop0 1.3G 1.3G 0 100% /rofs
/cow 8.2G 118M 7.7G 2% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
tmpfs 16G 4.0K 16G 1% /tmp
none 5.0M 0 5.0M 0% /run/lock
none 16G 0 16G 0% /run/shm
none 100M 16K 100M 1% /run/user
/dev/sdb3 4.0G 4.0K 4.0G 1% /media/silver/DATA
/dev/sdb2 8.2G 118M 7.7G 2% /media/silver/casper-rw
I assume /cow is the casper live partition, and it is showing the correct size.
I find it interesting that I can actually SEE the casper-rw partition as /dev/sdb2 when I'm using the machine - is this correct??
I'm also not convinced about the 3-way partitioning. In Windows(10) I can only see the OS partition, not the Data partition.
Many thanks to all, and I hope my experience helps someone else! (Note - I'm a sufficiently new to Stack Exchange usage that I can't comment - hence my answer is new, rather than a comment on Yu Jia's).
Update
I'm trying this again to create another 'Pen-Machine' and I've realised that the larger your ISO image, the more room you need to create for the casper-rw file when you run the UUInstaller.
My latest ISO image is something like 3.8 GB (it's a customised Ubuntu 16.04) and after three tries I have to create the initial OS partition at 5 GB and use a 1 GB persistence setting in the installer - otherwise you run out of storage when updating the initramfs functions.
Note - I finally resorted to setting the OS partition at 8 GB, just to make sure I had room!