Which file on a USB live installation made with UNetBootin persists file information?
Okay after a bit of googling to figure out what I was looking for exactly...
sudo mkdir /media/casper
sudo mount ./casper-rw /media/casper -o loop
And now I can mount it and get back at my files again.