Will a live usb with persistence store files automatically?
Solution 1:
Yes, if a Live USB is persistent then it will store all changes to programs and files, including newly installed software package. However, you should not use persistence as a replacement for a regular installation, only a temporary measure. Ubuntu's persistence slows down dramatically over time because of the architecture of the Live USB.
A brief primer on persistence and Live CD/DVD/USBs
Linux's Live CD feature is very efficient. Using a read-only filesystem called SquashFS, you can compress a ~4GB filesystem image down to around 700MB, which is small enough to fit on a CD. Recently, Canonical decided that, because of the rapidly declining usage of CDs, it wasn't worth it to go to the effort of making their CD (now DVD/USB) images fit on a CD, so they decided to target their images toward DVDs instead. This means that you can no longer fit a stock Ubuntu image onto a CD. However, that is no excuse for letting the (compressed) image balloon up, so Ubuntu (along with most other Linux distros) still uses SquashFS for compression. However, there is a drawback to getting ridiculously high compression rates: as mentioned above, SquashFS is read-only.
So what about persistence? How is persistence possible is Live USBs use a read-only filesystem? Well, what happens is that, when you exit a persistent session, Ubuntu saves all file changes to a persistence file. Then, the next time that the Live USB is booted up, Ubuntu first loads the "vanilla" Ubuntu installation. After that has completed, it then gets the persistence file and uses the data in it to re-modify the files back to where they were when you last left off.
This is a pretty clever mechanism, but is has a few drawbacks, one of them being that, as you use the system more and update some of the packages, that persistence file becomes larger and larger, which means longer and longer boot times, not to mention other nasty side effects.
So, the lesson is that, while persistence is a very handy feature for a Live USB to have, you should only use it when necessary, e.g., your main computer is down.
Solution 2:
Live-only
You can install program packages into a live-only operating system (booted from DVD or USB). But the installed packages will disappear at shutdown or reboot.
Persistent live
Yes, you can install program packages into a persistent live operating system (booted from USB). The installed packages will survive shutdown and reboot. There are limitations: You cannot use new kernels and kernel drivers (for example graphics drivers) because these are started before the overlay for persistence is activated.
help.ubuntu.com/community/mkusb
help.ubuntu.com/community/mkusb/persistent
The following link provides a method and tool to keep your personal data and tweaks (in the home directory) while making the persistent live system up to date. But you have to re-install the program packages, that you installed.
Make persistent live drives with casper-rw and home-rw partitions
Installed system in USB drive
It is also possible to install Ubuntu into a USB drive like installing into an internal drive. Such a system works well in a fast USB3 pendrive (and even better in an SSD or HDD in an external box).
Such a system is as flexible as any installed Ubuntu system. You can install all kinds of program packages including kernels and kernel drivers and keep the system up to date. It is also portable between computers.
But live-only and persistent live systems are more portable, so if you want a USB pendrive, that works in most computers (that you can borrow temporarily), a persistent live system is a good option.
Boot Ubuntu from external drive