How does the Ubuntu upgrade process work?
How does Ubuntu upgrade seamlessly to a newer distribution, while the operating system is still running? I'm upgrading from 10.10 to 11.04, and I've upgraded several times before, and it's as simple as running update-manager -d and downloading and installing them, then rebooting.
How exactly does this work though? How is the upgrade manager able to update the operating system while it is still in use?
Solution 1:
Here a more detailed description of the process. Sorry the text got so long.
My experience stems from Debian, for which the entire packaging and upgrade system used in Ubuntu was originally invented. The daily Ubuntu security upgrades correspond to running apt-get upgrade which normally doesn't remove any software. The big release upgrades correspond to a apt-get dist-upgrade during which software packages can be exchanged completely.
Actually the very low level components are normally not exchanged during a release upgrade. Immediately after the upgrade you should find two kernel and initrd images in your /boot directory. This is because unlike programs kernel components are not interchangable so well. If the need arises to load new device drivers during the upgrade, those must be compatible with the running kernel. After the system boots up with the new kernel, the old one can be removed. Last time I checked this one thing had to be done manually, I don't know how the current updater handles this. This is BTW. the main reason, why the kernel image bears its version number in the file name - so you can have different kernel versions installed at the same time. Same for the module path (/lib/modules/...)
Software packages are upgraded one by one, starting with the lowest packages in the dependency hirarchy. Those are normally program librarys like libc and other ones. The sequence in which packages are updated is however not hardcoded but dynamically calculated as package dependencies are resolved. In most cases old programs can function with new librarys, so it's not so problematic if those librarys get replaced first.
You have to understand here, that the system differentiates between manually installed packages (i.e. the packages which installation you directly requested yourself, i.e. chromium) and automatically installed packages, which where only installed to fulfill dependencies of manually installed packages (and dependencies of those dependencies).
For each manually installed program, the updater just looks for a newer version. Often those programs are just meta packages like "ubuntu-desktop", which contains no data, and just dependencies. New versions of dependent libraries will be pulled in, as they are requested by directly updated (manually requested) programs. The updater will always try to install the newest usable version of any dependent package (during any upgrdade not just release upgrades).
Programs which cannot function with the new library versions cannot be started during the time after the library is upgraded, and before the program itself is upgraded as well. If those programs should already be running before the library upgrade they will however continue to run, because the old library version remains in memory as long as it remains in use. Same goes for programs which were started before they get upgraded. Those will not provide new features until they are terminated and restarted.
After the update some libraries (or dependencies in general) will be orphaned. Those are libraries which were required by old program versions, but are no longer required by the new versions. Since these packages are marked as automatically installed, and since no manually installed prgram relates to them anymore these packages can easily be located and removed. You can even observe this as the last step of the update process (the updater sais "removing obsolete packages" or something similar).
Some packages will be installed, which where not installed before, those are simply new dependencies, which are marked as automatically installed, and can be removed, should the requirement for them vanish in the future.
This mechanism even allows the exchange of entire user programs. Like for example switching from Gnome2 to Unity. Since both are just automatic dependencies of ubuntu-desktop, which is one of the few packages, for which new versions are actually requested in the first place.
Programs do normally not depend on a specific version of the OS kernel, so they will normally work fine with the running kernel.
Apart from all this I suspect the Ubuntu updater throws some specific fixes and workarounds into the mix, to circumvent situations where this theory breaks.
As you can see during the update there are very well conditions in which the system is only usable to a limited part. Should something go wrong during the update you will most likely be left with a broken system. Often even one which cannot easily be repaired, since the upgrade program can be affected as well. Remember, programs with broken dependencies can continue to function, but not be restarted, as long as the dependencies are broken, this goes for the updater as well.
You can use the command line program apt-mark to find out which packages are marked as manually installed and which have been automatically installed. You can also switch those marks using the same program. This will directly affect the update process.
In a more complex software setup the Updater will sometimes ask you to resolve a dependency manually. I.e. when one manually installed program gets updated and requests a new version of a library while another manually installed program depends on the old version of the same library and cannot work with the new one. You will then have to make your choice, either to relinquish one of those programs or to refrain from upgrading both. Since dependencies are often complex this can get very messy very fast (you might have heard of the term "dependency hell").
Now to the specific questions:
- When low-level infrastructure is changed (low-level as in kernel, drivers, libraries, etc anything that a user doesn't directly interact with), what happens to the deprecated binaries?
- OK... I already covered this one
- What happens to an app that has been completely deprecated? For example, Unity 2D (or any other software abandoned/no maintainers whose package is not in new repo).
- If the App was once manually installed it will remain in the system, often causing the dependency hell I described.
- ubuntu-desktop is a metapackage that pulls default Ubuntu applications as dependencies. If I removed Firefox and installed Chrome, will Firefox still get pulled in as part of upgrades?
- As long as it is the standard browser of the new Release, yes. Chromium will also be upgraded. I am not sure if you can remove Firefox without removing ubuntu-desktop. Note that apart from strict dependencies, the package system knows also the concept of recommendations, where a software will normally be installed like a dependency, but can be uninstalled later without affecting anything else but its own sole dependencies.
- Moreover, what happens if a single app used to depend on package-x and no longer does so in new release; will package-x be upgraded alongside the rest of the packages despite being orphaned?
- Nope.
Just ask me if you still have any more questions.
Solution 2:
From my experience I would assume that while packages and modules are running they are held in memory and don't refer back to their copy on the hard drive much. You can see this if you run a program in ubuntu and then remove the related packages while its running. It will keep running, but if you close it, you can't restart it.
I would assume the same happens with a distribution upgrade. All of the packages related to the original version of ubuntu are still running even though they have been removed and replace with the new ones, so when they are finally stopped on a system restart, the new packages take over.