apt forgot about more than 1,000 packages, and I'm unable to reinstall
I'm running 20.04.2 LTS (Focal Fossa)
I hadn't run aptitude for a couple days. It always worked fine, but when I tried it now, it started giving me the "e Examine" dialog each time I run it. When I type "e", it goes into a 10-20 minute run with the following showing at the top of the page:
open: aaa; closed: bbb; defer: ccc; conflict: ddd
Where 'aaa', 'bbb', 'ccc', 'ddd' are increasing numeric values ... and 'aaa' and 'bbb' are increasing rapidly, up to more than 50,000.
After a while, aptitude simply exits.
At this point, if I run dpkg --configure -a, it quickly returns to the command prompt with no error. But then, when I run apt-get install -f, I get a list of more than 1,000 package names followed by this ...
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Most of these packages (probably all of them, actually) are items that I use from day to day, and they should not be removed.
Oh, also, if I capture the huge list of packages that are shown when I do apt-get install -f, and if I generate a thousand or so apt install PKG commands for each of these package names and then run this, I just get multiple lists of those same thousand-or-so packages for each of these commands, followed by that same "use sudo apt autoremove" message after each apt install PKG run.
Somehow, my apt caches seem to have gotten hosed. I don't know what could have caused this.
However, I don't really want to spend time trying to figure out why this happened. I just want to fix it so that all those packages are once again properly installed and manageable via apt.
Can anyone point me to docs that could help me, or perhaps to suggest a solution?
Thank you very much in advance.
PS: my /etc/apt/sources.list hasn't changed for weeks, nor have any files in /etc/apt/source.list.d changed in a long time. Actually, nothing under /etc/apt seems to have changed since a couple months ago.
PPS*: Here are the last dozen or so lines of /var/log/apt/history.log ...
Start-Date: 2021-07-24 20:01:24
Requested-By: ljz (1000)
Install: linux-headers-5.4.0-80-generic:amd64 (5.4.0-80.90, automatic), linux-modules-5.4.0-80-generic:amd64 (5.4.0-80.90, automatic), linux-headers-5.4.0-80:amd64 (5.4.0-80.90, automatic), linux-image-5.4.0-80-generic:amd64 (5.4.0-80.90, automatic), linux-modules-extra-5.4.0-80-generic:amd64 (5.4.0-80.90, automatic)
Upgrade: networkd-dispatcher:amd64 (2.0.1-1, 2.1-2~ubuntu20.04.1), linux-headers-generic:amd64 (5.4.0.77.80, 5.4.0.80.84), linux-libc-dev:amd64 (5.4.0-77.86, 5.4.0-80.90), libcurl4:amd64 (7.68.0-1ubuntu2.5, 7.68.0-1ubuntu2.6), ruby2.7:amd64 (2.7.0-5ubuntu1.4, 2.7.0-5ubuntu1.5), libsystemd0:amd64 (245.4-4ubuntu3.7, 245.4-4ubuntu3.11), linux-image-generic:amd64 (5.4.0.77.80, 5.4.0.80.84), libsystemd-dev:amd64 (245.4-4ubuntu3.7, 245.4-4ubuntu3.11), python3-pip:amd64 (20.0.2-5ubuntu1.5, 20.0.2-5ubuntu1.6), udev:amd64 (245.4-4ubuntu3.7, 245.4-4ubuntu3.11), ruby2.7-dev:amd64 (2.7.0-5ubuntu1.4, 2.7.0-5ubuntu1.5), ruby2.7-doc:amd64 (2.7.0-5ubuntu1.4, 2.7.0-5ubuntu1.5), libudev1:amd64 (245.4-4ubuntu3.7, 245.4-4ubuntu3.11), systemd-timesyncd:amd64 (245.4-4ubuntu3.7, 245.4-4ubuntu3.11), libxnvctrl0:amd64 (460.73.01-0ubuntu0.20.04.1, 470.57.01-0ubuntu0.20.04.1), libglib2.0-dev-bin:amd64 (2.64.6-1~ubuntu20.04.3, 2.64.6-1~ubuntu20.04.4), libruby2.7:amd64 (2.7.0-5ubuntu1.4, 2.7.0-5ubuntu1.5), systemd-sysv:amd64 (245.4-4ubuntu3.7, 245.4-4ubuntu3.11), libglib2.0-bin:amd64 (2.64.6-1~ubuntu20.04.3, 2.64.6-1~ubuntu20.04.4), libpam-systemd:amd64 (245.4-4ubuntu3.7, 245.4-4ubuntu3.11), libglib2.0-dev:amd64 (2.64.6-1~ubuntu20.04.3, 2.64.6-1~ubuntu20.04.4), libglib2.0-doc:amd64 (2.64.6-1~ubuntu20.04.3, 2.64.6-1~ubuntu20.04.4), systemd:amd64 (245.4-4ubuntu3.7, 245.4-4ubuntu3.11), libnss-systemd:amd64 (245.4-4ubuntu3.7, 245.4-4ubuntu3.11), libglib2.0-data:amd64 (2.64.6-1~ubuntu20.04.3, 2.64.6-1~ubuntu20.04.4), curl:amd64 (7.68.0-1ubuntu2.5, 7.68.0-1ubuntu2.6), linux-generic:amd64 (5.4.0.77.80, 5.4.0.80.84), python-pip-whl:amd64 (20.0.2-5ubuntu1.5, 20.0.2-5ubuntu1.6), libglib2.0-0:amd64 (2.64.6-1~ubuntu20.04.3, 2.64.6-1~ubuntu20.04.4), libcurl3-gnutls:amd64 (7.68.0-1ubuntu2.5, 7.68.0-1ubuntu2.6)
End-Date: 2021-07-24 20:02:46
Start-Date: 2021-07-25 19:31:25
Requested-By: ljz (1000)
Remove: linux-modules-extra-5.4.0-74-generic:amd64 (5.4.0-74.83), linux-modules-extra-5.4.0-73-generic:amd64 (5.4.0-73.82)
End-Date: 2021-07-25 19:31:28
Start-Date: 2021-07-27 18:56:53
Requested-By: ljz (1000)
Upgrade: libaspell15:amd64 (0.60.8-1build1, 0.60.8-1ubuntu0.1), aspell:amd64 (0.60.8-1build1, 0.60.8-1ubuntu0.1), libmysqlclient21:amd64 (8.0.25-0ubuntu0.20.04.1, 8.0.26-0ubuntu0.20.04.2), teamviewer:amd64 (15.19.3, 15.20.3)
End-Date: 2021-07-27 18:57:05
aptitude worked fine around 2021-07-24, so whatever broke my setup must have occurred after that.
Output of sudo aptitude search ?broken
iB anydesk - The fastest remote desktop software on the market.
iBA apt - commandline package manager
iBA aptitude - terminal-based package manager
iBA base-files - Debian base system miscellaneous files
iBA base-passwd - Debian base system master password and group files
iBA bash - GNU Bourne Again SHell
iBA bsdutils - basic utilities from 4.4BSD-Lite
iB chromium-common - web browser - common resources used by the chromium packages
iB chromium-sandbox - web browser - setuid security sandbox for chromium
iB containerd.io - An open and reliable container runtime
iBA coreutils - GNU core utilities
iBA dash - POSIX-compliant shell
iBA debianutils - Miscellaneous utilities specific to Debian
iBA diffutils - File comparison utilities
iBA dpkg - Debian package management system
iBA e2fsprogs - ext2/ext3/ext4 file system utilities
iBA fdisk - collection of partitioning utilities
iBA findutils - utilities for finding files--find, xargs
iB google-chrome-stable - The web browser from Google
iBA grep - GNU grep, egrep and fgrep
iBA gzip - GNU compression utilities
iBA hostname - utility to set/show the host name or domain name
iBA libc-bin - GNU C Library: Binaries
iBA libcurl3-gnutls - easy-to-use client-side URL transfer library (GnuTLS flavour)
iBA libglib2.0-0 - GLib library of C routines
iBA libnss-systemd - nss module providing dynamic user and group name resolution
iBA libpam-systemd - system and service manager - PAM module
iBA libsystemd0 - systemd utility library
iBA libudev1 - libudev shared library
iBA linux-base - Linux image base package
iBA linux-headers-5.4.0-80-generic - Linux kernel headers for version 5.4.0 on 64 bit x86 SMP
iBA linux-image-5.4.0-80-generic - Signed kernel image generic
iBA linux-image-generic - Generic Linux kernel image
iBA linux-modules-extra-5.4.0-80-generic - Linux kernel extra modules for version 5.4.0 on 64 bit x86 SMP
iBA login - system login tools
iBA mount - tools for mounting and manipulating filesystems
iBA ncurses-bin - terminal-related programs and man pages
iB net.downloadhelper.coapp - Video DownloadHelper companion app
iBA networkd-dispatcher - Dispatcher service for systemd-networkd connection status changes
iB ocenaudio - Ocenaudio is a fast, cross-platform audio editor.
iB palemoon - Firefox-based, efficient and easy to use web browser
iBA perl-base - minimal Perl system
iB rpi-imager - Raspberry Pi imaging utility
iBA sed - GNU stream editor for filtering/transforming text
iB signal-desktop - Private messaging from your desktop
iB signal-desktop-beta - Private messaging from your desktop
iBA systemd - system and service manager
iBA systemd-timesyncd - minimalistic service to synchronize local time with NTP servers
iBA sysvinit-utils - System-V-like utilities
iBA tar - GNU version of the tar archiving utility
iBA util-linux - miscellaneous system utilities
iB vdpau-va-driver - VDPAU-based backend for VA API
iB zoom - Zoom, #1 Video Conferencing and Web Conferencing Service
And here's more:
% sudo apt update
Hit:1 https://linux.teamviewer.com/deb stable InRelease
Hit:2 https://updates.signal.org/desktop/apt xenial InRelease
Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Hit:4 http://us.archive.ubuntu.com/ubuntu focal InRelease
Hit:5 http://us.archive.ubuntu.com/ubuntu focal-updates InRelease
Get:6 http://us.archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Hit:7 http://archive.canonical.com/ubuntu focal InRelease
Fetched 214 kB in 1s (178 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
3 packages can be upgraded. Run 'apt list --upgradable' to see them.
% sudo apt list --upgradeable
Listing... Done
signal-desktop-beta/xenial 5.11.0-beta.3 amd64 [upgradable from: 5.8.0-beta.1]
signal-desktop/xenial 5.10.0 amd64 [upgradable from: 5.9.0]
ubuntu-advantage-tools/focal-updates 27.2.2~20.04.1 amd64 [upgradable from: 27.1~20.0
% sudo apt-get install -f
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
fonts-ubuntu fonts-uralic fotoxx fotoxx-common friendly-recovery ftp gdal-data gegl genisoimage geoclue-2.0 geoip-database
[ ... more than 100 lines containing more package names ... ]
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
Still more ...
grep -r "^deb" /etc/apt/ --include=*.list
/etc/apt/sources.list:deb http://us.archive.ubuntu.com/ubuntu/ focal main restricted
/etc/apt/sources.list:deb-src http://us.archive.ubuntu.com/ubuntu/ focal restricted main universe multiverse
/etc/apt/sources.list:deb http://us.archive.ubuntu.com/ubuntu/ focal-updates main restricted
/etc/apt/sources.list:deb-src http://us.archive.ubuntu.com/ubuntu/ focal-updates restricted main universe multiverse
/etc/apt/sources.list:deb http://us.archive.ubuntu.com/ubuntu/ focal universe
/etc/apt/sources.list:deb http://us.archive.ubuntu.com/ubuntu/ focal-updates universe
/etc/apt/sources.list:deb http://us.archive.ubuntu.com/ubuntu/ focal multiverse
/etc/apt/sources.list:deb http://us.archive.ubuntu.com/ubuntu/ focal-updates multiverse
/etc/apt/sources.list:deb http://us.archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse
/etc/apt/sources.list:deb-src http://us.archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse
/etc/apt/sources.list:deb http://archive.canonical.com/ubuntu focal partner
/etc/apt/sources.list:deb-src http://archive.canonical.com/ubuntu focal partner
/etc/apt/sources.list:deb http://security.ubuntu.com/ubuntu focal-security main restricted
/etc/apt/sources.list:deb-src http://security.ubuntu.com/ubuntu focal-security restricted main universe multiverse
/etc/apt/sources.list:deb http://security.ubuntu.com/ubuntu focal-security universe
/etc/apt/sources.list:deb http://security.ubuntu.com/ubuntu focal-security multiverse
/etc/apt/sources.list.d/teamviewer.list:deb https://linux.teamviewer.com/deb stable main
/etc/apt/sources.list.d/signal-xenial.list:deb [arch=amd64 signed-by=/usr/share/keyrings/signal-desktop-keyring.gpg] https://updates.signal.org/desktop/apt xenial main
Possible solution ...
What if I do the following? ...
- run
sudo apt-get install -f >apt-installs - edit that file, removing the lines that precede and follow the package names, and putting "sudo apt install " at the beginning of each line
- run "sudo apt autoremove"
- run "bash apt-installs" to reinstall each and every package.
Here's what the "apt-installs" file would look like after step 2 ...
sudo apt install accountsservice acpi-support acpid adb alsa-base android-libadb android-libbase android-libboringssl android-libcrypto-utils
sudo apt install android-libcutils android-liblog android-sdk-platform-tools-common apt-rdepends aptdaemon aptdaemon-data apturl apturl-common
sudo apt install aria2 audacious audacious-plugins audacious-plugins-data bind9-dnsutils blueman bluez bluez-cups brltty brltty-x11 brotli
sudo apt install busybox-static byobu catdoc catfish cheese-common cinnamon-desktop-data cinnamon-l10n clang-format-11 clang-tidy-11
sudo apt install clang-tools-11 command-not-found cppcheck cvs dc dctrl-tools debhelper debtree devilspie devilspie2 dh-autoreconf
sudo apt install dh-elpa-helper dh-strip-nondeterminism diffstat distro-info dmz-cursor-theme doc-base docbook-xml dwz efibootmgr elfutils
[ ... etc. ... for more than 100 lines ]
What do you folks think? Could this fix my problem?
More follow-up ...
After more investigation about apt, I now see that the sudo apt autoremove will indeed uninstall all of those packages. That's over 1,000 packages, and the my system will indeed become unusuable after that. And it's unclear to me whether running all of those sudo apt install ... commands will even work, given that so much other software would have been uninstalled.
So, I'm still stuck. :(
I don't want to uninstall any software. I just want to get the apt caches back into a usable state, without changing any existing software installations.
Solution 1:
I figured out how to fix this problem. I needed to remove and reinstall all of those 1,000-plus packages, and I did so one-by-one. And then I needed to reinstall aptitude. To acomplish this fix, I performed the following steps ...
Step 1:
% sudo apt-get install -f >apt-installs
After running this, the apt-installs file contained this data ...
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
activity-log-manager apg audacious audacious-plugins cppcheck cracklib-runtime dh-strip-nondeterminism docbook-xml
[ ... more than 180 lines containing lists of package names ... ]
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
Step 2:
I edited this apt-installs file to remove the top 4 lines and the bottom 2 lines, and I reformatted it so that there is one package name per line.
Step 3:
I prepended the string fixit to the beginning of each line.
Step 4:
I created a shell function called fixit to remove and then install the specified package, and I put that function at the top of the apt-installs file. Then, here's what the first dozen or so lines of apt-installs look like ...
fixit() {
[[ $# -lt 1 ]] && return 0
package="${1}"
shift
echo "### fixing ${package}"
apt-get -y remove "${package}"
apt-get -y install "${package}"
echo ''
}
fixit activity-log-manager
fixit apg
fixit audacious
fixit audacious-plugins
fixit cppcheck
fixit cracklib-runtime
fixit dh-strip-nondeterminism
fixit docbook-xml
fixit gkbd-capplet
fixit gnome-bluetooth
fixit gnome-control-center-data
[ ... etc. ... more than 1,500 lines ]
Step 5:
I ran this:
% sudo bash apt-installs
This removed and reinstalled each of the packages.
The advantage of these piecemeal "remove/install" runs for each package is that only one, single package is uninstalled and then reinstalled at any given moment, instead of blowing away more than 1,500 pieces of software via an apt autoremove and thereby rendering my system un-runnable.
Step 6:
I then ran these two commands ...
% sudo apt-get purge aptitude
% sudo apt-get install aptitude
Before I did this, aptitude was still hanging and crashing, even after the reinstalls of all the packages. This purge/install fixed the aptitude crashes.
And now, apt, apt-get, and aptitude are all working fine again, and my system is still usable.
All's well that ends well.