Title: Manually fixing apt's "Unmet dependencies" error Date: 2025-08-05 22:00 I am, despite my best judgment and common sense, acting as a system administrator for a handful of projects/users, sometimes on my own, sometimes as part of a team. On the bright side, it's usually a great source of weird debugging/workaround stories, like this one. With the [new release of proxmox](https://www.proxmox.com/en/about/company-details/press-releases/proxmox-virtual-environment-9-0) and with 30 minutes of free time, a sysadmin team I'm part of thought it would be nice to upgrade a box together. An admin ran `pve8to9 --full`, changed the repositories from bookworm to trixie, and ran `apt dist-upgrade`, and… their screen was filled with ``WARNING: ld.so: invalid GLIBC_TUNABLES `glibc.malloc.perturb=1,glibc.malloc.check=1,glibc.malloc.tcache_count=0': ignored.``. Panic. So they hit `` to investigate what was happening. Turns out, another admin threw those configuration options somewhere to "harden" the glibc allocator, and they're now deprecated, hence the message. No big deal, this can be fixed by removing those options, and running `apt dist-upgrade` again. Or can it? ```console # apt dist-upgrade Reading package lists... Done Building dependency tree... Done Reading state information... Done You might want to run 'apt --fix-broken install' to correct these. The following packages have unmet dependencies: libanyevent-perl : Depends: perlapi-5.40.0 Recommends: libasync-interrupt-perl but it is not installed Recommends: libev-perl but it is not installed or libevent-perl but it is not installed Recommends: libguard-perl but it is not installed libcommon-sense-perl : Depends: perl (< 5.36.1~) but 5.40.1-6 is installed libnss-systemd : Depends: systemd (= 257.7-1) but 252.38-1~deb12u1 is installed libpam-systemd : Depends: systemd (= 257.7-1) but 252.38-1~deb12u1 is installed libsasl2-2 : Depends: libssl3t64 (>= 3.0.0) but it is not installed Recommends: libsasl2-modules (>= 2.1.28+dfsg1-9) but it is not installed libsasl2-modules-db : Depends: libdb5.3t64 but it is not installed libsystemd-shared : Depends: libssl3t64 (>= 3.4.0) but it is not installed openssl : Depends: libssl3t64 (>= 3.5.0) but it is not installed perl : Depends: perl-base (= 5.40.1-6) but 5.36.0-7+deb12u2 is installed Depends: libperl5.40 (= 5.40.1-6) but it is not installed perl-modules-5.40 : Depends: perl-base (>= 5.40.1-1) but 5.36.0-7+deb12u2 is installed systemd : Depends: libsystemd-shared (= 252.38-1~deb12u1) but 257.7-1 is installed Depends: libsystemd0 (= 252.38-1~deb12u1) but 257.7-1 is installed E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution). [100] # ``` Alright, let's try `apt --fix-broken install`: ```console # apt --fix-broken install Reading package lists... Done Building dependency tree... Done Reading state information... Done Correcting dependencies... Done The following packages were automatically installed and are no longer required: […] 75 upgraded, 64 newly installed, 50 to remove and 530 not upgraded. 17 not fully installed or removed. Need to get 0 B/118 MB of archives. After this operation, 215 MB of additional disk space will be used. Do you want to continue? [Y/n] y W: (pve-apt-hook) !! WARNING !! W: (pve-apt-hook) You are attempting to remove the meta-package 'proxmox-ve'! W: (pve-apt-hook) W: (pve-apt-hook) If you really want to permanently remove 'proxmox-ve' from your system, run the following command W: (pve-apt-hook) touch '/please-remove-proxmox-ve' W: (pve-apt-hook) run apt purge proxmox-ve to remove the meta-package W: (pve-apt-hook) and repeat your apt invocation. W: (pve-apt-hook) W: (pve-apt-hook) If you are unsure why 'proxmox-ve' would be removed, please verify W: (pve-apt-hook) - your APT repository settings W: (pve-apt-hook) - that you are using 'apt full-upgrade' to upgrade your system E: Sub-process /usr/share/proxmox-ve/pve-apt-hook returned an error code (1) E: Failure running script /usr/share/proxmox-ve/pve-apt-hook [100] # ``` We're absolutely not going to do that, so let's use `apt-mark` to tell `apt` to leave `proxmox-ve` alone, and try the whole manoeuvre again: ```console # apt-mark hold proxmox-ve proxmox-ve set on hold. # apt --fix-broken install Reading package lists... Done Building dependency tree... Done Reading state information... Done Correcting dependencies... failed. The following packages have unmet dependencies: libanyevent-perl : Depends: perlapi-5.40.0 Recommends: libasync-interrupt-perl but it is not installed Recommends: libev-perl but it is not installed or libevent-perl but it is not installed Recommends: libguard-perl but it is not installed libcommon-sense-perl : Depends: perl (< 5.36.1~) but 5.40.1-6 is installed libnss-systemd : Depends: systemd (= 257.7-1) but 252.38-1~deb12u1 is installed libpam-systemd : Depends: systemd (= 257.7-1) but 252.38-1~deb12u1 is installed libsasl2-2 : Depends: libssl3t64 (>= 3.0.0) but it is not installed Recommends: libsasl2-modules (>= 2.1.28+dfsg1-9) but it is not installed libsasl2-modules-db : Depends: libdb5.3t64 but it is not installed libsystemd-shared : Depends: libssl3t64 (>= 3.4.0) but it is not installed openssl : Depends: libssl3t64 (>= 3.5.0) but it is not installed perl : Depends: perl-base (= 5.40.1-6) but 5.36.0-7+deb12u2 is installed Depends: libperl5.40 (= 5.40.1-6) but it is not installed perl-modules-5.40 : Depends: perl-base (>= 5.40.1-1) but 5.36.0-7+deb12u2 is installed systemd : Depends: libsystemd-shared (= 252.38-1~deb12u1) but 257.7-1 is installed Depends: libsystemd0 (= 252.38-1~deb12u1) but 257.7-1 is installed E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages. E: Unable to correct dependencies [100] # ``` You might think that using `apt-mark hold` for all the packages with unmet dependencies is the way to go, but you would be wrong, as it doesn't fix shit. You can check `apt`'s documentation and ask the collective hivemind of the internet, but we didn't find anything to solve our problem. Then we realised that `apt` has a graph of all packages and their dependencies stored somewhere on the filesystem, and we found it via `strace -f -e openat 2>&1 apt install --fix-missing | grep -oP "\"(.*)\"" | sort | uniq -c`: it's `/var/lib/dpkg/status`. We can now edit it to remove the `Depends:` field of every package with broken dependencies to appease `apt`'s dependencies solver via this big fat lie. Once done: ```console # apt upgrade Reading package lists... Done Building dependency tree... Done Reading state information... Done Calculating upgrade... Done The following packages were automatically installed and are no longer required: […] Setting up ca-certificates (20250419) ... Updating certificates in /etc/ssl/certs... openssl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSSL_3.4.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSSL_3.2.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.5.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.4.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.2.0' not found (required by openssl) dpkg: error processing package ca-certificates (--configure): installed ca-certificates package post-installation script subprocess returned error exit status 1 Setting up busybox (1:1.37.0-6+b3) ... Setting up libklibc:amd64 (2.0.14-1) ... Setting up novnc-pve (1.6.0-3) ... Setting up libz3-4:amd64 (4.13.3-1) ... Setting up xz-utils (5.8.1-1) ... Setting up libfribidi0:amd64 (1.0.16-1) ... Setting up libopus0:amd64 (1.5.2-2) ... Setting up libquadmath0:amd64 (14.2.0-19) ... dpkg: dependency problems prevent configuration of python3-requests: python3-requests depends on python3-certifi; however: Package python3-certifi is not configured yet. python3-requests depends on ca-certificates; however: Package ca-certificates is not configured yet. dpkg: error processing package python3-requests (--configure): dependency problems - leaving unconfigured Setting up libproc2-0:amd64 (2:4.0.4-9) ... Setting up proxmox-backup-restore-image (1.0.0) ... Setting up ssl-cert (1.1.3) ... openssl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSSL_3.4.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSSL_3.2.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.5.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.4.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.2.0' not found (required by openssl) Signature algorithm of /etc/ssl/certs/ssl-cert-snakeoil.pem is not sha256. Recreating. Could not create certificate. Openssl output was: openssl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSSL_3.4.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSSL_3.2.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.5.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.4.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version `OPENSSL_3.2.0' not found (required by openssl) dpkg: error processing package ssl-cert (--configure): installed ssl-cert package post-installation script subprocess returned error exit status 1 Setting up libxcb-sync1:amd64 (1.17.0-2+b1) ... Setting up dtach (0.9-7) ... Setting up lsb-release (12.1-1) ... Setting up dbus-system-bus-common (1.16.2-2) ... systemd-sysusers: error while loading shared libraries: libsystemd-shared-252.so: cannot open shared object file: No such file or directory dpkg: error processing package dbus-system-bus-common (--configure): installed dbus-system-bus-common package post-installation script subprocess returned error exit status 127 Setting up libconfuse2:amd64 (3.3-4) ... Setting up libisl23:amd64 (0.27-1) ... Setting up libcpg4:amd64 (3.1.9-pve2) ... Setting up libcmap4:amd64 (3.1.9-pve2) ... Setting up libpulse0:amd64 (17.0+dfsg1-2+b1) ... dpkg: dependency problems prevent configuration of python3-pip-whl: python3-pip-whl depends on ca-certificates; however: Package ca-certificates is not configured yet. dpkg: error processing package python3-pip-whl (--configure): dependency problems - leaving unconfigured Setting up libquorum5:amd64 (3.1.9-pve2) ... Setting up iptables (1.8.11-2) ... Setting up fontconfig-config (2.15.0-2.3) ... Setting up uidmap (1:4.17.4-2) ... dpkg: dependency problems prevent configuration of cron: cron depends on cron-daemon-common; however: Package cron-daemon-common is not configured yet. dpkg: error processing package cron (--configure): dependency problems - leaving unconfigured Setting up python3-typeguard (4.4.2-1) ... Setting up rsyslog (8.2504.0-1) ... […] dpkg: error processing package pve-manager (--configure): dependency problems - leaving triggers unprocessed Processing triggers for man-db (2.11.2-2) ... Processing triggers for sgml-base (1.31+nmu1) ... Errors were encountered while processing: ca-certificates python3-certifi ssl-cert python3-requests dbus-system-bus-common cron-daemon-common liblwp-protocol-https-perl libwww-perl python3-pip-whl cron libpve-common-perl libproxmox-acme-perl python3-pyvmomi libpve-apiclient-perl dbus libpve-guest-common-perl libpve-storage-perl libpve-access-control libpve-cluster-perl libpve-http-server-perl pve-ha-manager libpve-cluster-api-perl libpve-notify-perl pve-manager E: Sub-process /usr/bin/dpkg returned an error code (1) [100] # ``` Things are still broken, but less so, at least `apt` is able to upgrade/install/manage/… packages. Now it's simply a matter of running `dpkg --force-all --configure -a`, `apt-get install --fix-missing` and `apt install ` (with `` being things listed in the `Errors were encountered while processing:` errors from the other commands) until you get no more errors. At this point, you can reopen `/var/lib/dpkg/status`, uncomment the lines you commented if they're still present, and resume your `apt dist-upgrade`. Still, it's sad that in 2025 a package manager is a simple `SIGINT` away from being completely broken.