Mysql won't install on Ubuntu Server 20.04

I have looked at all the similar posts I could find on AskUbuntu or outside but could not find any that solved my issue.. I'm simply trying to install mysql on my server but can't seem to get it working.

So far I have tried:

  • removing all mysql packages, /var/lib/mysql, /etc/mysql, cleaning apt cache and re-installing (several times)
  • disabling apparmor service before running the install
  • running mysqld --initialize (no effect)

Here is the output when trying to start the install:

ubuntu@central-2:/etc$ sudo apt install mysql-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libevent-core-2.1-7 libmecab2 libnuma1 mysql-client-8.0 mysql-client-core-8.0 mysql-common mysql-server-8.0 mysql-server-core-8.0
Suggested packages:
  tinyca
Recommended packages:
  libhtml-template-perl mecab-ipadic-utf8
The following NEW packages will be installed:
  libevent-core-2.1-7 libmecab2 libnuma1 mysql-client-8.0 mysql-client-core-8.0 mysql-common mysql-server mysql-server-8.0 mysql-server-core-8.0
0 upgraded, 9 newly installed, 0 to remove and 8 not upgraded.
Need to get 0 B/23.0 MB of archives.
After this operation, 190 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Preconfiguring packages ...
Selecting previously unselected package mysql-common.
(Reading database ... 96065 files and directories currently installed.)
Preparing to unpack .../0-mysql-common_5.8+1.0.5ubuntu2_all.deb ...
Unpacking mysql-common (5.8+1.0.5ubuntu2) ...
Selecting previously unselected package mysql-client-core-8.0.
Preparing to unpack .../1-mysql-client-core-8.0_8.0.20-0ubuntu0.20.04.1_amd64.deb ...
Unpacking mysql-client-core-8.0 (8.0.20-0ubuntu0.20.04.1) ...
Selecting previously unselected package mysql-client-8.0.
Preparing to unpack .../2-mysql-client-8.0_8.0.20-0ubuntu0.20.04.1_amd64.deb ...
Unpacking mysql-client-8.0 (8.0.20-0ubuntu0.20.04.1) ...
Selecting previously unselected package libevent-core-2.1-7:amd64.
Preparing to unpack .../3-libevent-core-2.1-7_2.1.11-stable-1_amd64.deb ...
Unpacking libevent-core-2.1-7:amd64 (2.1.11-stable-1) ...
Selecting previously unselected package libmecab2:amd64.
Preparing to unpack .../4-libmecab2_0.996-10build1_amd64.deb ...
Unpacking libmecab2:amd64 (0.996-10build1) ...
Selecting previously unselected package libnuma1:amd64.
Preparing to unpack .../5-libnuma1_2.0.12-1_amd64.deb ...
Unpacking libnuma1:amd64 (2.0.12-1) ...
Selecting previously unselected package mysql-server-core-8.0.
Preparing to unpack .../6-mysql-server-core-8.0_8.0.20-0ubuntu0.20.04.1_amd64.deb ...
Unpacking mysql-server-core-8.0 (8.0.20-0ubuntu0.20.04.1) ...
Setting up mysql-common (5.8+1.0.5ubuntu2) ...
update-alternatives: using /etc/mysql/my.cnf.fallback to provide /etc/mysql/my.cnf (my.cnf) in auto mode
Selecting previously unselected package mysql-server-8.0.
(Reading database ... 96281 files and directories currently installed.)
Preparing to unpack .../mysql-server-8.0_8.0.20-0ubuntu0.20.04.1_amd64.deb ...
Unpacking mysql-server-8.0 (8.0.20-0ubuntu0.20.04.1) ...
Selecting previously unselected package mysql-server.
Preparing to unpack .../mysql-server_8.0.20-0ubuntu0.20.04.1_all.deb ...
Unpacking mysql-server (8.0.20-0ubuntu0.20.04.1) ...
Setting up libmecab2:amd64 (0.996-10build1) ...
Setting up mysql-client-core-8.0 (8.0.20-0ubuntu0.20.04.1) ...
Setting up libevent-core-2.1-7:amd64 (2.1.11-stable-1) ...
Setting up libnuma1:amd64 (2.0.12-1) ...
Setting up mysql-client-8.0 (8.0.20-0ubuntu0.20.04.1) ...
Setting up mysql-server-core-8.0 (8.0.20-0ubuntu0.20.04.1) ...
Setting up mysql-server-8.0 (8.0.20-0ubuntu0.20.04.1) ...
update-alternatives: using /etc/mysql/mysql.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode
Renaming removed key_buffer and myisam-recover options (if present)
Specified filename /var/lib/mysql/ibdata1 does not exist.
mysqld will log errors to /var/log/mysql/error.log
2020-06-14T13:15:44.220560Z 0 [ERROR] [MY-010946] [Server] Failed to start mysqld daemon. Check mysqld error log.
Warning: Unable to start the server.
Created symlink /etc/systemd/system/multi-user.target.wants/mysql.service → /lib/systemd/system/mysql.service.
Job for mysql.service failed because the control process exited with error code.
See "systemctl status mysql.service" and "journalctl -xe" for details.
invoke-rc.d: initscript mysql, action "start" failed.
● mysql.service - MySQL Community Server
     Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Sun 2020-06-14 13:15:46 UTC; 14ms ago
    Process: 38879 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=1/FAILURE)
dpkg: error processing package mysql-server-8.0 (--configure):
 installed mysql-server-8.0 package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of mysql-server:
 mysql-server depends on mysql-server-8.0; however:
  Package mysql-server-8.0 is not configured yet.

dpkg: error processing package mysql-server (--configure):
 dependency problems - leaving unconfigured
Processing triggers for systemd (245.4-4ubuntu3.1) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9) ...
Errors were encountered while processing:
 mysql-server-8.0
 mysql-server
[ Rootkit Hunter version 1.4.6 ]
File updated: searched for 179 files, found 137
needrestart is being skipped since dpkg has failed
E: Sub-process /usr/bin/dpkg returned an error code (1)

mysql error log contains the following:

ubuntu@central-2:/etc$ cat /var/log/mysql/error.log
2020-06-14T13:15:43.844787Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.20-0ubuntu0.20.04.1) initializing of server in progress as process 38755
2020-06-14T13:15:43.858140Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2020-06-14T13:15:43.864981Z 1 [ERROR] [MY-012576] [InnoDB] Unable to create temporary file; errno: 13
2020-06-14T13:15:43.865050Z 1 [ERROR] [MY-012929] [InnoDB] InnoDB Database creation was aborted with error Generic error. You may need to delete the ibdata1 file before trying to start up again.
2020-06-14T13:15:43.865252Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2020-06-14T13:15:43.865309Z 0 [ERROR] [MY-013236] [Server] The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it.
2020-06-14T13:15:43.865395Z 0 [ERROR] [MY-010119] [Server] Aborting
2020-06-14T13:15:43.865816Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.20-0ubuntu0.20.04.1)  (Ubuntu).
2020-06-14T13:15:44.192647Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.20-0ubuntu0.20.04.1) starting as process 38762
2020-06-14T13:15:44.220277Z 1 [ERROR] [MY-011011] [Server] Failed to find valid data directory.
2020-06-14T13:15:44.220414Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2020-06-14T13:15:44.220492Z 0 [ERROR] [MY-010119] [Server] Aborting
2020-06-14T13:15:44.227182Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.20-0ubuntu0.20.04.1)  (Ubuntu).

mysqld service status is as follows:

ubuntu@central-2:/etc$ systemctl status mysql.service
● mysql.service - MySQL Community Server
     Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sun 2020-06-14 13:15:48 UTC; 7min ago
    Process: 39031 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=1/FAILURE)

Jun 14 13:15:48 central-2 systemd[1]: mysql.service: Scheduled restart job, restart counter is at 7.
Jun 14 13:15:48 central-2 systemd[1]: Stopped MySQL Community Server.
Jun 14 13:15:48 central-2 systemd[1]: mysql.service: Start request repeated too quickly.
Jun 14 13:15:48 central-2 systemd[1]: mysql.service: Failed with result 'exit-code'.
Jun 14 13:15:48 central-2 systemd[1]: Failed to start MySQL Community Server.

journalctl -xe contains the following output around the time mysql is trying to start:

Jun 14 13:15:45 central-2 systemd[1]: Starting MySQL Community Server...
-- Subject: A start job for unit mysql.service has begun execution
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A start job for unit mysql.service has begun execution.
--
-- The job identifier is 2681.
Jun 14 13:15:46 central-2 mysql-systemd-start[38879]: MySQL system database not found in /var/lib/mysql. Please run mysqld --initialize.
Jun 14 13:15:46 central-2 systemd[1]: mysql.service: Control process exited, code=exited, status=1/FAILURE
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- An ExecStartPre= process belonging to unit mysql.service has exited.
--
-- The process' exit code is 'exited' and its exit status is 1.
Jun 14 13:15:46 central-2 systemd[1]: mysql.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- The unit mysql.service has entered the 'failed' state with result 'exit-code'.
Jun 14 13:15:46 central-2 systemd[1]: Failed to start MySQL Community Server.
-- Subject: A start job for unit mysql.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A start job for unit mysql.service has finished with a failure.
--
-- The job identifier is 2681 and the job result is failed.
Jun 14 13:15:46 central-2 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=mysql comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Jun 14 13:15:46 central-2 audit: EXECVE argc=5 a0="systemctl" a1="status" a2="--full" a3="--no-pager" a4="mysql.service"
Jun 14 13:15:46 central-2 systemd[1]: mysql.service: Scheduled restart job, restart counter is at 1.
-- Subject: Automatic restarting of a unit has been scheduled
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Automatic restarting of the unit mysql.service has been scheduled, as the result for
-- the configured Restart= setting for the unit.
Jun 14 13:15:46 central-2 systemd[1]: Stopped MySQL Community Server.
-- Subject: A stop job for unit mysql.service has finished
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A stop job for unit mysql.service has finished.
--
-- The job identifier is 2785 and the job result is done.
Jun 14 13:15:46 central-2 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=mysql comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jun 14 13:15:46 central-2 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=mysql comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

The root cause is the package libpam-tmpdir ( installed by https://github.com/konstruktoid/hardening/blob/master/scripts/17_packages )

This package change the format of the file or folder created by mktemp .

With libpam-tmpdir :

root@test-ub-64:~# mktemp
/tmp/user/0/tmp.qqRG7A24WV

Without libpam-tmpdir :

root@test-ub-64:~# mktemp
/tmp/tmp.HIUu5D514X

The format use by libpam-tmpdir break the script /var/lib/dpkg/info/mysql-server-8.0.postinst .

The simple way is :

  1. to remove the package
  2. exit your shell
  3. create a new root shell
  4. check with mktemp that file is like /tmp/tmp.XXXXXXXXXX