CentOS 7 ISO Kickstart Installation - "Pane is dead"

EDIT: I retried this generically, and see the same problem. To reproduce, follow my steps below. I am starting to think that it might be a problem with the hardware, but I am not certain.

When installing a CentOS 7 ISO using kickstart, I get an error saying "Pane is dead" with no additional logs. Basically, I take the CentOS 7 ISO and added a kickstart file. When installing with this on VirtualBox, everything works fine. When I attempt to install it on real hardware with multiple interfaces, I get the "Pane is dead" error after all the packages are installed.

enter image description here

I am at a loss as to what could be the problem. Some things I've tried:

  • There are no logs generated that indicate the problem. I've even added the logging --level=debug line to the kickstart, but there was nothing of value to see.
  • There is no post-installation step (I removed that from the ks since that is last log before the problem is seen), so that cannot be the issue.
  • I did see somewhere on the web that someone said there is a difference between CentOS 6 and 7 ISOs, but there was no additional information to indicate quantitatively what those differences might be.

Here are the complete instructions on how to reproduce. If you can see what I am doing wrong, I would greatly appreciate it.

Thanks!

Complete Instructions for How to Reproduce

My production machine is an up-to-date CentOS 7 server. If you are missing any applications, I am sure you can install them easily with yum.

The basic steps are:

  • Create a working directory
  • Create a kickstart file
  • Update isolinux.cfg
  • Download packages
  • Create repository
  • Create the ISO

Create Working Directory

I copied the CentOS 7 ISO to my temp directory and mounted it to get all the files:

>cd /tmp
>wget http://centos.mirror.iweb.ca/7/isos/x86_64/CentOS-7-x86_64-DVD-1503-01.iso
... hours later ...
>mkdir CentOS7-CD
>sudo mount -l /tmp/CentOS-7-x86_64-DVD-1503-01.iso /tmp/CentOS7-CD
mount: block device /tmp/CentOS-7-x86_64-DVD-1503-01.iso is write-protected, mounting read-only

I copied all the files out into a working directory.

>cd /tmp/
>cp -r CentOS7-CD/ CentOS7

Kickstart File

Add my kickstart file:

>cat << @@EOF@@ > /tmp/CentOS7/ks.cfg
#version=RHEL7
# System authorization information
auth --enableshadow --enablemd5
# Install OS instead of upgrade
install
# Reboot after installation
reboot --eject
# License agreement
eula --agreed
# Use text mode install
text
# Firewall configuration
firewall --disabled
firstboot --disable
ignoredisk --only-use=sda
# Keyboard layouts
# old format: keyboard us
# new format:
keyboard --vckeymap=us --xlayouts='us'
# System language
lang en_CA.UTF-8
# Installation logging level
logging --level=debug
# Network information
%include /tmp/network.ks

# Use CDROM installation media
cdrom
repo --name="CentOS" --baseurl=cdrom:sr0
# Use Network installation
#url --url=http://centos.mirror.iweb.ca/7/os/x86_64/

# Root password
rootpw --plaintext password
# SELinux configuration
selinux --disabled
# Do not configure the X Window System
skipx
# System timezone
timezone Etc/UTC --isUtc
# System bootloader configuration
bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=sda
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart --all --initlabel 
# Disk partitioning information
part / --fstype="ext4" --size=10000
part /data --fstype="ext4" --size=18461
part /info --fstype="ext4" --size=2000
part /boot --fstype="ext3" --size=256
part swap --fstype="swap" --size=2048

%packages --nobase --excludedocs
@core
-aic94xx-firmware
-alsa-*
-iwl*-firmware
-ivtv-firmware
-libertas-*-firmware
-btrfs-progs
-cronie-anacron
-cronie
-crontabs
-ModemManager-glib
-NetworkManager*
-postfix
-wpa_supplicant
%end

%addon com_redhat_kdump --enable --reserve-mb='auto'

%end

%pre
interface=$(ip addr | grep -i broadcast | awk '{ print $2 }' | head -1 | sed 's/://g')
echo "network --bootproto=dhcp --device=${interface} --onboot=on" > /tmp/network.ks
%end
@@EOF@@

Note: If using these instructions and cutting and pasting, you may get "> " prefixing every line. Use the following to remove them.

>sed -i 's|^> ||' /tmp/CentOS7/ks.cfg

I've tried to limit the number of packages installed, hence all the exclusions in the %packages section.

ISOLINUX.CFG

Next, edit my isolinux.cfg file and add inst.ks=cdrom:/dev/cdrom:/ks.cfg to each append line and make the non-test version the default.

>cd /tmp/CentOS7/isolinux/
>sed -i 's|\(LABEL=\S*\)|\1 inst.ks=cdrom:/dev/cdrom:/ks.cfg|' isolinux.cfg
# Delete the old default (check)
>sed -i '/menu default/d' isolinux.cfg
# Add the new default (linux)
>sed -i '0,/  kernel vmlinuz/s//  menu default\n&/' isolinux.cfg

Add Packages

I empirically determined that there are 240 packages to add for my minimal installation. I did this by performing a network installation on VirtualBox with the following changes to the kickstart file:

# Use CDROM installation media
#cdrom                                    # <-- comment this out for ntwk install
#repo --name="CentOS" --baseurl=cdrom:sr0 # <-- comment this out for ntwk install
# Use Network installation
url --url=http://centos.mirror.iweb.ca/7/os/x86_64/

So I commented out the CDROM installation, and instead used the network repository. Installation took 10 minutes. Once complete, I logged in to get the list of installed packages:

VB> rpm -qa > pkg.list

I copied this file back to my production machine and obtained the locations for downloading these files:

> cd /tmp/CentOS7
> rm -rf Packages
> cd Packages
# Copy the pkg.list from the VB server to this location
> cat pkg.list | xargs yumdownloader --urls | grep "^http" > pkg.urls
> wget -nv -nc -i pkg.urls

This takes a few minutes to complete as it downloads all the packages.

Create Repository

Creating the repository is a simple task.

>cd /tmp/CentOS7
>mv repodata/*comps.xml.gz comps.xml.gz
>gzip -d comps.xml
>rm -rf repodata
>mkdir repodata
>cp comps.xml repodata/
>createrepo -g repodata/comps.xml .

Create the ISO

>cd /tmp
>mkisofs -o custom.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -V 'CentOS 7 x86_64' -boot-load-size 4 -boot-info-table -R -J -v -T CentOS7/

I've finally solved my own issue! I'll put this as an answer so that if anyone else sees it they will know the cause. However, I assume only Canadians will be affected...

It turns out that the problem was with the lang en_CA.UTF-8 line in the kickstart file. I changed this to lang en_US.UTF-8 and the installation completes normally. I may ask another question on how to use en_CA but at least I solved this one.

Thanks!

For anyone who is interested, how I solved it was by doing this:

  • I created a DVD with the CentOS-7-x86_64-DVD-1503-01.iso on it
  • Put this on my server and booted
  • When asked, changed the grub configuration line and added text to it
  • During anaconda installation, I configured minimum installation over the network.
  • Installation completed and an anaconda-ks.cfg file was created in /root/
  • I copied this file off and used it exactly as my own template for the my ks.cfg file.
  • I also re-created the Packages and repodata directories based on the set of installed packages in this installation.
  • Created a new iso with the new kickstart file and installation worked!
  • Line by line, I changed the differences in the new ks.cfg file to match the old (problematic one), recreated the ISO and attempted installation.
  • Painfully found that the only line that caused the error was the lang en_CA.UTF-8 line.

The issue seems to be quite clear as I might say. Notice that the Pane is dead comes when the post-installation script runs.

I'd rather cut that section from ks file or even better log it to understand what exactly went wrong.

In order to log you can use --log /path/to/logfile directive in your kickstart. Once you got pane message just switch to one of TTYs and check the logs.

So the offending section is %post.

I hope this helps.