Reboot fails on RHEL 6.1 after kernel update on Amazon EC2
Everytime I start a new RHEL instance, I like to perform a yum update to start with the most up-to-date system. However, everytime I try to reboot the instance after that, the instance never goes live anymore.
I troubleshooted the root cause to be the kernel update, here are the minimum steps to reproduce the problem:
- Launch a new RHEL 6.1 instance
- Connect, then run:
yum update kernel*reboot
The system stays down. The system log from the EC2 management console doesn't help much:
...
Please stand by while rebooting the system...
md: stopping all md devices.
xenbus_dev_shutdown: device/console/0: Initialising != Connected, skipping
Restarting system.
No mention of a kernel panic, startup problem, etc. Just stuck there.
From what I can read around, it looks like you can't use the default kernels with an EC2 instance. If that is true, how can I avoid to accidentally update the kernel with yum, and thus killing my instance, when performing a generic yum update?
just edit the file: /etc/yum.conf and add an entry in it like this:
exclude = kernel*
After it, yum will exclude all packages, having kernel in its starting name.
You can read more details in this url:
http://www.cyberciti.biz/faq/redhat-centos-linux-yum-update-exclude-packages/
Regarding the issue that you are unable to upgrade kernel, then you are right that you CANNOT upgrade the kernel like you upgrade in normal PC. Amazon EC2 uses customized settings in case of kernel setup, for detailed info, please follow this URL that how to perform kernel upgrade on EC2 instance:
http://www.vincestross.com/2009/04/upgrade-an-ec2-instance/
It looks like you've encountered a bug in the version of the kernel included in RHEL. There was a thread about this problem on the Xen development email list last year: http://old-list-archives.xen.org/archives/html/xen-devel/2010-11/msg00616.html
You should probably try rebooting your instance via the EC2 console or command line utilities.
Regarding the issue that you are unable to upgrade the kernel: that is no longer the case for modern Amazon Machine Images (AMIs). Newer AMIs like the RHEL 6 AMI boot with PV-GRUB, which allows you to upgrade the kernel like you would on any other server.
For more information about PV-GRUB, refer to the EC2 documentation here: http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?UserProvidedkernels.html
the problem is the name of the block device changes from /dev/xvda to /dev/xvde you can change the menu.lst to point to /dev/xvde1 and fstab to mount xvde 1 2 3 or label the drives and poit to the labels
i use labels and the problem is fixed.
I was really blocked by this issue but it seems I have got arround it now. The kernel hint doesn't seem to be relevant now, but I found that removing /var/lib/dhcp-client/* files is really helping.
Taken directly by some tech-trainers at AWSomeday, the 2 checks done by Amazon in the monitoring are :
- CPU is not IDLE
- Network interface is present
So first thing to do is to check at what monitoring step you are going to. If 1/2 are OK it might be a good chance that you have a DHCP problem. And so above solution might work for you.