Should I upgrade kernel packages on EC2 instances?
Solution 1:
The short answer is, yes, you should keep your systems up to date with respect to security patches.
How exactly you roll out the security patches depends on your tolerance for risk. Here are some options that I have used to answer this question in the past:
Apply the upgrades to a set of QA systems that mimic your production environment and run all your regression tests to make sure that the changes don't break any functionality or cause performance issues. Once you're satisfied, roll out the upgrades to your production systems.
Wait a day and see if there is a public outcry about problems caused by the updates. If all seems peaceful, upgrade your production systems.
Apply every security patch on your production systems as soon as it's available.
I have used a combination of all three of these approaches using Ubuntu, and have gradually moved towards option 3 over the years. Security patches are heavily tested before released and great care is taken to not break existing functionality. I've never had a problem upgrading within the Ubuntu supported images (though I did once have an issue years ago when I was using a non-Ubuntu kernel with Ubuntu on EC2).
Note that upgrading the kernel also requires a reboot to apply the changes.
The above experience and recommendations apply only to upgrading within an Ubuntu release (e.g., 11.04). Upgrading to a new Ubuntu release is a much larger and riskier task and definitely requires testing before you roll it out to your production systems.
Here's an article on this topic that was just published by RightScale about how to manage security upgrades in their environment:
http://blog.rightscale.com/2011/09/28/security-patching-in-the-rightscale-universe/