ec2 rebooted my instance?
I found out that my CentOS 5 server on EC2 was rebooted, without my command to do so. I did not even log in on the day it was rebooted. When I look at 'last' in linux, it says:
jeroen pts/0 128.97....... Thu Apr 7 15:02 - 16:28 (01:25)
reboot system boot 2.6.18-xenU-ec2- Wed Apr 6 15:48 (1+05:27)
jeroen pts/2 128.97....... Tue Apr 5 19:31 - 23:17 (03:45)
So it looks like nobody logged on that day. Any suggestions to what could have happened? Does EC2 reboot instances under any circumstances? Or might it be hacked?
Solution 1:
This happens occasionally when Amazon is looking to decommission the physical server your instance is running on. They might be killing the server to replace a failing hardware component or the server might have simply reached its end of life. Either way from what I understand, their process works something like this:
- The physical server gets marked as decommissioned so no new instances get launched on it.
- If they can (e.g. they're not dealing with a critical hardware failure), Amazon will wait for some period of time to see if the instances running on the server shutdown or reboot on their own (rebooting an EC2 instance usually results in it getting launched on a different physical server).
- After that period completes, Amazon will force the remaining instances to reboot moving them to other physical servers.
As a general rule of thumb, due to this and other quirks in the environment, you should treat any individual server running on EC2 as Ephemeral. Any server may reboot or flat out disappear at any time. Accordingly automating deployment with something like chef or puppet, having solid monitoring, and designing your application to work around failure is critical.
Solution 2:
Maybe an EC2 problem. See Reboot?
I had the same problem and I found in AWS Service Health Dashboard that there had been a problem in EC2.
Conclusion : Reboots should be expected, but no more frequently than you'd expect them with commodity hardware.