Easiest way to automatically check EC2 disk space and be alerted if it is running low?
Running the Amazon Linux AMI. It seems that CloudWatch does not check for free disk space. I have a number of servers and ideally don't want to have to configure each one with a mail server, script to check disk space etc.
Is there a simpler way to do this?
Amazon provides scripts for this as of march 2012:
Amazon CloudWatch Monitoring Scripts for Linux: http://aws.amazon.com/code/8720044071969977
There is no way for the EC2 control and monitoring tools to give you this data because the file system of your instances is ONLY accessible by the instance itself. Both the basic architecture of the hardware and the security model demand this limitation. Think about how bad it would be if software outside your computer could poke around at the files on your hard drives!
Here is a low key way to make cron (installed on most systems anyway) check this data for your periodically. Your systems should have the minimum requirements to handle root mail notifications anyway. I recommend having at least a materialistic outgoing mail agent and configuring the root or administrator alias to forward to you on all systems you administer. Many programs including cron
expect this configuration.
You could add this to your crontab:
0 0 * * * test $(df / | grep ^/ | awk '{print $4}') -lt 1048576 && echo "Warning: Free disk space is less than 1G on /"
To break that down, this
- Creates a job that runes once a day at 00:00.
- Cron automatically handles emailing the system administrator with the output of jobs. This job only produces output if there is an error or if the disk space is low
- The
test
command sets up a simple shell comparison using the-lt
less than operator and a fixed value equivolent to 1Gb free space. - The
df
command tests free space on the/
file system - The
grep
gets you just the line of output you need instead of the headersdf
includes. - The
awk
get's just the fourth column in the output, the free space number. - The
&&
says to run the next command only if the first one (thetest x -lt y
) returns true.
I wrote a script as I needed to check several servers within my EC2 group. It needs a file with a list of each server IP/domain name on a single line.
#! /bin/bash
ADMIN="[email protected]"
ALERT=85
for SERVER in `cat ~/scripts/servers.txt` do
ssh -i ~/.ssh/yourkey.pem $SERVER df -H | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $1 }' | while read output;
do
echo $output
usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
partition=$(echo $output | awk '{ print $2 }' )
if [ $usep -ge $ALERT ]; then
echo "Running out of space \"$partition ($usep%)\" on $SERVER as on $(date)" |
mail -s "Alert: Almost out of disk space $usep" $ADMIN
fi
done done
Step by step instructions for setting this up on an EC2 instance with CloudWatch:
http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-scripts.html