Repeat monit alerts
How can I have monit continue to alert me on an interval until the condition has been fixed? Here's an example config:
check filesystem datafs with path /dev/sda1 if space usage > 80% for 5 times within 15 cycles then alert
Here I'l get an alert once and then no more. I want monit to not shut up until the problem is fixed.
From the monit manpage:
alert foo@bar with reminder on 1 cycle
This will repeat the alert every time the process check repeats once it is in a failed state (eg, quite a lot). You may want to set this so it only reminds you ever hour or less
This seems to be set where you'd set an alert destination, rather than in the check stanza. However, you can specify an alert destination for individual checks/services, as well as the global one.
check filesystem datafs with path /dev/sda1
alert foo@bar on { resource } with reminder on 10 cycles
if space usage > 80% for 5 times within 15 cycles then alert
The default is to alert when the trigger matches, and again when the service succeeds. So if you flap between 79% and 81% disk utilization, you'll receive alerts as the thresholds are reached.
However, you want reminders after an alert has been triggered. You can use the reminder functionality within Monit to do this.
Here's a sample config from Monit:
For example if you want to be notified each tenth cycle if a service remains in a failed state, you can use:
alert foo@bar with reminder on 10 cycles
Likewise if you want to be notified on each failed cycle, you can use:
alert foo@bar with reminder on 1 cycle
You can extend your stanza with this language:
check filesystem datafs with path /dev/sda1
if space usage > 80% for 5 times within 15 cycles then alert
But you'll have to modify your global set alert
statement or change it at the check stanza level:
set alert [email protected] with reminder on 5 cycles