Alerting on Failed Puppet Status

Puppet's "tagmail" report might do what you want. See the official tagmail documentation for more information.