Collate output in Python logging MemoryHandler with SMTPHandler

You might want to use or adapt the BufferingSMTPHandler which is in this test script.

In general, you don't need to add a handler to a logger if it's the target of a MemoryHandler handler which has been added to a logger. If you set the level of a handler, that will affect what the handler actually processes - it won't process anything which is less severe than its level setting.

Instead of buffering for email, consider posting unbuffered to a message stream on a messaging app, e.g. on Matrix, Discord, Slack, etc. Having said that, I wrote my own beastly thread-safe implementation of BufferingSMTPHandler (backup link) which sends emails from a separate thread. The primary goal is to not block the main thread.

As written, it uses two queues - this seemed necessary in order to implement some useful class-level parameters that are defined in the "Configurable parameters" section of the code. Although you can use the code as-is, it's probably better if you study and use it to write your own class.

Issues:

• Some class-level parameters can perhaps be instance-level instead.
• Either threading.Timer or the signal module could perhaps be used to avoid loops that run forever.

For this purpose I use the BufferingSMTPHandler suggested by Vinay Sajip with one minor tweak: I set the buffer length to something really big (say 5000 log records) and manualy call the flush method of the handler every some seconds and after checking for internet conectivity.

# init
log_handler1 = BufferingSMTPHandler(
    'smtp.host.lala', "[email protected]", ['[email protected]'], 'Log event(s)',5000)
...
logger.addHandler(log_handler1)
...

# main code
...
if internet_connection_ok and seconds_since_last_flush>60:
    log_handler1.flush() # send buffered log records (if any)

If you are using django - here is simple buffering handler, which will use standard django email methods:

import logging

from django.conf import settings
from django.core.mail import EmailMessage


class DjangoBufferingSMTPHandler(logging.handlers.BufferingHandler):
    def __init__(self, capacity, toaddrs=None, subject=None):
        logging.handlers.BufferingHandler.__init__(self, capacity)

        if toaddrs:
            self.toaddrs = toaddrs
        else:
            # Send messages to site administrators by default
            self.toaddrs = zip(*settings.ADMINS)[-1]

        if subject:
            self.subject = subject
        else:
            self.subject = 'logging'

    def flush(self):
        if len(self.buffer) == 0:
            return

        try:
            msg = "\r\n".join(map(self.format, self.buffer))
            emsg = EmailMessage(self.subject, msg, to=self.toaddrs)
            emsg.send()
        except Exception:
            # handleError() will print exception info to stderr if logging.raiseExceptions is True
            self.handleError(record=None)
        self.buffer = []

In django settings.py you will need to configure email and logging like this:

EMAIL_USE_TLS = True
EMAIL_PORT = 25  
EMAIL_HOST = ''  # example: 'smtp.yandex.ru'
EMAIL_HOST_USER = ''  # example: '[email protected]'
EMAIL_HOST_PASSWORD = ''
DEFAULT_FROM_EMAIL = EMAIL_HOST_USER
SERVER_EMAIL = EMAIL_HOST_USER

LOGGING = {
    'handlers': {
        ...
        'mail_buffer': {
            'level': 'WARN',
            'capacity': 9999,
            'class': 'utils.logging.DjangoBufferingSMTPHandler',
            # optional: 
            # 'toaddrs': '[email protected]'
            # 'subject': 'log messages'
        }
    },
    ...
}

Updated Vinay Sajip's answer for python3.

import logging
from logging.handlers import BufferingHandler

class BufferingSMTPHandler(BufferingHandler):
    def __init__(self, mailhost, fromaddr, toaddrs, subject, capacity):
        logging.handlers.BufferingHandler.__init__(self, capacity)
        self.mailhost = mailhost
        self.mailport = None
        self.fromaddr = fromaddr
        self.toaddrs = toaddrs
        self.subject = subject
        self.setFormatter(logging.Formatter("%(asctime)s %(levelname)-5s %(message)s"))

    def flush(self):
        if len(self.buffer) > 0:
            try:
                import smtplib
                port = self.mailport
                if not port:
                    port = smtplib.SMTP_PORT
                smtp = smtplib.SMTP(self.mailhost, port)
                msg = '''From: {}\r\nTo: {}\r\nSubject: {}\r\n\r\n'''.format(
                            self.fromaddr,
                            ",".join(self.toaddrs),
                            self.subject
                            )
                for record in self.buffer:
                    s = self.format(record)
                    print (s)
                    msg = msg + s + "\r\n"
                smtp.sendmail(self.fromaddr, self.toaddrs, msg)
                smtp.quit()
            except:
                self.handleError(None)  # no particular record
            self.buffer = []

#update for @Anant
if __name__ == '__main__'
    buff_smtp_handler=BufferingSMTPHandler(...your args)
    buff_smtp_handler.setLevel(logging.ERROR)
    handlers=[buff_smtp_handler]
    logging.basicConfig(handlers=handlers)