logging with filters

I'm using Logging (import logging) to log messages.

Within 1 single module, I am logging messages at the debug level my_logger.debug('msg');

Some of these debug messages come from function_a() and others from function_b(); I'd like to be able to enable/disable logging based on whether they come from a or from b;

I'm guessing that I have to use Logging's filtering mechanism.

Can someone show me how the code below would need to be instrumented to do what I want?

import logging
logger = logging.getLogger( "module_name" )

def function_a( ... ):
    logger.debug( "a message" )

def function_b( ... ):
    logger.debug( "another message" )

if __name__ == "__main__":
    logging.basicConfig( stream=sys.stderr, level=logging.DEBUG )

    #don't want function_a()'s noise -> ....
    #somehow filter-out function_a's logging
    function_a()

    #don't want function_b()'s noise -> ....
    #somehow filter-out function_b's logging
    function_b()

If I scaled this simple example to more modules and more funcs per module, I'd be concerned about lots of loggers;

Can I keep it down to 1 logger per module? Note that the log messages are "structured", i.e. if the function(s) logging it are doing some parsing work, they all contain a prefix logger.debug("parsing: xxx") - can I somehow with a single line just shut-off all "parsing" messages (regardless of the module/function emitting the message?)

Solution 1:

Just implement a subclass of logging.Filter: http://docs.python.org/library/logging.html#filter-objects. It will have one method, filter(record), that examines the log record and returns True to log it or False to discard it. Then you can install the filter on either a Logger or a Handler by calling its addFilter(filter) method.

Example:

class NoParsingFilter(logging.Filter):
    def filter(self, record):
        return not record.getMessage().startswith('parsing')

logger.addFilter(NoParsingFilter())

Or something like that, anyway.

Solution 2:

Do not use global. It's an accident waiting to happen.

You can give your loggers any "."-separated names that are meaningful to you.

You can control them as a hierarchy. If you have loggers named a.b.c and a.b.d, you can check the logging level for a.b and alter both loggers.

You can have any number of loggers -- they're inexpensive.

The most common design pattern is one logger per module. See Naming Python loggers

Do this.

import logging

logger= logging.getLogger( "module_name" )
logger_a = logger.getLogger( "module_name.function_a" )
logger_b = logger.getLogger( "module_name.function_b" )

def function_a( ... ):
    logger_a.debug( "a message" )

def function_b( ... ):
    logger_b.debug( "another message" )

if __name__ == "__main__":
    logging.basicConfig( stream=sys.stderr, level=logging.DEBUG )
    logger_a.setLevel( logging.DEBUG )
    logger_b.setLevel( logging.WARN )

    ... etc ...