Logging different python files in the same file

I have a custom module called bar.py, which I import in some of my scripts. It is actually imported via pip install from the pypi index. That bar.py module looks like this (this is actually an oversimplified example of what I actually have so I can ringfence my doubt here.

import logging

def do_bar_logged():
    logging.INFO('START DOING SOMETHING')
    do_bar()
    logging.INFO('END DOING SOMETHING')

One of the scripts in which I import bar's do_bar_logged is called foo.py. It looks something like this.

import logging
from bar import do_bar_logged  # I need to install via pip bar before being able to import this, just like a regular module.

def main():
    logging.INFO('START CALLING DO_BAR')
    do_bar_logged()
    logging.INFO('END CALLING DO_BAR')

My question here is: would it be possible to have both loggings pointing to the same file without harcoding anything?

Solution 1:

You can use logging module like this

bar.py

import logging


def aux_print():
    logging.warning('This is my auxiliary script')

main.py

import logging
from bar import aux_print

logging.basicConfig(
    filename="log.txt",
    filemode='a',
    format='[%(asctime)s,%(msecs)d] [%(levelname)s]: %(message)s',
    datefmt='%H:%M:%S',
    level=logging.DEBUG
)


def main():
    logging.info('This is my main script')
    aux_print()


if __name__ == '__main__':
    main()

Output log.txt:

[18:58:23,708] [INFO]: This is my main script
[18:58:23,708] [WARNING]: This is my auxiliary script

Solution 2:

The logging module lets you replace calls to print with calls to special logging functions that can be configured to run in all sorts of ways. The top end logging calls stay the same while you substitute back end handlers to log where you want.

Logging multiple scripts to a single destination can be tricky. You can't just write a single file because the logs will tend to overwrite each other (a locking file handler can help). One of the simplest things to do is use the SysLogHandler (linux/mac) and NTEventLogHandler (windows) handlers to log to the regular system log.

Or you could use some of the low level handlers (HTTP, socket, dgram, queue), to write your own logging server that aggregates logs. You could even use a SQL or NOSQL handler to log to a database. Logging is a big deal and there are lots of 3rd party logging modules out there. Its worth a look to see if any are useful to you.