Skip to content

Instantly share code, notes, and snippets.

@timss
Created April 11, 2017 07:56
Show Gist options
  • Save timss/8f03ae681256f21e25f8b0a16327c26c to your computer and use it in GitHub Desktop.
Save timss/8f03ae681256f21e25f8b0a16327c26c to your computer and use it in GitHub Desktop.
Python logging across multiple modules with custom handler/filter and splitting loglevels between stdout/stderr
#!/usr/bin/env python3
import logging
import sys
import sublog
logger = logging.getLogger()
# http://stackoverflow.com/a/24956305/1076493
# filter messages lower than level (exclusive)
class MaxLevelFilter(logging.Filter):
def __init__(self, level):
self.level = level
def filter(self, record):
return record.levelno < self.level
def main():
# redirect messages to either stdout or stderr based on loglevel
# stdout < logging.WARNING <= stderr
formatter = logging.Formatter('%(asctime)s %(levelname)s [%(module)s]: %(message)s')
logging_out = logging.StreamHandler(sys.stdout)
logging_err = logging.StreamHandler(sys.stderr)
logging_out.setFormatter(formatter)
logging_err.setFormatter(formatter)
logging_out.addFilter(MaxLevelFilter(logging.WARNING))
logging_out.setLevel(logging.DEBUG)
logging_err.setLevel(logging.WARNING)
# root logger, no __name__ as in submodules further down the hierarchy
global logger
logger.addHandler(logging_out)
logger.addHandler(logging_err)
logger.setLevel(logging.DEBUG)
logger.info("An INFO message from " + __name__)
logger.error("An ERROR message from " + __name__)
sublog.log()
sys.exit(1)
if __name__ == '__main__':
main()
$ python3 log.py
2017-04-11 09:54:55,316 INFO [log]: An INFO message from __main__
2017-04-11 09:54:55,316 ERROR [log]: An ERROR message from __main__
2017-04-11 09:54:55,316 INFO [sublog]: An INFO message from sublog
2017-04-11 09:54:55,316 ERROR [sublog]: An ERROR message from + sublog
#!/usr/bin/env python3
import logging
logger = logging.getLogger(__name__)
def log():
logger.info("An INFO message from " + __name__)
logger.error("An ERROR message from + " + __name__)
@MCilento93
Copy link

Hello @timss
Thank you for your snippets. I turned around the net to understand why a so simple task I was unable to do.
My error was to assign the name name to the logger, without assigning it as global.

Now I endided up with:

main.py

import logging

global logger
logger=logging.getLogger()
logger.setLevel(logging.DEBUG)
file_handler = logging.FileHandler(os.path.join(current_dir,'logs','logfile.log'))
formatter    = logging.Formatter('%(asctime)s : %(levelname)s : %(name)s : %(message)s')
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)
logger.info('started main')

module2.do_something()

module2.py

import logging
logger = logging.getLogger(__name__)
def do_something():
   logger.info('write something')

output

2020-11-05 11:47:30,855 : INFO : root : started main
2020-11-05 11:47:33,915 : INFO : module2 : write something

@timss
Copy link
Author

timss commented Nov 8, 2020

@MCilento93 Glad to hear it was useful for someone! :)

@MCilento93
Copy link

My pleasure.
Actually even without the global dichiaration it is working 🤗

@pratik-m
Copy link

this helped!

@gbernat
Copy link

gbernat commented Jul 7, 2021

It saved my day!
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment