Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Change Python logger format in AWS Lambda
# Python logger in AWS Lambda has a preset format. To change the format of the logging statement,
# remove the logging handler & add a new handler with the required format
import logging
import sys
def setup_logging():
logger = logging.getLogger()
for h in logger.handlers:
logger.removeHandler(h)
h = logging.StreamHandler(sys.stdout)
# use whatever format you want here
FORMAT = '%(asctime)s %(message)s'
h.setFormatter(logging.Formatter(FORMAT))
logger.addHandler(h)
logger.setLevel(logging.INFO)
return logger
def lambda_handler(event, context):
logger = setup_logging()
logger.info("This is a test log statement!")
return
# Expected output from Lambda:
#
# START RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f Version: $LATEST
# 2017-10-06 22:40:59,653 This is a test log statement!
# END RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f
# REPORT RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f Duration: 0.41 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 21 MB
@davi5e

This comment has been minimized.

Copy link

davi5e commented Sep 22, 2017

doesn't seem to be working anymore... at least in my case

@niranjv

This comment has been minimized.

Copy link
Owner Author

niranjv commented Oct 6, 2017

Still works. I updated the gist to make it a self-contained Python 3.6 function that you can copy & paste in to the AWS Lambda Console in a web browser. Expected output is shown at the bottom. The string "This is a test log statement" should be present in the output. All other output details will differ.

@mikeokner

This comment has been minimized.

Copy link

mikeokner commented Dec 22, 2017

Lambda appears to be doing some additional trickery behind the scenes in its default handler to convert newlines within messages to \r so that CloudWatch groups them together in one log. Replacing the handler loses this behavior, so tracebacks then get split across multiple entries.

In order to preserve the original behavior and keep tracebacks grouped together in one message, you can just update the format of the existing handler like:

FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
logger = logging.getLogger()
for h in logger.handlers:
  h.setFormatter(logging.Formatter(FORMAT))
logger.setLevel(logging.INFO)
@mrtj

This comment has been minimized.

Copy link

mrtj commented Mar 22, 2018

I managed to extract the default log format from lambda runtime and it is:

"[%(levelname)s]\t%(asctime)s.%(msecs)dZ\t%(aws_request_id)s\t%(message)s\n"

The most interesting thing here is aws_request_id that is the identifier of the lambda request, injected by the default formatter when running on lambda. So if you like the default format you can just insert %(name)s somewhere in the string above.
For completeness, the default date format is the standard ISO format:

"%Y-%m-%dT%H:%M:%S"

The code to get these two strings:

logger = logging.getLogger()
return [{ 'fmt' : handler.formatter._fmt, 'datefmt': handler.formatter.datefmt } for handler in logger.handlers]
@thnee

This comment has been minimized.

Copy link

thnee commented Jul 26, 2018

Thanks for pointing in the right direction, but:

You do not need to remove the handler

You can just call setFormatter on an existing handler.

You should not apply a change to all handlers

Regardless of if you remove a handler, or just change the formatter, your loop is overly broad.

If you make a change to all handlers in root.handlers, you run the risk of hitting a handler that you did not intend to. It makes the code brittle and non-portable. At the very least check the type of the handler before applying your change to it.

However, when I do type(handler) I get <class '__main__.LambdaLoggerHandler'> instead of the expected <class 'awslambda.bootstrap.LambdaLoggerHandler'>, not sure what is up with that.

@ryanjdillon

This comment has been minimized.

Copy link

ryanjdillon commented Feb 14, 2019

@thnee What is the correct handler to change the formatter for? I would simply like to add the logger name to the handler as @mrtj mentions, so I will have a better idea where an error occurs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.