Skip to content

Instantly share code, notes, and snippets.

@niranjv
Last active February 7, 2024 11:03
Show Gist options
  • Save niranjv/fb95e716151642e8ca553b0e38dd152e to your computer and use it in GitHub Desktop.
Save niranjv/fb95e716151642e8ca553b0e38dd152e to your computer and use it in GitHub Desktop.
Change Python logger format in AWS Lambda
# Python logger in AWS Lambda has a preset format. To change the format of the logging statement,
# remove the logging handler & add a new handler with the required format
import logging
import sys
def setup_logging():
logger = logging.getLogger()
for h in logger.handlers:
logger.removeHandler(h)
h = logging.StreamHandler(sys.stdout)
# use whatever format you want here
FORMAT = '%(asctime)s %(message)s'
h.setFormatter(logging.Formatter(FORMAT))
logger.addHandler(h)
logger.setLevel(logging.INFO)
return logger
def lambda_handler(event, context):
logger = setup_logging()
logger.info("This is a test log statement!")
return
# Expected output from Lambda:
#
# START RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f Version: $LATEST
# 2017-10-06 22:40:59,653 This is a test log statement!
# END RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f
# REPORT RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f Duration: 0.41 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 21 MB
@ahlinc
Copy link

ahlinc commented Aug 4, 2020

has anyone managed to find a nice way to get "aws_request_id" when using the approach of removing the default handler and using your own?

It's accessible through second handler parameter usually named as the context:

def handler(event, context):
    aws_request_id = context['aws_request_id']

@joshenders
Copy link

A bit inefficient but this worked for me:

log = logging.getLogger()
for h in log.handlers:
    h.setFormatter(logging.Formatter("%(aws_request_id)s [%(levelname)s] %(message)s"))

log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)

@scirelli-c1
Copy link

scirelli-c1 commented Jul 8, 2021

Why not just create a logger, add your handler and then set propagate to False?

    FORMAT = '%(asctime)s %(message)s'
    h = logging.StreamHandler(sys.stdout)
    h.setFormatter(logging.Formatter(FORMAT))

    logger = logging.getLogger(__name__)
    logger.addHandler(h)
    logger.setLevel(logging.INFO)
    logger.propagate = False

This would prevent anything being logged using logger from propagating up to parent loggers. In the Lambda case the logger that Lambda adds.

@jhodge-chwy
Copy link

Or... use the aws-lambda-powertools logger which has a lot of this, structured logs, and you get things like setting correlation ids.

And you can customize the formatter if you really need via https://awslabs.github.io/aws-lambda-powertools-python/latest/core/logger/#bring-your-own-formatter

@mubeta06
Copy link

I prefer keep the number of 3rd party dependencies in my lambdas low but I do like to have control of the logging configuration. I have had some success using something similar to the following:

import logging
import logging.config


config = {
    'version': 1,
    'formatters': {
      'lambda': {
        'class': "logging.Formatter",
        'format': "%(asctime)s %(levelname)s %(aws_request_id)s %(module)s:%(lineno)d %(message)s"
      }
    },
    'filters': {
        'lambda': {
            '()': '__main__.LambdaLoggerFilter'
        }
    },
    'handlers': {
      'lambda': {
        'class': "logging.StreamHandler",
        'level': "DEBUG",
        'formatter': "lambda",
        'filters': ['lambda']
      }
    },
    'loggers': {
      'lambda': {
        'level': "DEBUG",
        'handlers': ["lambda"],
        'propagate': False
      }
    }
}

logging.config.dictConfig(config)

LOG = logging.getLogger('lambda')

def handle(event, context):
    LOG.info('Logging from Lambda with Request ID.')

Obviously this comes with the risk of assuming AWS do not change the runtime bootstrapping code (tested on python3.8 runtime).

@matthewpick
Copy link

I was able to retain aws_request_id in the log output:
https://gist.github.com/matthewpick/3aa01abfeda36eae717837a99994d3ed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment