Last active
February 7, 2024 11:03
-
-
Save niranjv/fb95e716151642e8ca553b0e38dd152e to your computer and use it in GitHub Desktop.
Change Python logger format in AWS Lambda
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Python logger in AWS Lambda has a preset format. To change the format of the logging statement, | |
# remove the logging handler & add a new handler with the required format | |
import logging | |
import sys | |
def setup_logging(): | |
logger = logging.getLogger() | |
for h in logger.handlers: | |
logger.removeHandler(h) | |
h = logging.StreamHandler(sys.stdout) | |
# use whatever format you want here | |
FORMAT = '%(asctime)s %(message)s' | |
h.setFormatter(logging.Formatter(FORMAT)) | |
logger.addHandler(h) | |
logger.setLevel(logging.INFO) | |
return logger | |
def lambda_handler(event, context): | |
logger = setup_logging() | |
logger.info("This is a test log statement!") | |
return | |
# Expected output from Lambda: | |
# | |
# START RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f Version: $LATEST | |
# 2017-10-06 22:40:59,653 This is a test log statement! | |
# END RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f | |
# REPORT RequestId: 1a2b3c4d-abcd-1234-efgh-1a2b3c4d5e6f Duration: 0.41 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 21 MB |
I prefer keep the number of 3rd party dependencies in my lambdas low but I do like to have control of the logging configuration. I have had some success using something similar to the following:
import logging
import logging.config
config = {
'version': 1,
'formatters': {
'lambda': {
'class': "logging.Formatter",
'format': "%(asctime)s %(levelname)s %(aws_request_id)s %(module)s:%(lineno)d %(message)s"
}
},
'filters': {
'lambda': {
'()': '__main__.LambdaLoggerFilter'
}
},
'handlers': {
'lambda': {
'class': "logging.StreamHandler",
'level': "DEBUG",
'formatter': "lambda",
'filters': ['lambda']
}
},
'loggers': {
'lambda': {
'level': "DEBUG",
'handlers': ["lambda"],
'propagate': False
}
}
}
logging.config.dictConfig(config)
LOG = logging.getLogger('lambda')
def handle(event, context):
LOG.info('Logging from Lambda with Request ID.')
Obviously this comes with the risk of assuming AWS do not change the runtime bootstrapping code (tested on python3.8 runtime).
I was able to retain aws_request_id
in the log output:
https://gist.github.com/matthewpick/3aa01abfeda36eae717837a99994d3ed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Or... use the aws-lambda-powertools logger which has a lot of this, structured logs, and you get things like setting correlation ids.
And you can customize the formatter if you really need via https://awslabs.github.io/aws-lambda-powertools-python/latest/core/logger/#bring-your-own-formatter