Monitor Docker container logs with AWS CloudWatch

awslogs.md

Monitor Docker container logs with AWS CloudWatch

In this post, we are going to see how to monitor docker logs from AWS Cloudwatch logs. Docker supports different logging drivers one of which is awslogs

For this excersice, Docker container need NOT be running on EC2. We can send docker logs from anywhere to AWS.

Prerequisite

• Ubuntu 14.04
• Docker version 1.12 and above
• AWS account

AWS Client Setup

1. Install AWS client

   pip install awscli

2. Configure aws client with your AWS Access Key and AWS Access Key ID

   aws configure

   Enter AWS Access Key ID

   Enter AWS Secret Access Key

   Enter region us-west-2

3. Verify Configuration

   • Configuration will be stored at ~/.aws/config
   • Credentials will stored at ~/.aws/credentials

Docker Setup

1. Configure docker deamon

   sudo vim /etc/default/docker

   Add following lines to this file

   export AWS_ACCESS_KEY_ID=your access key id

   export AWS_SECRET_ACCESS_KEY=your secret key

   export AWS_SHARED_CREDENTIALS_FILE=/home/you/.aws/credentials

   sudo service docker restart

AWS CloudWatch settings

1. Create a log group

aws logs create-log-group --log-group-name my-group

2. Create a log stream

aws logs create-log-stream --log-group-name my-group --log-stream-name my-stream

Run Docker container with awslogs driver

sudo docker run -it -d --log-driver="awslogs" --log-opt awslogs-region="us-west-2" --log-opt awslogs-group="my-group" --log-opt awslogs-stream="my-stream" busybox echo "hello docker fans. This message is going to CloudWatch"

Log into AWS Console and Check logs ( See the following Screenshot )

awslogs.png

This doesn't seem to work anymore, at least for me. Ubuntu 18.04 and up uses Systemd, so slightly different server configuration is required.

Also, would be nice to explain the steps using an IAM user.