Skip to content

Instantly share code, notes, and snippets.

@danteay
Last active September 16, 2020 04:09
Show Gist options
  • Save danteay/98d2d3a102abb1f9ec3bbde02f7713ba to your computer and use it in GitHub Desktop.
Save danteay/98d2d3a102abb1f9ec3bbde02f7713ba to your computer and use it in GitHub Desktop.
Add logs from EC2 instance to CloudWatch logs

CloudWatch-EC2

Agent Config

1.- Create new Log Group in CloudWatch

2.- Download agent in EC2 instance

sudo wget http://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py

2.- Run Agent

sudo chmod +x awslogs-agent-setup.py
sudo python ./awslogs-agent-setup.py --region=us-west-2

Output from the shell:

Step 1 of 5: Installing pip ...DONE
Step 2 of 5: Downloading the latest CloudWatch Logs agent bits ... DONE
Step 3 of 5: Configuring AWS CLI ... 

If the step 2 of 5 fails run the next command and excec the python installer again:

export LC_ALL=C
  • Insert AWS_KEY and AWS_SECRET

    AWS Access Key ID [****************ASEA]: xxxYOUR_KEYxxx
    AWS Secret Access Key [****************Mg6d]: xxxYOUR_SECRETxxx
    
  • Define AWS region and output format

    Default region name [us-west-2]: us-west-2 
    Default output format [None]: text
    

Output from the shell:

Step 4 of 5: Configuring the CloudWatch Logs Agent ... 
  • Define log file path and CloudWatch group
    Path of log file to upload [/var/log/syslog]: /my/path/msg.log
    Destination Log Group name [/home/ubuntu/awslogs/foreman.log]: my-cloudwatch-group
    

3.- Define log-stream name

Choose Log Stream name:
1. Use EC2 instance id.
2. Use hostname.
3. Custom.
Enter choice [1]: 1

4.- Define timestamp format

Choose Log Event timestamp format:
1. %b %d %H:%M:%S    (Dec 31 23:59:59)
2. %d/%b/%Y:%H:%M:%S (10/Oct/2000:13:55:36)
3. %Y-%m-%d %H:%M:%S (2008-09-08 11:52:54)
4. Custom
Enter choice [1]: 3

5.- Define initial position

Choose initial position of upload:
1. From start of file.
2. From end of file.
Enter choice [1]: 2
More log files to configure? [Y]: n

6.- Verify AWSlogs status

sudo service awslogs status

Manual Config

Modify Config file as a root:

sudo nano /var/awslogs/etc/awslogs.conf

Modify this lines at the end:

[/my/path/msg.log]
datetime_format = %Y-%m-%d %H:%M:%S
file = /my/path/msg.log
buffer_duration = 5000
log_stream_name = {instance_id}
initial_position = end_of_file
log_group_name = my-cloudwatch-group
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment