cloudwatch_ec2_integration.md

Make AWS IAM role/user for access cloudwatch

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "logs:DescribeLogStreams"
            ],
            "Resource": [
                "arn:aws:logs:us-west-2:374287944309:log-group:mmc-application:log-stream:*"
            ]
        }
    ]
}

Install cloudwatch

curl https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O
sudo python ./awslogs-agent-setup.py --region us-west-2

Launching interactive setup of CloudWatch Logs agent ...

Step 1 of 5: Installing pip ...DONE

Step 2 of 5: Downloading the latest CloudWatch Logs agent bits ... DONE

Step 3 of 5: Configuring AWS CLI ...
AWS Access Key ID [None]: SAHBcdsksadjhneuindcasadcf
AWS Secret Access Key [None]: CsadzJBgBesadK7addffdsgvfvtbsfdsfeafGB
Default region name [us-west-2]:
Default output format [None]:

Step 4 of 5: Configuring the CloudWatch Logs Agent ...
Path of log file to upload [/var/log/syslog]: /var/log/tomcat7/catalina.out
Destination Log Group name [/var/log/tomcat7/catalina.out]: mmc-application

Choose Log Stream name:
  1. Use EC2 instance id.
  2. Use hostname.
  3. Custom.
Enter choice [1]: 3
Enter Log Stream name [None]: tomcat_mmc_98

Choose Log Event timestamp format:
  1. %b %d %H:%M:%S    (Dec 31 23:59:59)
  2. %d/%b/%Y:%H:%M:%S (10/Oct/2000:13:55:36)
  3. %Y-%m-%d %H:%M:%S (2008-09-08 11:52:54)
  4. Custom
Enter choice [1]: 1

Choose initial position of upload:
  1. From start of file.
  2. From end of file.
Enter choice [1]: 2
More log files to configure? [Y]: N

Step 5 of 5: Setting up agent as a daemon ...DONE

- Configuration file successfully saved at: /var/awslogs/etc/awslogs.conf
- You can begin accessing new log events after a few moments at https://console.aws.amazon.com/cloudwatch/home?region=us-west-2#logs:
- You can use 'sudo service awslogs start|stop|status|restart' to control the daemon.
- To see diagnostic information for the CloudWatch Logs Agent, see /var/log/awslogs.log
- You can rerun interactive setup using 'sudo python ./awslogs-agent-setup.py --region us-west-2 --only-generate-config'

File /var/awslogs/etc/awslogs.conf

[/var/log/tomcat7/catalina.out]
datetime_format = %b %d %H:%M:%S
file = /var/log/tomcat7/catalina.out
buffer_duration = 5000
log_stream_name = tomcat_mmc_98
initial_position = end_of_file
log_group_name = mmc-application

[/var/log/tomcat7/catalina.out]
datetime_format = %b %d %H:%M:%S
file = /opt/tomcat7_odc/logs/catalina.out
buffer_duration = 5000
log_stream_name = tomcat_mmc_98
initial_position = end_of_file
log_group_name = odc-application

Cloudwatch integration Disk usage monitor

1. Download script which send data to cloudwatch and install packages

sudo apt-get -y install libwww-perl libdatetime-perl
sudo wget http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip -O /opt/CloudWatchMonitoringScripts-1.2.1.zip
cd /opt && sudo unzip /opt/CloudWatchMonitoringScripts-1.2.1.zip
cd aws-scripts-mon && sudo cp awscreds.template awscreds.conf

2. Create cronjob file /etc/cron.d/cloudwatch-monitor

*/5 * * * * root /opt/aws-scripts-mon/mon-put-instance-data.pl --disk-space-util --disk-path=/ --from-cron

3. Change permission of cronjob file and restart cron to reload new cron job

chmod +x /etc/cron.d/cloudwatch-monitor
/etc/init.d/cron restart

4. Create IAM user with following policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1471956281000",
            "Effect": "Allow",
            "Action": [
                "cloudwatch:PutMetricData"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

5. Update application AWS access file /opt/aws-scripts-mon/awscreds.conf

AWSAcdfsdcessKeyId=AKsadfas7BIGsasaddsadffvF6RZsadFZW3A AWSSecretKey=CzJBgsadsfBeKasdrfdsfdasfdrgtrgtsadgkluiGB