Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

A guide to setting up cloudwatch agents in AWS if you do not have internet access

1. Adds the needed endpoints:

The folowing endpoints must be created inside you AWS VPC

ssm.region.amazonaws.com
ssmmessages.region.amazonaws.com
ec2messages.region.amazonaws.com
logs.region.amazonaws.com

https://docs.aws.amazon.com/systems-manager/latest/userguide/setup-create-vpc.html

If you use the terraform-aws-modules/vpc/aws module for vpcs you can enable these endpoints with the folowing Note: enable_ssmmessages_endpoint not strictly required (it allows remote ssh acess via the gui)

 //allows ssm to manadge ec2 instances
  enable_ssm_endpoint              = true
  ssm_endpoint_private_dns_enabled = true
  ssm_endpoint_security_group_ids  = [aws_security_group.ssm.id] # should allow port 443 https ingress
  //allows ssh sesshon from ssm
  enable_ssmmessages_endpoint              = true
  ssmmessages_endpoint_private_dns_enabled = true
  ssmmessages_endpoint_security_group_ids  = [aws_security_group.ssm.id]
  //allows run command from ssm
  enable_ec2messages_endpoint              = true
  ec2messages_endpoint_private_dns_enabled = true
  ec2messages_endpoint_security_group_ids  = [aws_security_group.ssm.id]
  //allows acess to cloudwatch
  enable_logs_endpoint = true
  logs_endpoint_private_dns_enabled = true
  logs_endpoint_security_group_ids = [aws_security_group.ssm.id]

2. Create a config for cloud watch

Here is an example of config to monitor a test log file this should be saved into parameter store

{
	"agent": {
		"metrics_collection_interval": 60,
		"run_as_user": "cwagent"
	},
	"logs": {
		"logs_collected": {
			"files": {
				"collect_list": [
					{
						"file_path": "/tmp/testlog.txt",
						"log_group_name": "testlog.txt",
						"log_stream_name": "{instance_id}"
					}
				]
			}
		}
	},
	"metrics": {
		"append_dimensions": {
			"AutoScalingGroupName": "${aws:AutoScalingGroupName}",
			"ImageId": "${aws:ImageId}",
			"InstanceId": "${aws:InstanceId}",
			"InstanceType": "${aws:InstanceType}"
		},
		"metrics_collected": {
			"disk": {
				"measurement": [
					"used_percent"
				],
				"metrics_collection_interval": 60,
				"resources": [
					"*"
				]
			},
			"mem": {
				"measurement": [
					"mem_used_percent"
				],
				"metrics_collection_interval": 60
			}
		}
	}
}

A wisard on the ec2 instance can also generate these configs wiht user input: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-cloudwatch-agent-configuration-file-wizard.html

3. Permissions

The ec2 instances need to be attached to a role with the following AWS managed policies:

  • CloudWatchAgentServerPolicy
  • AmazonSSMManagedInstanceCore
  • AmazonSSMDirectoryServiceAccess

4. Start cloudwatch agent

once all the flowing has been done there are ssm cmds to run to install and configure the cloud watch agent on the instance.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-on-EC2-Instance-fleet.html

A better solution may be to install the agent during the build of amis and the configure step as a user_data bootstrap step

A recurring configure step could be secdualed so that changes to config are pushed to ec2 instances automatically (else the change will have to be run manually)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment