Skip to content

Instantly share code, notes, and snippets.

@misostack
Created January 6, 2024 01:46
Show Gist options
  • Save misostack/31c5599b7f589d6cbea4f6fae5ce7eb2 to your computer and use it in GitHub Desktop.
Save misostack/31c5599b7f589d6cbea4f6fae5ce7eb2 to your computer and use it in GitHub Desktop.
AWS Labs

Design a good architecture with AWS

image

@misostack
Copy link
Author

misostack commented Jan 8, 2024

How to create a secure AWS RDS instances in private subnet

You can't select subnet for your AWS DB Instances

Managed-RDS: MYSQL, PostgreSQL ( Full Managed )
RDS Aurora (Full Managed and ServerLess)

All you need to do is

  • Create subnets in diffrent AZs
  • Create subnet group
  • Select VPC
  • Select AZs
  • Select Subnets

Create subnets in diffrent AZs

image
image
image

Create subnet group and select your VPC, AZs, Subnets
image
image

Create your RDS instance with selected VPC, subnet group, AZ

image

Grab a cup of coffee while you're waiting for your RDS instance is ready to used
image

After few minutes
image

image
image

Test connection with nodejs and mysql2 package

const mysql = require("mysql2");

var connection = mysql.createConnection({
  host: "jsguru-rds-us-east-1a-main.c5u2qmocqh5e.us-east-1.rds.amazonaws.com",
  user: "jsguru",
  password: "12345678",
  port: 3306,
});

connection.connect(function (err) {
  if (err) {
    console.error("Database connection failed: " + err.stack);
    return;
  }

  console.log("Connected to database.");
});

connection.end();

@misostack
Copy link
Author

Create and connect with read replicate DB Instance

image
image
image

@misostack
Copy link
Author

How to create an encrypted Database Instances

image

You can't change encryption mode after database created, so the way to convert an uncrypted database into encrypted database in make a snapshot via ebs, s3, then create an encrypted version of them to create an encrypted database

@misostack
Copy link
Author

misostack commented Jan 8, 2024

RDS Proxy

image

Any DB instance that encounters "too many connections" errors is a good candidate for associating with a proxy. This is often characterized by a high value of the ConnectionAttempts CloudWatch metric. The proxy enables applications to open many client connections, while the proxy manages a smaller number of long-lived connections to the DB instance

image
image

@misostack
Copy link
Author

Load Balancing your RDS with Route53

Create private hosted zone

image

image

@misostack
Copy link
Author

Example network of EC2 Instances connect to AWS RDS in private network

image

@misostack
Copy link
Author

Create ALB for your EC Instances

image
image

@misostack
Copy link
Author

misostack commented Jan 8, 2024

image
image
image

Add new CNAME record into your hosted zone
image

@misostack
Copy link
Author

Monitor with CloudWatch

image

Install CloudWatchAgent

Create an IAM Role

image

Install CloudWatch

wget https://amazoncloudwatch-agent.s3.amazonaws.com/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
sudo dpkg -i -E ./amazon-cloudwatch-agent.deb

Attach IAM Role

image

Install aws cli and test

sudo apt install unzip 
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Create config file

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

Start

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json

Sample config

{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "cwagent"
  },
  "metrics": {
    "aggregation_dimensions": [["InstanceId"]],
    "append_dimensions": {
      "AutoScalingGroupName": "${aws:AutoScalingGroupName}",
      "ImageId": "${aws:ImageId}",
      "InstanceId": "${aws:InstanceId}",
      "InstanceType": "${aws:InstanceType}"
    },
    "metrics_collected": {
      "cpu": {
        "measurement": [
          "cpu_usage_idle",
          "cpu_usage_iowait",
          "cpu_usage_user",
          "cpu_usage_system"
        ],
        "metrics_collection_interval": 60,
        "resources": ["*"],
        "totalcpu": false
      },
      "disk": {
        "measurement": ["used_percent", "inodes_free"],
        "metrics_collection_interval": 60,
        "resources": ["*"]
      },
      "diskio": {
        "measurement": ["io_time"],
        "metrics_collection_interval": 60,
        "resources": ["*"]
      },
      "mem": {
        "measurement": ["mem_used_percent"],
        "metrics_collection_interval": 60
      },
      "statsd": {
        "metrics_aggregation_interval": 60,
        "metrics_collection_interval": 10,
        "service_address": ":8125"
      },
      "swap": {
        "measurement": ["swap_used_percent"],
        "metrics_collection_interval": 60
      }
    }
  }
}

Stress Test

sudo apt install stress
sudo apt install stress-ng

stress-ng --cpu 4 --io 2 --vm 1 --vm-bytes 250m --timeout 60s --metrics-brief

image

image

@misostack
Copy link
Author

CloudWatch Labs

Command to access AWS CLI

sudo apt update -y
sudo apt install unzip -y
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Command to do stress test

sudo apt update -y
sudo apt install stress-ng -y
stress-ng --cpu 4 --io 2 --vm 1 --vm-bytes 250m --timeout 60s --metrics-brief

CloudWatch Metrics

CloudWatch Standard Metrics

image

CloudWatch Custom Metrics

Create Policy, Create Role, and Attach IAM Role for EC2 Instance
image
image
image
image
image

Test

aws cloudwatch list-metrics

Test put-metric

[
  {
    "MetricName": "New Posts",
    "Timestamp": "Tuesday, January 08, 2024 8:28:20 AM",
    "Value": 0.50,
    "Unit": "Count"
  }
]
aws cloudwatch put-metric-data --namespace "JSGuruCustomMetric" --metric-data file://metric.js
aws cloudwatch put-metric-data --metric-name CloudWatchAlarmValue --namespace JSGuruCustomMetric --unit Percent--value 50 --dimensions InstanceID=$(curl -s -X GET -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id),InstanceType=$(curl -s -X GET -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-type)

image
image

Request EC2 API Token

TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
echo $TOKEN

Get EC2 meta-data and user-data

curl -X GET -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data
curl -X GET -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id

curl -X GET -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/user-data

Create Alarm

image

Notification

image

Trigger Lambda Function

image

Trigger AutoScale Group with Action

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment