Skip to content

Instantly share code, notes, and snippets.

@MdSahil-oss

MdSahil-oss/deploy_scalable_application_aws-cloud.md

Last active June 6, 2024 19:02
Show Gist options
  • Save MdSahil-oss/5682a587735919df287511f20c2e420b to your computer and use it in GitHub Desktop.
Save MdSahil-oss/5682a587735919df287511f20c2e420b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
deploy_scalable_application_aws-cloud.md

Deploy scalable application on AWS

Image

Technologies (I used):

  • AWS
  • Packer
  • Consul
  • Terraform
  • Ansible
  • Bash script

Link to the project

Description (Workflow):

Created Resources:

  • In this project, Firstly I built an AWS AMI (Amazon machine image) with followings using Packer & Ansible:
    • A java based web application source code.
    • AWS CloudWatch Agent.
    • Consul.
  • And created other following resources in my AWS environment using Terraform:
    • A Load-Balancer with running consul as service registry.
    • A EC2 Launch Template.
    • An AutoScaling group.
    • A RDS (Database Instance).
    • And A SecretManager.

Working:

  • AMI that I built is utilized by the created Auto-scaling group via Launch Template I created to create EC2 instances dynamically.
  • Every time on instance creation by Autoscaling group using Launch Template, A script is executed on each instance to run/start followings:
    • Web Application.
    • CloudWatch Agent.
    • And consul as a service that register itself (that instance) with service-registry (load-balancer).
  • And web application running on instances fetches credentials from Secret Manager using AWS API to access data from the RDS (Database Instance).
  • And the logs generated by each instance is sent to CloudWatch via CloudWatch agent on the instance.

After All, On accessing application using LoadBalancer's public IP, Application is totally accessible, and we can also check the generated logs by all the instances from CloudWatch Agent.

On increasing and decreasing overall load on instances AutoScaling group changes count of service instances as per its configuration.

min_size         = 2
max_size         = 4
desired_capacity = 2
instance_refresh {
  strategy = "Rolling"
  preferences {
    min_healthy_percentage = 50
  }
}

How did I build this project ?

  • Firstly I built an AWS AMI (Amazon machine image) using Packer & Ansible:

    # aws_ubuntu.pkr.hcl
packer {
  required_plugins {
    amazon = {
      version = ">= 0.0.2"
      source  = "github.com/hashicorp/amazon"
    }
    ansible = {
      source  = "github.com/hashicorp/ansible"
      version = "~> 1"
    }
  }
}

source "amazon-ebs" "ubuntu" {
  ami_name      = "petclinic_on_ubuntu_20.04"
  instance_type = "t2.micro"
  region        = "us-east-1"
  source_ami    = "ami-0261755bbcb8c4a84"
  ssh_username  = "ubuntu"
}

build {
  name = "ami-maker"
  sources = [
    "source.amazon-ebs.ubuntu"
  ]

  provisioner "ansible" {
    playbook_file = "./playbook.yaml"
  }
}
    # playbook.yaml:
---
- name: Installs required tools
  hosts: default
  tasks:
    - name: Updating apt package manager
      ansible.builtin.shell:
      args:
        chdir: /home/ubuntu/
        cmd: sudo apt update
    - name: Installing JDK & JRE
      ansible.builtin.shell:
      args:
        chdir: /home/ubuntu/
        cmd: sudo apt install -y openjdk-17-jdk openjdk-17-jre
    - name: Installing git
      ansible.builtin.shell:
      args:
        chdir: /home/ubuntu/
        cmd: sudo apt install -y git
    - name: Fetching spring-petclinic source code
      ansible.builtin.shell:
      args:
        chdir: /home/ubuntu/
        cmd: git clone https://github.com/spring-projects/spring-petclinic.git
    - name: Building spring-petclinic source code
      ansible.builtin.shell:
      args:
        chdir: /home/ubuntu/spring-petclinic/
        cmd: ./mvnw package
    - name: Installing cloudwatch agent
      ansible.builtin.shell:
      args:
        chdir: /home/ubuntu/
        cmd: wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb && sudo dpkg -i amazon-cloudwatch-agent.deb
    - name: Installing consul
      ansible.builtin.shell:
      args:
        chdir: /home/ubuntu/
        cmd: curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add - && sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main" && sudo apt-get update && sudo apt-get install -y consul

  • And created other following resources in my AWS environment using Terraform:

    • A Load-Balancer with running consul as service registry. 
      resource "aws_instance" "load_balancer" {
  depends_on    = [
    aws_iam_instance_profile.cloud_watch_agent_server_profile,
    data.aws_key_pair.MdSahil-oss
  ]
  ami           = "ami-06aa3f7caf3a30282"
  instance_type = "t2.micro"
  security_groups = [
    data.aws_security_group.ssh.name,
    data.aws_security_group.http.name,
    data.aws_security_group.https.name,
    data.aws_security_group.consul-ports.name,
  ]
  tags = {
    Name = "load-balancer"
  }
  key_name = data.aws_key_pair.MdSahil-oss.key_name
  connection {
    host        = self.public_ip
    type        = "ssh"
    user        = "ubuntu"
    private_key = "${file("/home/sahil/aws/MdSahil-oss.pem")}"
  }
  provisioner "file" {
    source      = "./scripts/load_balancer.config.sh"
    destination = "/tmp/script.sh"
  }
  # provisioner "file" {
  #   source      = "$HOME/aws/MdSahil-oss.pem"
  #   destination = "$HOME/MdSahil-oss.pem"
  # }
  provisioner "remote-exec" {
    inline = [
      "chmod +x /tmp/script.sh",
      "/tmp/script.sh ${self.private_ip}",
    ]
  }
}
      #!/bin/bash
# load_balancer.config.sh
sudo apt-get update -y

# Configuring service Registry on the server
echo "Configuring Service-Registry on the server"
# wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
# echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt install consul -y

cat <<EOF > consul.hcl
"bind_addr" = "$1"
"client_addr" = "$1"
"data_dir" = "/var/consul"
"encrypt" = "ZENZNrsXU336Uma+S4XUj9sxvICj32N7XdEzrbYbRpY="
"datacenter" = "dc1"
"ui" = true
"server" = true
"log_level" = "INFO"
EOF
sudo rm -f /etc/consul.d/consul.hcl
sudo mv $HOME/consul.hcl /etc/consul.d/

sudo consul agent -dev -config-file=/etc/consul.d/consul.hcl 2>&1 >> consul-agent.log &
echo "Done! Configuring Service-Registry on the server"

# Configuring load-balancer on the server
sudo apt-get update -y
echo "Configuring load-balancer on the server"
sudo apt-get install unzip -y
sudo apt install nginx -y
sudo curl -L  https://releases.hashicorp.com/consul-template/0.30.0/consul-template_0.30.0_linux_amd64.zip -o /opt/consul-template.zip
sudo unzip /opt/consul-template.zip -d  /usr/local/bin/
sudo cat <<EOF > load-balancer.conf.ctmpl
upstream backend {
 {{- range service "backend" }} 
  server {{ .Address }}:{{ .Port }}; 
 {{- end }} 
}

server {
   listen 80;

   location / {
      proxy_pass http://backend;
   }
}
EOF
sudo mv $HOME/load-balancer.conf.ctmpl /etc/nginx/conf.d/


sudo cat <<EOF > consul-template.hcl
consul {
 address = "$1:8500"

 retry {
   enabled  = true
   attempts = 12
   backoff  = "250ms"
 }
}
template {
 source      = "/etc/nginx/conf.d/load-balancer.conf.ctmpl"
 destination = "/etc/nginx/conf.d/load-balancer.conf"
 perms       = 0600
 command = "service nginx reload"
}
EOF
sudo mv $HOME/consul-template.hcl /etc/nginx/conf.d/

sudo rm /etc/nginx/sites-enabled/default

sudo systemctl restart nginx

# Immediate below line should be executed by consul services into load-balancer
# to refresh load-balancer's configuration to add new services as backends and crate new /etc/nginx/conf.d/load-balancer.conf.
sudo consul-template -config=/etc/nginx/conf.d/consul-template.hcl 2>&1 >> consul-template.log &

echo "Done! Configuring load-balancer on the server"
    • A EC2 Launch Template. 
      resource "aws_launch_template" "project_7_launch_template" {
  depends_on = [
    aws_instance.load_balancer,
    aws_iam_instance_profile.cloud_watch_agent_server_profile,
    # aws_db_instance.default,
    data.aws_security_group.ssh,
    data.aws_security_group.http,
    data.aws_security_group.https,
    data.aws_security_group.consul-ports
  ]
  name          = "petclinic_on_ubuntu_20.04_launch_template"
  image_id      = "ami-0480430c9bf96dd8f"
  instance_type = "t2.micro"
  key_name      = "MdSahil-oss"
  security_group_names = [
    data.aws_security_group.ssh.name,
    data.aws_security_group.http.name,
    data.aws_security_group.http.name,
    data.aws_security_group.consul-ports.name
  ]
  user_data = base64encode(templatefile("${path.module}/scripts/launch_template.config.sh",
    {
      CONSUL_SERVER_PRIVATE_IP = aws_instance.load_balancer.private_ip
    }
  ))
  iam_instance_profile {
    name = aws_iam_instance_profile.cloud_watch_agent_server_profile.name
  }
}
      #!/bin/bash
sudo apt update -y
sudo apt install nginx -y
export INSTANCE_PRIVATE_IP=$(curl http://169.254.169.254/latest/meta-data/local-ipv4)

echo "Started running initial script in $INSTANCE_PRIVATE_IP"

## Starts cloudwatch agent on the instance.
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s

## Starting `spring-petclinic` application on 8080
$HOME/spring-petclinic/mvnw package
java -jar $HOME/spring-petclinic/target/*.jar 2>&1 >> $HOME/spring-petclinic.log &

sudo cat <<EOF > default
server {
    listen 80;
    server_name _;

    location / {
        # proxy_set_header   X-Forwarded-For $remote_addr;
        # proxy_set_header   Host $http_host;
        proxy_pass         "http://127.0.0.1:8080";
    }
}
EOF
sudo mv $HOME/default /etc/nginx/sites-available
sudo systemctl stop nginx
sudo systemctl start nginx

sudo apt install consul -y

sudo cat <<EOF > consul.hcl
"bind_addr" = "$INSTANCE_PRIVATE_IP"
"server" = false
"datacenter" = "dc1"
"data_dir" = "/var/consul"
"encrypt" = "ZENZNrsXU336Uma+S4XUj9sxvICj32N7XdEzrbYbRpY="
"log_level" = "INFO"
"enable_script_checks" = true
"enable_syslog" = true
"leave_on_terminate" = true
"start_join" = ["${CONSUL_SERVER_PRIVATE_IP}"]
"node_name" = "backend-$INSTANCE_PRIVATE_IP"
EOF
sudo rm -f /etc/consul.d/consul.hcl
sudo mv $HOME/consul.hcl /etc/consul.d/

sudo cat <<EOF > backend.hcl
"service" = {
  "Name" = "backend"
  "Port" = 80
  "check" = {
    "args" = ["curl", "localhost"]
    "interval" = "3s"
  }
}
EOF
sudo mv $HOME/backend.hcl /etc/consul.d/backend.hcl

sudo consul agent -config-dir /etc/consul.d/ 2>&1 >> $HOME/consul.log &
echo "Done! running initial script in $INSTANCE_PRIVATE_IP"

echo "Connecting to load-balancer"

sudo cat <<EOF > mdsahiloss.txt
-----BEGIN RSA PRIVATE KEY-----
XYZ
-----END RSA PRIVATE KEY-----
EOF
sudo chmod 400 mdsahiloss.txt

ssh -i $HOME/MdSahil-oss.pem ubuntu@$CONSUL_SERVER_PRIVATE_IP sudo consul-template -config=/etc/nginx/conf.d/consul-template.hcl 2>&1 >> consul-template.log &

# Refreshing Nginx on loadbalancer.
# sudo consul-template -config=/etc/nginx/conf.d/consul-template.hcl 2>&1 >> consul-template.log &
# exit

echo "Done! executing script on load-balancer"
    • An AutoScaling group. 
      resource "aws_autoscaling_group" "project_7_asg" {
  depends_on       = [aws_launch_template.project_7_launch_template]
  name             = "petclinic_on_ubuntu_autoscaling_group"
  min_size         = 2
  max_size         = 2
  desired_capacity = 2
  tag {
    key                 = "Name"
    value               = "petclinic_on_ubuntu_autoscaling_group"
    propagate_at_launch = true
  }
  launch_template {
    id = aws_launch_template.project_7_launch_template.id
  }
  instance_refresh {
    strategy = "Rolling"
    preferences {
      min_healthy_percentage = 50
    }
    # triggers = ["tag"]
  }
  availability_zones = var.VPC_availablity_zones
}
    • A RDS (Database Instance). 
      resource "aws_db_instance" "default" {
  allocated_storage    = 10
  db_name              = "mydb"
  engine               = "mysql"
  engine_version       = "5.7"
  instance_class       = "db.t3.micro"
  username             = var.mysql_db_username
  password             = var.mysql_db_password
  parameter_group_name = "default.mysql5.7"
  skip_final_snapshot  = true
}

After execution of all of these configuration files project works as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment