Continuous Integration

CI.md

Continuous Integration

Security Groups

Using security groups you can allow/restrict access by adding protocol rules. In our project we use only TCP protocol rules.

Security groups:

• ec2-security-group (Inbound ports: 80, 22)
• AutoScaling-Security-Group (Inbound ports: 80, 22)
• mongodb-security-group (Inbound ports: 27017, 22)

Target Groups

Load balancer routes requests to the targets in a target group using the protocol and port that you specify, and performs health checks on the targets using the health check settings.

Target groups:

• myproject-dev
• myproject-prod

Health check settings:

Protocol: HTTP
Path: /test
Port: traffic port
Healthy threshold: 2
Unhealthy threshold: 2
Timeout: 3
Interval: 5
Success codes: 200

Load Balancers

Load balancer determines to which target group instances all requests should be routed.

Load balancers:

• myproject-dev-load-balancer (Security group: ec2-security-group, Target group: myproject-dev)
• myproject-prod-load-balancer (Security group: ec2-security-group, Target group: myproject-dev)

Launch Configurations

Launch configurations are used to specify how to configure EC2 instance at creation state. We use Ubuntu 14.04 template.

Launch configurations:

• myproject-dev-launch (Security group: AutoScaling-Security-Group)
• myproject-prod-launch (Security group: AutoScaling-Security-Group)

User data template:

#!/bin/bash
apt-add-repository ppa:brightbox/ruby-ng
apt-get update
apt-get -y install linux-image-extra-$(uname -r) linux-image-extra-virtual
apt-get -y install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
apt-get update
apt-get -y --force-yes install docker-ce
apt-get -y install python-pip
pip install docker-compose
apt-get -y install ruby2.0
apt-get -y install wget
wget https://aws-codedeploy-us-east-1.s3.amazonaws.com/latest/install
chmod +x ./install
./install auto
rm install
service codedeploy-agent start
echo NODE_ENV=YOUR_ENVIRONMENT >> /etc/environment

Auto Scaling Groups

Initially we manually created two auto scaling groups with respective launch configurations. Later at blue-green deployment they are replaced with new ones. At each deployment new auto scaling group is created. In order to stop an instance you just need to remove its Auto Scaling Group. Do not worry to lose your data when stopping development or production instances because all your data is stored in separate instance.

NOTE: If deployment fails, newly created auto scaling groups will not be removed automatically.

Launch configurations:

• CodeDeploy_development_d-XXXXXXXXX (Launch configuration: myproject-dev-launch, Target group: myproject-dev)
• CodeDeploy_production_d-XXXXXXXXX (Launch configuration: myproject-prod-launch, Target group: myproject-prod)

CodeDeploy

CodeDeploy allows you to deploy the code on EC2 instances. Instances are detected by auto scaling groups. For blue-green deployment you must specify load balancers. In this project we use blue-green deployment.

Deployment groups:

• development (Auto scaling group: CodeDeploy_development_d-XXXXXXXXX, Load balancer: myproject-dev-load-balancer)
• production (Auto scaling group: CodeDeploy_production_d-XXXXXXXXX, Load balancer: myproject-prod-load-balancer)

Application structure:

-> myproject-backend
 |
  -> appspec.yml
  -> codedeploy
   |
    -> after_install.sh
    -> application_start.sh
    -> validate_service.sh

appspec.yml:

version: 0.0
os: linux
files:
  - source: /
    destination: /home/ubuntu/myproject-backend
hooks:
  AfterInstall:
    - location: codedeploy/after_install.sh
      runas: root
  ApplicationStart:
    - location: codedeploy/application_start.sh
      runas: root
  ValidateService:
    - location: codedeploy/validate_service.sh
      runas: root

after_install.sh:

#!/bin/bash
cd /home/ubuntu/myproject-backend
docker-compose build

application_start.sh:

#!/bin/bash
cd /home/ubuntu/myproject-backend
docker-compose up -d

validate_service.sh:

#!/bin/bash
if ! lsof -i:80
then
    exit 1
else
    exit 0
fi

Blue-Green Deployment

With a blue/green deployment, you provision a new set of instances on which CodeDeploy installs the latest version of application. CodeDeploy then reroutes load balancer traffic from an existing set of instances running the previous version of your application to the new set of instances running the latest version. After traffic is rerouted to the new instances, the existing instances are terminated. If there is an issue with the newly deployed application version, CodeDeploy rolls back to the previous version.

CodePipeline

CodePipeline periodically checks for code changes on github repository. If there was any code change, it will copy whole code from github to S3 repository then will send that code as input to CodeDeploy.

Pipelines:

• myproject-dev-pipeline (dev branch)
• myproject-prod-pipeline (master branch)

CloudWatch

CloudWatch stores all logs from each environment. You can access them in Logs section. For each deployment new stream is created.

Log groups:

• development
• production