Skip to content

Instantly share code, notes, and snippets.

@laser

laser/aws-prez.md

Last active January 2, 2018 22:38
Show Gist options
  • Save laser/2ef3f5c06261a5cc887076c5fe717d34 to your computer and use it in GitHub Desktop.
Save laser/2ef3f5c06261a5cc887076c5fe717d34 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
aws-prez.md

What’s This Presentation About?

What This Presentation Is Not

  • a deep dive into Docker
  • a comparison of Docker with other container services
  • a comparison of AWS with other PaaS and IaaS offerings

Assumptions

  • basic familiarity with AWS
  • basic familiarity with Docker

By the End of This Presentation, You'll Have:

--- diagram goes here ---

  • a tiered VPC with public and private subnets in the us-east-1 region
  • a highly available, Fargate-backed ECS cluster deployed across two AZs
  • 4 instances of a Rails 5 API fronted by an ALB
  • an RDS-managed Postgres database
  • a GitHub-integrated CI/CD server for zero-downtime deploys to ECS cluster
  • an ECR repository for our application's Docker images
  • centralized logging, with CloudWatch

Questions?

  • great, hold them until the end of the prez

What's Docker?

  • a lightweight container virtualization technology that does not require a hypervisor
  • programs are executed in a container isolated from other processes
  • helps us to package the application and its dependencies together

What's Docker Compose?

  • a tool for defining and running multi-container Docker applications
  • you use a Compose file to configure your application's services (app, postgres, redis, etc.)
  • using a single command, you create and start all the services
  • helps us to simulate our AWS environment locally and in CI/CD environment

What's AWS?

  • collection of IaaS and PaaS offerings from Amazon
  • some services we'll use in this presentation include:
    • CloudFormation
    • RDS (Relational Database Service)
    • ECR (Elastic Container Registry)
    • ECS (Elastic Container Service)
    • CodePipeline
    • ALB (Application Load Balancer)
    • VPC (Virtual Private Cloud)

Let's Go!

Create Rails 5 API

  • clone repo
  • create GitHub personal acccess token, for CodePipeline
  • rails new an application
  • rails generate scaffolding for a Post model + routes
  • create application secret
  • replace database.yml

Create Dockerfile

  • use the Ruby 2.5.0 Alpine image as our base
  • install system dependencies w/APK
  • copy Gemfile and bundle install before copying app sources, for caching
  • COPY application sources into image
  • define a very flexible entrypoint

Run the Docker Image

  • docker build -t mysweetapp .
  • docker run mysweetapp "rails db:migrate && rails server -b 0.0.0.0"

Some Problems

  • we need a database
  • we don't want to rebuild the image every time we make a code change

Create a Docker Compose File

  • define a Postgres service, built from the offical Postgres image
  • web service:
    • gets a shared volume, which overrides the COPY instruction
    • gets some environment variables which tell it how to connect to DB
    • runs Ruby script to ensure DB is up before starting Rails
    • exposes web app port to the host OS

Running the Tests (One-Off)

docker-compose -f ./websvc/docker-compose.yml run --rm websvc "\
  ruby ./wait-for-postgres.rb \
    && rails db:migrate \
    && rails db:test:prepare \
    && rails test"

Running the Tests (Guard)

Add Guard-related dependencies to Gemfile:

group :development do
  gem 'guard'
  gem 'guard-minitest'
end

Then, install dependencies and update the Gemfile.lock:

docker-compose -f ./websvc/docker-compose.yml run --rm websvc "\
  bundle install \
    && guard init minitest \
    && cp /opt/bundle/Gemfile.lock /opt/app/"

Commit changes. Then, run guard:

docker-compose -f ./websvc/docker-compose.yml run --rm websvc "\
  ruby ./wait-for-postgres.rb \
    && rails db:migrate \
    && rails db:test:prepare \
    && guard"

Bringing the Application Online

docker-compose -f ./websvc/docker-compose.yml up

Taking the App Down

The following command stops the running Docker containers and deletes anonymous volumes attached to containers:

docker-compose -f ./websvc/docker-compose.yml up --volumes

To the Cloud!

What's CloudFormation?

  • a Amazon tool which provided a language for describing and provisioning AWS infrastructure
  • artifacts under version control
  • declarative

CloudFormation Basics: Nouns

  • stack (collection of resources)
  • resource (VPC, ECS cluster, ALB, IAM user)
  • parameter (inputs to template)
  • output (outputs from stack)

CloudFormation Basics: Verbs

  • deploy --no-execute-changeset (declare the final state that we desire)
  • describe-change-set (see what needs to happen in order for that to become a reality)
  • deploy (do it)

CloudFormation Example Template

# s3stuff.yml

Parameters:
  BucketName:
    Type: String

Resources:
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: PublicRead
      BucketName: !Ref BucketName

Outputs:
  BucketWebsiteURL:
    Description: Returns the Amazon S3 website endpoint for the specified bucket.
    Value: !GetAtt S3Bucket.WebsiteURL

Deploying a Stack

$ aws cloudformation deploy \
  --stack-name foobarbazblix \
  --template-file ~/Desktop/s3stuff.yaml \
  --parameter-overrides BucketName=foobarbazblix
  

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - foobarbazblix

Getting Stack Output

$ aws cloudformation describe-stacks \
  --stack-name foobarblix
  
{
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:166875342547:stack/foobarbazblix/37c71cd0-f008-11e7-9c6b-5044763dbb7b",
            "LastUpdatedTime": "2018-01-02T22:04:27.801Z",
            "Parameters": [
                {
                    "ParameterValue": "foobletronica",
                    "ParameterKey": "BucketName"
                }
            ],
            "Tags": [],
            "Outputs": [
                {
                    "Description": "Returns the Amazon S3 website endpoint for the specified bucket.",
                    "OutputKey": "BucketWebsiteURL",
                    "OutputValue": "http://foobletronica.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "EnableTerminationProtection": false,
            "CreationTime": "2018-01-02T22:04:20.457Z",
            "StackName": "foobarbazblix",
            "NotificationARNs": [],
            "StackStatus": "CREATE_COMPLETE",
            "DisableRollback": false,
            "ChangeSetId": "arn:aws:cloudformation:us-east-1:166875342547:changeSet/awscli-cloudformation-package-deploy-1514930659/9657151d-0e00-40f0-a8b2-1abcc9d8d307",
            "RollbackConfiguration": {}
        }
    ]
}

$ aws cloudformation describe-stacks \
  --stack-name foobarbazblix \
  --query 'Stacks[0].Outputs[0].OutputValue' \
  | tr -d "\""
  
http://foobletronica.s3-website-us-east-1.amazonaws.com

Our CloudFormation Template

Six parts, several-hundred lines:

  • networking and security
  • ECR
  • RDS
  • ECS and ALB
  • CI/CD pipeline

CloudFormation: Networking and Security

  • public and private subnets
  • virtualized network equipment (NAT gateway, IGW) which permits stuff in VPC to talk to Internet (and vice-versa)
  • security groups to control ingress to various parts of system

CloudFormation: ECR

  • holds our Docker images

CloudFormation: RDS

  • hosts Postgres for us

CloudFormation: ECS and ALB

  • cluster
  • service
  • task definition
  • load balancer

CloudFormation: ECS Cluster

  • a collection of services with a name

CloudFormation: ECS Service

  • number of desired applications behind load balancer
  • deployment strategy (blue/green deploy? outage?)
  • ingress rules / security groups
  • registration with load balancer

CloudFormation: ECS Task Definition

  • resources (CPU units, memory) required for each app
  • Docker command to run the app
  • container port-mappings
  • logging configuration

CloudFormation: Load Balancer

  • healthcheck paths and ports
  • forwarding rules

CloudFormation: CloudWatch Log Group

  • applications log to stdout
  • logs are sent to single CloudWatch group
  • log streams are linked to the origin container

Finally, CodePipeline

  • polls GitHub using an OAuth token
  • pulls changes when detected
  • builds application using Docker Compose
  • runs tests
  • pushes image to ECR
  • updates ECS service definition to use new image

A Note on Deploys

  • blue/green or outage depends on "minimum healthy percentage" and "maximum healthy percentage" and "desired count"-properties of ECS service
  • blue/green ("zero-downtime") deploys require careful processes to ensure backwards compatibility!
  • Rails has some tools which aim to prevent multiple apps from simultaneously running migrations

In Summary

  • not particularly difficult to package a Rails application as a Docker container
  • use Docker Compose locally and in CI to simulate integrated environment
  • deploy everything to a sophisticated AWS environment via version-controlled artifacts

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment