Skip to content

Instantly share code, notes, and snippets.

@iSarCasm

iSarCasm/AWS Developer Associate Certification notes.md

Last active February 28, 2022 10:30
Show Gist options
  • Save iSarCasm/7f8870acc53e34debf9c2d08b245c62a to your computer and use it in GitHub Desktop.
Save iSarCasm/7f8870acc53e34debf9c2d08b245c62a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
AWS Developer Associate Certification notes.md

AWS Developer Associate Certification notes

IAM

Concepts

  • Users
    • End users and people
  • Groups
    • A collections of users or roles
  • Roles
    • Users, Application, or Services
    • Web Identity Federation
      • Authentication with 3rd party (Facebook, Google, etc.)
      • Assume rome with STS API call
      • Temporary credentials
    • Cross-Account Acesss
  • Policy
    • AWS Managed Policies
    • Customer Managed Policies
    • Inline Policies
Example Policy 
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "FirstStatement",
      "Effect": "Allow",
      "Action": ["iam:ChangePassword"],
      "Resource": "*"
    },
    {
      "Sid": "SecondStatement",
      "Effect": "Allow",
      "Action": "s3:ListAllMyBuckets",
      "Resource": "*"
    },
    {
      "Sid": "ThirdStatement",
      "Effect": "Allow",
      "Action": [
        "s3:List*",
        "s3:Get*"
      ],
      "Resource": [
        "arn:aws:s3:::confidential-data",
        "arn:aws:s3:::confidential-data/*"
      ],
      "Condition": {"Bool": {"aws:MultiFactorAuthPresent": "true"}}
    }
  ]
}

Features

  • Shared access to your AWS account
  • Granular permissions
  • Identity federation
  • PCI DSS Compliance
  • Integrated with many AWS services
  • MFA (Multi-Factor Authentication)
  • Password Rotation

Service Details

  • Scope: Universal
    • IAM Users, Groups and Roles are available in all AWS regions
  • Pricing: Free

Cognito

  • Cognito acts as an Identity Broker
  • AWS Best Practice for Web ID Federation
  • Uses mobile Push Notification (SNS) to sync data across devices
  • Manages User Pools
    • User directories for Sign-in and Sign-up
  • Manages Identity Pools
    • Temporary credentials to AWS services

Service Details

  • Scope: User Pools are per-Region
  • Pricing:
    • First 50k MAU - free
    • Over 50k MAU - from $0.00550 to $0.00250

EC2

EC2 101

  • Elastic Compute Cloud

  • Pay as you use

  • Failure resistant

  • Types:

    • On Demand - pay fixed rate by hour (more recently to by the second - linux). Good for unpredictable workloads.
    • Reserved - capacity reservation. pay upfront for discounts. Predicable workloads
    • Spot - Bid for instance capacity (based on supply/demand). Set bid price (by hour/second). Good for applications with flexible start and end times.
    • Dedicated hosts - Physical EC2 servers - often used for licensing cost optimization. Not multi-tenant. Good for regulatory requirements.

  • Instance Types:

    • General Purpose
      • Mac - Mac Instances
      • T - Burtsable General Purpose Instances
      • M - General Purpose Intel Xeon Instances
      • A - ARM Instance with cost savings
    • Compute Optimized
      • C
      • Hpc6a
    • Memory Optimized
      • R
      • X
      • High Memory
      • z1d
    • Storage Optimized
      • H
      • D
      • I
    • Accelerated Computing
      • VT
      • F
      • G
      • Inf
      • Trn
      • DL
      • P

Read EC2 FAQ

  • Think of an Elastic IP address as a STATIC public ipv4 address
  • limited to 5 Elastic IP addresses per region
  • you can configure the reverse DNS record of your Elastic IP address
  • One Availability Zone name (for example, us-east-1a) in two AWS customer accounts may relate to different physical Availability Zones.
  • You can retrieve metrics data for any Amazon EC2 instance up to 2 weeks from the time you started to monitor it. After 2 weeks, metrics data for an Amazon EC2 instance will not be available if monitoring was disabled for that Amazon EC2 instance. If you want to archive metrics beyond 2 weeks you can do so by calling mon-get-stats command from the command line and storing the results in Amazon S3 or Amazon SimpleDB.
  • Q. Can I use a Spot instance with a paid AMI for third-party software (such as IBM’s software packages)? Not at this time.
  • Q. What is the difference between Stop and Hibernate interruption behaviors? In the case of Hibernate, your instance gets hibernated and the RAM data persisted. In the case of Stop, your instance gets shutdown and RAM is cleared.
  • Can I use my existing Windows Server license with EC2. Yes you can. After you’ve imported your own Windows Server machine images using the ImportImage tool, you can launch instances from these machine images on EC2 Dedicated Hosts
  • NTP endpoint at a link-local IP address (169.254.169.123) accessible from any instance running in a VPC

EBS (Elastic Block Store)-

  • EBS (Elastic Block Storage)
  • Storage of OS, Databases and Applications
  • Automatically replicated WITHIN an AZ
  • Types:
    • General Purpose SSD (GP3/GP2). Balance price/performances. Ratio of 3 IOPS per GB, up to 10000 IOPS, can burst to 3000 IOPS for volumes >= 3334GB
    • Provisioned IOPS SSD (IO2/IO1). Use if > 10000 IOPS is required, can go up to 20000 IOPS
    • IO2 Block Express. Up to 64 TB and 256k IOPS per volume. SAN in the cloud performance.
    • Throughput Optimized HDD (ST1). Log processing, Data warehousing. Good for sequenced data. CANNOT be boot volumes. About 500 MBs
    • Cold HDD (SC1). Low cost - for infrequently access workloads. CANNOT be a boot volume. About 250 MBs
    • Magnetic (Standard). Lowest cost per gigabyte. IS bootable.
  • You CAN NOT mount 1 EBS volume to multiple ECS 2 instances - instead use EFS
  • Termination Protection is turned OFF by Default
  • On EBS backed instance, the default for root volume will be deleted when the instance is terminated
  • Using the default AMI Root volumes on EC2 - they are NOT encrypted. Or create a copy of the AMI, then encrypt the root device volume. Additional volumes (non-root) can be encrypted.
  • If you take a snapshot of a EBS volume in a RAID array the cached data is not included - You need to take an application consistent snapshot.

EBS Volumes & Snapshots - Upgrading/Moving-

  • EC2 and it's EBS volume MUST BE in the same AZ

  • Can modify volumes for everything except Standard Magnetic

  • To Migrate EBS volume from one AZ to another, must snapshot first then create volume off the snapshot

  • To Migrate EBS volume from one Region to another, must snapshot then copy to the new region

  • Image from snapshot makes it bootable

  • Volumes exist on EBS (Virtual Hard Disk)

  • Snapshots exist on S3

  • Snapshots are point in time copies of the Volumes

  • Snapshots are incremental (only changed blocks moved to S3)

  • It's recommended that then you snapshot a root EBS volume, SHOULD stop the instance before taking snapshot

  • However you can take a snap while the instance is running (memory cached data will be lost)

  • You can create AMI (Amazon Machine Images)'s from both Volumes and Snapshots

  • You CAN change EBS volumes sizes on the fly, including change the size and storage type

  • To move an ECS volume from one AZ/Region to another, take a snap or an image of it, then copy it to the new AZ/Region

  • Security

    • Snapshots of encrypted volumes, the snapshots are encrypted automatically
    • Volumes restored from encrypted snapshots are encrypted automatically
    • You can share snapshots, but only if un-encrypted
      • Can be shared with other AWS accounts (or made public)

AMIs Types - EBS Root Device Volumes vs Instance Store

  • AMI's are regional!

  • You can select your AMI based on:

    • Region
    • OS
    • Architecture (32/64bit)
    • Launch Permissions
    • Storage for the Root Device (Root Device Volume)
      • Instance Store (Ephemeral Storage)
      • EBS Backed Volumes

  • All AMIs are categorized as either backed up EBS or backed by Instance store

  • For EBS volumes: the root device for an instance launched from the AMI is an EBS volume created from an EBS snapshot

  • For Instance Store Volumes: the root device for an instance launched from the AMI is an instance store volume created from a template stored in S3

  • Instance Store Volumes are sometimes called Ephemeral Storage

  • Instance Store volumes CANNOT be stopped If the underlying host fails, you will lose your data

  • EBS instances can be stopped, and Data will not be lost

  • By default, both ROOT volumes will be deleted on termination, however with EBS volumes, you can tell AWS to keep the root device volume

EBS Pricing

  • Pay for Storage, IOPS and Throughput

Elastic Load Balancer

  • Application Load Balancer
    • Intelligent balancing for HTTP/HTTPS traffic
    • X-Forwarded-For HTTP Header
  • Network Load Balancer
    • High-performance TCP traffic
  • Classic Load Balancer
    • Legacy
  • Gateway Load Balancer
    • 3rd party AWS virtual appliances

Pricing: Pay for hours and LCU (Load balancer capacity units)

Route53

  • DNS service
  • Maps a domain name to EC2, ELB or S3
  • Concepts:
    • Hosted Zone - A container for DNS records for your domain
    • Alias - ilovecloud.com
    • A Record - Allows to route traffic to a resource using IPv4 address.

RDS

  • RDS Database Types:
    • SQLServer, Oracle, MySQL, PostgreSQL, MariaDB, Amazon Aurora
  • RDS is for OLTP (Online Transactions Processing) Workloads
  • Not suitable for OLAP (Use other tools like RedShift)

Pricing: Same as EC2

Backups vs Snapshots

  • Backups
    • Automated, enabled by default
    • Snapshots + transaction logs
    • Retention period of up to 35 days
  • Snapshots
    • User-initiated, ad-hoc
    • Snapshot only
    • Stored indefinetely (s3)

Encryption

  • Enable at Creation
  • Integrated with KMS
  • To add lated: Snapshot -> Encrypt -> RDS from Snapshot

Multi-AZ vs Read Replica

  • Multi-AZ
    • Exact copy of production database
    • Disaster Recovery
  • Read Replica
    • Same AZ, cross-AZ, cross-region
    • scale read performance

ElastiCache

  • Memcached (mem-cache-d)
    • In-memory, key-value data store
    • No persistance or Multi-AZ
  • Redis
    • Supports data sorting and ranking
    • Advanced data types
    • Supports Persistance and Multi-AZ

Pricing: per-hour

Parameter Store

  • Store Confidential Information

    • Passwords, database connection string, license codes, etc
    • Plain Text or Encrypted
    • Referenced in bootstrap scipts
    • Integrated with AWS services

  • Pricing

    • Standart - free;
    • Advanced - per-parameter or per-API call

S3

API Gateway

  • Front door to applications running in AWS
  • Serverless
  • Throttling (prvents applications from overload)

Lambda

Intro

  • Extremely cost-effective
  • Scales out automatically
  • Event-driven
  • Independant - each event will trigger a single function. Lambd function can trigger other functions
  • API Gatewat, DynamoDB, SQS, SNS, etc.

Lambda Versions

  • $LATEST - is latest version of the lambda function
  • Custom version aliases are available (e.g. "Prod"). Must be manually updated

Limits

  • Default limit - 1,000 concurrent executions per second (over = 429 http code).
  • Limit can be changed by contacting AWS support
  • You can set reserved concurrency for a function

Lambda and VPCs

  • You can enable Lambda to access resources in a private VPC
    • Provide a VPC Config (Private subnet ID, and security group ID)
  • Lambda configures an ENI, using an IP from the private subnet CIDR range
  • The security group then allows your function to access resources in VPC

Step Functions

  • Workflow Types
    • Standard Workflows
      • Long-running (up-to 1 year)
      • At-most-once
      • Non-Idempotent
    • Express Workflows
      • Short-lived (up to 5 minutes)
      • At-least-once
      • Idempotent
    • Synchronous Express Workflows
      • The workflow must complete before the next step begins
      • e.g., a customer must provide an addres before order can be complete
    • Asynchronous Express Workflows
      • Tasks are not dependant on the completion of the workflow
      • e.g., a mesaging system

DynamoDB

Intro

  • Fully managed NoSQL database

  • Single digit millisecond latency

  • Document and Key-Value data models

  • Stored on SSD storage

  • Spread across 3 geographically distinct data centres

  • Eventual Consistent Reads (Default)

    • Consistency is usaully reached within a second (Best Read Performance - app must wait up to 1 second)

  • Strongly Consistent Reads (SLOWER read performance)

    • Returns a result that reflects all writes that received a sucessful response response prior to the read. (app cannot wait up to 1 second)

  • Basics:

    • Tables
    • Items (This a row of data in table)
    • Attributes (Think of a column of data in a table)

  • Pricing:

    • Provisioned throughput capacity
      • Write Throughput $0.0065 per hour for every 10 units
      • Read Throughput $0.0065 per hours for every 50 units
    • On-demand capacity
      • Good for unpredictable traffic
      • Per-per-use model
    • First 25 GB stored per month is FREE, then $0.25 per GB (per month) there after.

Primary Keys

  • 2 Types of Primary Keys available

      • No 2 items in a table can have the same primary key value!
    • Single Attribute (think unique ID)
      • Partition Key (aka Hash key) composed of one attribute
    • Composite (think unique ID and a date range)
      • Partition Key & Sort Key (aka Range key) composed of two attributes

  • Partition Key

    • DynamoDB uses the partition keys value as input to an internal hash function. The output from the hash function determine the partition (this is simply the physical location in which the data is stored)

  • Partition Key and Sort Key

    • DynamoDB uses the the partition keys value as input into an internal hash function. The output from the has function determines the partition (this is simply th ephysical location in which the data is stored)
    • However!, 2 items can have the same partition key, but they MUST have a different sort key
    • All items with the same partition key are stored together, in sorted order by sort key value

  • Indexes

  • DynamoDB Streams (Store for a max of 24 hrs)

    • used to capture any kind of modification of the DynamoDB tables
      • If a new items is added (captures entire item and it's attributes)
      • If an item is update (captures the before and after)
      • If a new item is deleted (captures it prior to deletion)
    • can trigger a lambda function (e.g. replicate table in another region)

Scan vs Query-

  • Query

    • Find items in a table using only primary key attribute values. you must provide a partitiion attribute name and a distinct valye to search for

    • Can optionally provide a sort key attrivute name and value, and use a comparison operator to refine the search results

    • By default, a Query returns all the data attributes for items with the specified primary key(s);

    • However, you can use the ProjectionExpression parameter so that the query only retuns some of the attributes, rather than all

    • Results will always be sorted by the sort key (if you have one). if the data type of the sort key is a number, the results are returned in numeric order;

    • otherwise, the results are returned in order of ASCCI char code values.

    • By default, the sort order is ASCENDING, to reverse order set the ScanIndexForward paramater to false

    • By default, queries are eventually consistent but chan be changed to be strongly consistent.

    • Query is MORE efficient than Scan

  • Scan

    • Dumps every item in the table
    • By default, a Scan returns all the data attributes
    • However, you can use the ProjectionExpression parameter so that the query only retuns some of the attributes, rather than all

  • For quicker response times, design your tables in a way that can use the:

    • Query, Get or BatchGetItem APIs
    • OR - design your application to use Scan in a way that minimizes the impact on your table's request rate.

  • Optimizing Scans

    • Use smaller page size
    • Segregating mission-critical and scan traffic
    • Using parallel scans
    • Redesign table

  • Exam tips

    • Query find items in a table using only primary key attribute values. You must provide a partition key attribute name and a distinct valye to search for.
    • Scan examines EVERY item in the table. By default, a Scan returns all of the data attributes for every item; however you can use the ProjectExpression parameter so that the Scan only returns some of the attributes, rather than all.
    • Query results are always sorted by the sort key in ascending order. Set ScanIndexForward parameter to false to reverse it
    • Try use Query operation rather than Scan, as query is more efficient

DAX (DynamoDB Accelerator)

  • In-Memory write-through cache
  • Improves response times for eventually consistent reads only.
  • Write operation are also written directly to cache
  • Point API calls to DAX cluster instead of your table
  • Cache Hit
  • Not Suitable for write-intensive applications

DynamoDB TTL

  • Time To Live
    • Add an attribute with UNIX timestamp
    • Tell DynamoDB to treat an attribute as a TTL
    • Removal can take up to 48 hours after expiration
  • Use Cases: Great for removing irrelevant or old data
  • Helps to Save Money

Provisioned Throughput Calculations

  • Unit of Read provisioned throughput

    • ALL reads are round up in increments of 4 KB
    • Eventualy Consistent Reads (default) consist of 2 reads per second.
    • Strongly Consistent Reads consist of 1 read per second
    • Formula
      • Size of Read round to nearest 4KB chunk / 4 KB) x # of items = read throughput
      • Divide by 2 if eventually consistent

  • Unit of Write provisioned throughput

    • All writes are 1KB
    • All writes conists of 1 write per second

  • HTTP 400 Status Code - ProvisionedThroughputExceededException

    • You exceeded your maximum allowed provisioned throughput for a table OR for one or more global secondary indexes.
    • Use Exponential Backoff by sending retry requests using progressively longer waits (Built-in for AWS SDKs)

  • Question 1: You have an application that requires to read 10 items of 1 KB per second using eventual consistency. What should you set the read throughput to?

    • Determine the amount of read units per item we need
      • 1 KB rounded up to the nearest 4 KB increment = 4
      • 4 KB / 4 KB = 1 read unit per item
    • 1 x 10 read items = 10
    • Using eventual consistency we get 10 / 2 = 5
    • 5 units of read throughput required
    • (4 / 4) x 10 = 10, 10 / 2 == 5

  • Question 2: You have an application that requires to read 10 items of 6 KB per second using eventual consistency. What should you set the read throughput too?

    • Determine the amount of read units per item we need
      • 6KB rounded up to the nearest 4KB increment = 8 KB
      • 8 KB / 4 KB = 2 read units per item
    • 2 x 10 read items = 20
    • Using eventual consistency we get 20 / 2 = 10
    • 10 units of read throughput required
    • (8 / 4) * 20 = 2, 20 / 2 == 10

  • Question 3: You have an application that requires to read 5 items of 10 KB per second using eventual consistency. What whould you set the read throughput too?

    • Determine the amount of read units per item we need
      • 10 KB rounded up to the nearest 4KB increment = 12 KB
      • 12 KB / 4 KB = 3 read units per item
    • 3 x 5 read items = 15
    • Using eventual consistency we get 15 / 2 = 7.5
    • 8 units of read throughput required
    • (12 / 4) * 5 = 15, 15 /2 = 7.5 (round up)

  • Question 4: You have an application that requires to read 5 items of 10 KB per second using strong consistency. What should you set the read throughput too?

    • Determine the amount of read units per item we need
      • 10 KB rounded up to the nearest 4KB incement = 12 KB
      • 12 KB / 4KB = 3 read units per item
    • 3 x 5 read items = 15
    • Using strong consistency we get 15 / 1 = 15
    • 15 units of read throughput required
    • (12 / 4) * 5 = 15, 15 / 1 = 15

  • Question 5: You have an application that requires to write 5 items, with ech item being 10 KB in size per second. What should you set the write throughput too?

    • As write units are 1KB, we need 10 write units per item
    • We have 5 items, therefore 10 x 5 items = 50
    • 50 units of write throughput required

  • Question 6: You have an application that requires to write 12 items of 100 KB per item each second. What should you set the write throughput too?

    • As write units are 1KB, we need 100 write units per item
    • We have 12 items, therefore 100 x 12 = 1200
    • 1200 units of write throughput required

Web Identity Auth to DynamoDB

Steps to authenticate: 1. User Authenticates with ID provider (OpenID Connect provider such as Facebook/Google) 2. They are passed a Token by their ID provider 3. Your code calls 'AssumeRoleWithWebIdentity' API, and provides the providers token and specifies the ARN for the IAM Role 4. App can now access DynamoDB from between 15 mins to 1 hr (Default is 1 hr)

Other important bits

  • Conditional Writes

    • put logic in place as part of the write operation to only update/write if that condition is met.
    • Example 'If item = $10 then update to $12"
      • Conditional writes are 'idempotent' - meaning you can send the same conditional write request multiple times, but it will have no further effect on the item after the first rime Dynamo DB erforms the specified update
      • For example suppose you issue a request to update the price of a book item by 10%, with the expectation that the price is currently $20.
      • However, before you get a response, a network error occurs and you don't know whether your request was successful or not....
      • Becuase a conditional update is an 'idempotent' operation, you can send the same request again, and DynamoDB will update the price only if the current price is still $20.

  • Atomic Counters

    • DynamoDB support atomic counters
    • Where you use UpdateItem operation to increment / decrement the value on an existing attribute,
    • without interfering with other write requests. All write operations are applied in the order in which they were received
    • For example: a web app might want to maintain a counter per vistor to the site. In this case, the application would need increment this counter regardless of it's current value.
    • Atomic counter updates are NOT 'idempotent' - meaning that the counter will increment each time you call UpdateItem
    • If you suspect that a previous request was unsucessful, your application could retry the UpdateItem operation;
    • However, this would risk updating the counter twice.
    • This might be acceptable for a web site counter, because you can tolerate with slightly over or under-counting the vistors
    • However, in a banking application it would be safer to use a conditional update rather than an atomic counter

  • Batch Operations

    • If you application needs to read multiple items, you can use the BatchGetItem API.
    • A single BatchGetItem request can retrieve up to 1 MB of data, which can contain up to 100 items
    • Also, a single BatchGetItem request can retrieve items from multiple tables

CLI Commands

  • CreateTable, UpdateTable (e.g. change settings), ListTables, DescribeTable, DeleteTable
  • PutItem, GetItem (use primary key), UpdateItem, DeleteItem
  • Scan (read all then filter), Query (uses partition key)

KMS

  • AWS-Managed Customer Master Keys (CMKs)
    • Generated by integration with services like S3, SNS, etc.
  • Customer-managed CMK
    • Requires Alias, Description and Key Material (Use KMS, provide or Cloud HSM)
    • Key Administrative Permissions
      • Users or Roles with Administrative permissions
    • Key Usage Permissions
      • Users or Roles with Usage permissions
    • Usage
      • CMK is used to encrypt data up to 4KB
      • CMK can be used to generate data-key
      • CMK can be used to encrypt data-key (Envelope Encryption)
      • Envelope Encryption avoids sending all the data to the KMS
    • API calls
      • aws kms encrypt
      • aws kms decrypt
      • aws kms re-encrypt
      • aws kms enable-key-rotation
      • aws kms generate-data-key

SQS

  • Distributed queue system

  • Basically a temporary repository for messages that are awaiting processing

  • Good for decoupling application components

  • Autoscaling EC2 instances based on SQS queue metrics is available

  • Web service that gives you access to a message queue, that can be used to store messages whilst awaiting processing

  • Queue types:

    • Standard Queues (default)
      • Nearly unlimited TPS
      • Guarantee's a message is delivered atleast once, however:
      • Occasionally (due to distributed arch) more than 1 message may be delivered out of order
      • Best effort ordering - ensures the messages are generally delivered in the same order they were sent
    • FIFO (First in First Out) Queues
      • First-In-First-Out
      • Gaurantee's order is preserved
      • No duplicates
      • Limited to 300 TPS

  • SQS Settings

    • Visibility Timeout (ChangeMessageVisiblity)
      • Items becomes invisible when picked-up then visible after the timeout
      • Default: 30 seconds
      • Maximum: 12 hours
    • Retention Period
      • Messages can be kept in the queue from 1 minute to 14 days (default is 4 days)

  • Short polling vs Long polling

    • Short polling
      • Returns a response immediately even if the queue is empty
      • Pay for each response
    • Short polling
      • Wait for the message to show-up until timeout
      • Long poll save you money, by reducing the amount of time polled.
      • Maximum Long Poll Time Out = 20 seconds

  • Payload size

    • SQS can store messages of size up to 256 KB
    • Messages biggen than 256 KB can be stored in S3
    • To manipulate S3 messages the AWS SDK for Java is required

  • SQS Fanning Out (kinda like broadcasting a message)

    • Create an SNS topic first using SNS. Then create and subscribe multiple SQS queues to the SNS topic
    • Now whenever a message is sent to the SNS topic, the message will be fanned out to the SWS queues
    • i.e SNS will deliver the message to all the SQS queues that are subscribed to the topic

SNS

  • Instant, push-based delivery
  • Uses Pub / Sub model
    • Consumers need to subscribe to a topic
  • Supports:
    • Push notifications, SMS, Email, SQS, HTTP
  • Fanout
    • Can fanout messages to a large amount of recipients

SES

  • Simple Email Service
  • Used for incoming and outgoing emails
  • Not Subscription-based
    • Knowing an email is enough to start sending emails

Kinesis

  • Streaming Data - generated continuously by thousands of data sources, sending records simultaneously, and generally small sizes

    • Stock Prices
    • Game data (as they play)
    • Social network data
    • Geospatial data (think uber - continuous driver position data)
    • IoT sensor data

  • 'Producers' (e.g. EC2, phone, etc) send data to Kinesis

  • 'Consumers' (e.g. EC2 - then store in S3, DB etc)

  • Kinesis

    • Streams
      • Capture and store streaming video and data for real-time processing and analysis
      • Consumer applications process and analyse the data in real-time
      • Consists of Shards (default storage is 24hrs, can be increased to 7 days)
      • Can have multiple Shards in a streams
      • Best practice to have # consumers <= # of shards. Extra consumers for fail-over cases
      • Can setup autoscaling group based on CPu utilization for situation when adding more Shards
    • Firehose
      • Capture, transform and load data continiously into AWS data stores
      • BI apps and tools for near real-time analysis
      • 'Automated' in comparison with Streams
      • Don't need to manage shards (doesn't have them) / retention
      • Analysis via Lambda
      • Results to S3, then onwards to Redshift, etc.
    • Analytics
      • Uses data from Kinesis Data Streams or Kinesis Data Firehose
      • Real-time analytics using SQL
      • Results to S3, Redshift or Elasticsearch Cluster

CodeCommit, CodeBuild, CodeDeploye, CodePipeline

CodeArtifact

Elastic Container Service (ECS)

CodeStar

Elastic Beanstalk

  • Basically just upload your code (for ppl that are not very with AWS - or want low management overhead)

  • You can deploy, monitor and scale an application quickly

  • Provides developers/end users with the ability to provision application infrastructure in an almost transparent way

  • It has a highly abstract focus towards infrastructure, focusing on compoents and performance - not configuration and specifications

  • Attempts to remove, or significantly simplify infrastructure management, allowing applocations to be deployed into infrastructure environments easily.

  • Customizing Configuration

    • Amazon Linux 1
      • Using the .ebextensions folder
        • Applicable for Amazon Linux 1
        • Located in the top-level application directory
        • Files need to have .config extension
        • YAML or JSON can be used
    • Amazon Linux 2
      • Using Buildfile
        • Root level file
        • Suitable for operations which exit upon completion
      • Using Procfile
        • Suitable for long-running processes. For instance, custom command to start application
      • Platform Hooks
        • Various scripts to run at different stages of EC2 provisioning
        • .platform/hooks/prebuild
        • .platform/hooks/predeploy
        • .platform/hooks/postdeploy

  • Deployment Types

    • All at once (worst)
      • Updates all instances at the same time
      • Involves service interruption
    • Rolling
      • The new colde is deployed in batches
      • Reduced capacity during deployment
      • Rolling back requires a further Rolling update
    • Rolling with Additional Batch
      • Maintains full capacity
      • Rolling back requires a further Rolling update.
    • Immutable (best)
      • Maintains full capacity
      • To roll back: delete the new instances and update ELB
    • Traffic Splitting (best)
      • Performns Immutable deployment
      • Splits the traffic between the old and new deployments
      • Enables Canary Testing

  • Deploying RDS within Elastic Beanstalk

    • Inside Beanstalk
      • Quick and easy
      • Terminating EB environment also terminsated database
      • Suitable for Dev and Test envs
    • Outside Beanstalk
      • Requires additional configuration steps (Security Group, db connection)
      • Terminating EB env does not affect the database
      • Suitable for Prod env

  • Migration to Elastic Beanstalk

    • Your Data Center -> Elastic Beanstalk
    • AWS provided Migration Assistant Tool
      • Windows Web Application Migration Assistant
        • Formerly known an .NET migration assistant
        • Interactive PowerShell Utility
        • OpenSource and available on GitHub

  • Exam tips

    • You can have multiple versions of your applications
    • Your apps can be split into tiers (Web / App / DB)
    • You can update your application
    • you can update your configuration
    • You CAN SSH/RDP into your instaces
    • You pay for the resources that you use, EB is free
    • Language support
      • Apache Tomcat for Java
      • Apache HTTP server for PHP
      • Apache HTTP server for Python
      • Ngnix / Apache HTTP for Node.js
      • Passenger / Puma for Ruby
      • Microsoft IIS 7.5/8.0 and 8.5 for .NET
      • Java SE
      • Docker
      • Go

CloudFormation

CloudWatch

  • CloudWatch is all about monitoring the performance and health of your system
  • Gathers performance metrics from EC2 instances:
    • Default: CPU, Network, Disk and Status check
    • CloudWatch Agent: Memory Usage, Processes, CPU Idle Time
  • CloudWatch Logs
    • Monitors and stores logs
    • Gathers Logs from all AWS services (e.g. lambda)
  • CloudWatch Alarms
    • Alarm to monitor a metric and generate an alert
  • Concepts:
    • CloudWatch Metrics
      • A metric is a variable to monitor (e.g. CPU usage)
    • CloudWatch Namespaces
      • Container for cloudwatch metrics (e.g. AWS/EC2 namespace)
    • CloudWatch Dimensions
      • Filter metadata (e.g. InstanceID)
    • CloudWatch Dashboards
      • Customized homepage

CloudTrail

  • Records account activity (API calls) regarding creation/deletion of resources
  • Tracks failed logins

X-Ray

  • X-Ray helps developers analyze and debug distributed applications.

  • Service Map

    • Service Map is a visual representation of your application
    • Provides information about Latency, Status Codes and Errors

  • X-Ray Agent

    • Must be installed on your instance

  • X-Ray SDK

    • Instruments application to send traces to X-Ray

  • X-Ray Exam Tips:

    • Integrated with AWS Services
      • DynamoDB, Lambda, API Gateway, etc.
    • Your applications
      • Instrument data about HTTP requests using the SDK
    • Supported Platforms
      • EC2, Elastic Beanstalk, ECS and on-premises environments
    • Running X-Ray in ECS
      • Run the X-Ray daemon in its own Docker image, running alongside application
    • X-Ray Annotations
      • Add key-value pairs allowing to filter, index and search (e.g. game_name=TicTacTor, gameId=124)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment