S3
SQS
SNS
Amazon Simple Workflow (SWF)
EC2
Amazon DynamoDB
Amazon Elastic Beanstalk
AWS Cloud Formation
Cross Account Access
##Shared Security Responsibility AWS is responsible for portions of the cloud, and you as the customer have portions of the cloud that you are responsible for - thus creating shared security responsibility.
STS allows you to create temporary security credentials that grant trusted users access to your AWS resources.
Only called through API/SDK
Challenge - STS API Calls
AssumeRole: Cross-Account Delegation Federation Through a Custom Identity Broker
AssumeRoleWithWebIdentity: Federation Through a Web-based Identity Provider
AssumeRoleWithSAML: Federation Through an Enterprise Identity Provider Compatible with SAML 2.0
GetFederationToken: Federation Through a Custom Identity Broker
GetSessionToken: Temporary Credentials for Users in Untrusted Environments
Customer Master Keys (CMKs)
Customer-managed
AWS-managed
Data Keys
Envelope Encryption
API Actions
Encrypt
GenerateDataKey
Decrypt
Create unique identities for application users
Authenticate identities
Save mobile data
Use credentials obtained to sync data with Cognito sync
AMI: The operating system and other settings
Instance Type: The hardware compute power, RAM, network bandwidth
Network Interface: public, private or elastic IP addresses
Storage: The instances “hard driver”
Elastic Block Store (EBS) - which is “network persistent storage”
Elastic File System (EFS) - “scalable, elastic file storage”
Instance Store - which is “ephemeral storage” - the data on the volumes only exists for the duration of the life of the instance
EFS can accessed by one (or more) EC2 instances at the same time
Can be attached to an instance using userdata script
Is elastic
Fully managed by AWS
Shared file access across all your EC2 instances
Applications that span multiple EC2 instances can access the same data
Public key is stored on instance and you keep the private key (.pem file)
AWS Direct Connect
Managing session state with ELBs
Load balancer generated cookie stickiness
Application generated cookie stickiness
(Recommended) Non-ELB option - using Redis or Memcached for the session
AWS VPC - enables you to launch AWS resources into a virtual network that you’ve defined.
Ability to launch instances into a subnet
Ability to define custom CIDR (IP address range) inside each subnet
Ability to configure routes between subnets via route tables
Ability to configure an internet gateway to provide a route to the internet for resources launched inside the VPC
Ability to create a layered network of resources
To enable access to or from the internet for instances in a VPC subnet you must attach an internet gateway to your VPC
ensure that your subnet’s route table points to the IGW
ensure that instances in your subnet have a public/Elastic IP address
ensure that your network access control and security group rules allow the relevant traffic to and from your instance
Is a VPC component that allows communication between instances in your VPC and the internet
It imposes no availability risks or bandwidth constraints on your network traffic
Default VPC created for new accounts already has an IGW attached.
A route table contains a set of rules, called routes, that are used to determine where network traffic is directed
Has two main components
Destination: The CIDR block range of the target (where data is routed to)
Target: A name identifier of where the data is being routed to
Subnets must be associated with a route table
A public subnet has a route to the internet
It is associated with a route table that has an IGW attached
A private subnet does not have a route to the internet
It is associated with a route table that does not have an IGW attached
Instances launched into a private subnet can’t communicate with the internet. This is solved by routing traffic through a NAT instance/gateway.
By default all subnets traffic is allowed to each other available subnet within via the local target in the route table/ Security
Rules are evaluated in order, starting with the lowest rule number
The last rule in every ACL is a “catch all” deny rule - unless a protocol/port is explicitly allowed, it will be denied
Must have inbound rules with corresponding outbound rules
Security group - firewall on the instance level - stateful
Bastion Host/NAT Networking
Is an EC2 instance that lives in a public subnet
NAT Gateway
Is designed to provide EC2 instances in a private subnet with a route to the internet
Prevents any hosts outside of VPC from initiating a connection
Will only allow incoming traffic through if a request if a request for it comes from an instance in a private subnet
MUST be created in a public subnet
Be part of the private subnets route table
Boto3 is available by default in lambda runtime
Pip install -r requirements.txt -t . (pip install requirements in dir (-t ) current one
Open . (in terminal opens current dir)
$LATEST version is the only mutable (changeable) version
Use context object to get running time left
Lambda aliases are like a pointer to a specific lambda version
Aliases have a static ARM but can point to any version of the same function
Aliases can also be used to split traffic between Lambda version
Using Alias traffic splitting between versions can also help test new versions in production
Is a docker-compatible container service provided by AWS. It allows for easy and fast container deployment onto fleets of EC2 instances, with the added benefit of AWS highly available and fault tolerant infrastructure. ECS is great for distributed applications and microservices.
ECS is a container management service that supports Docker
It allows you to easily create and manage a fleet of Docker containers on a cluster of EC2 instance
Why use ECS/Container
Create distributed applications and microservices
Batch and ETL jobs
Continuous Integration and Deployment
A container registry is a repository where container/docker images are stored (Elastic Container Registry ECR, Docker Hub or self hosted registry) and accessed from when needed
A JSON file that contains the “blueprint” for your app
Which container/docker image to use
The repository (container registry) the image is located in
Which ports should be open on the container instance
What data volumes should be used with the containers
Actual representation of the task definition on an EC2 instance inside of your container cluster
ECS Agent will start/stop these tasks based on instruction/schedule
ECS Agent
It runs on each EC2 instance in the ECS cluster
It communicates information about instances to ECS such as running tasks, resource utilization
It is also responsible for starting/stopping tasks (when told to by ECS)
Can upload max size of up to 5TB
Single upload
Multipart upload
Allows for stopping/resuming file uploads
If transmission of any part fails, you can retransmit that part without affecting other parts
Required for objects 5GB and lager and highly suggested for objects 100MB and larger
Storage Gateway: connects local data center software appliances to cloud based storage such as S3
Gateway-Cached Volumes create storage volumes and mount them as iSCSI devices on the on-premises servers
The gateway will store the data written on this volume in S3 and will cache frequently accessed data on-premises in the storage device
Gateway-Stored Volumes
Store all the data locally (on-premises) in storage volumes
Gateway periodically takes snapshots of the data as incremental backups and stores them on S3.
Snowball
A physical hard-drive sent to you by AWS to upload data to S3.
AWS Import/Export
gives the ability to take on-premises data and physically mail to AWS
Aws will import the data to either S3, EBS or Glacier within one business day of the physical device arriving at AWS
Performance
Prepend unique prefixes to keys optimize workload .ie bucket/zyz-key/key/key (Not true anyone)
Use cloudfront
Permissions
All buckets and object are private by default
Resource Based Policies
Bucket policies are attached to s3 buckets not objects.
Written in JSON
ACLs can be used with s3 buckets and s3 objects
Written in XML
DynamoDB can have up to 256 tables per region
Fully managed NoSQL db
DynamoDB Accelerator (DAX) - cache - for faster response time
AWS scales compute resources for you
Schemaless, key-value store
Every db requires a partition key
Partition key of an item is also known as its hash attribute
Simple primary key is made of partition key only
Items must have unique partition keys
Composite primary key is made of partition key and sort key
Items must have a unique combination of partition key and sort key
Primary key attributes must have data type of string, number or binary
Provisioned throughput is the maximum amount of capacity that an application can consume from a table or index
Write Capacity Unit WCU = (items/sec) * size (strongly consistent) (round up to nearest 1kb)
Read Capacity Unit RCU = (items/sec) * (size/4) (roundup size to the next multiple of 4kb)
Eventually consistent reads require half the throughput of strongly consistent reads
Read operations:
GetItem, BatchGetItem, Query, Scan
Limit of 1MB of data returned
Eventual consistency does not guarantee the most recent writes to a table
Strong consistency does guarantee the read to reflect most recent writes
Secondary Indexes SI
A maximum of 5 local SI and 5 global SI per table
Indexes allow efficient queries of non-primary key attributes
Every SI is associated with only one table
Tables can have multiple SI
Is a petabyte-scale data warehousing service
Fully managed and scalable
Generally used for big-data analytics
Integrates well with popular business intelligence tools
Jaspersoft
Microstrategy
Pentaho
Tableau
Business Objects
Cognos
Caching strategies
Lazy loading
With lazy loading, you write data to the cache when a cache miss occurs
Lazy loading avoids filling up the cache with data that won’t be requested
Write through
When using a write through strategy, the cache is updated whenever a new write or update is made to the underlying database.
Adding Time To Live (TTL) A TTL is the length of time before a key expires
Topics require unique names limited to 256 characters
Topic names allow alphanumeric characters plus hyphens and underscores
Messages can only live for 14 days
Default visibility timeout is 30 seconds
Provides the ability to have hosted/highly available queues that can be used to send and receive messages being sent between producers and consumers.
Messages between servers are retrieved through polling.
Long polling (1-20 seconds)
Occurs only when the queue or the message has a ReceiveMessageWaitTimeSeconds > 0
Reduces API requests
Allows SQS service to wait until a message is available in a queue before sending a response
Short polling (0 second)
Occurs only when the queue or the message has a ReceiveMessageWaitTimeSeconds = 0
SQS samples a subset of servers and returns messages from just those servers
Will not return all possible messages in a poll
Increases API requests (over long polling), which increases costs
Each message can contain up to 256kb of text
Standard Queues
Support a ‘nearly unlimited’ number of messages per second
Support multiple producers and multiple consumers
Guarantee delivery of each message at least once with best effort ordering
Are used with applications that can tolerate duplicate messages
Support 120,000 in flight messages
FIFO Queue
Support up to 3,000 messages per second with batching and 300 messages per second without batching
Support multiple producers but only support multiple consumers through the use of Group IDs
Are designed for applications where the order of operations and events is critical, or where duplicates can’t be tolerated.
Support 20,000 in flight messages
Queue settings
Dead-Letter Queues: used to deal with malformed data from consumers
DelaySeconds: How long messages take to be added to the queue upon creation
VisibilityTimeout: How long messages are invisible after being received by a consumer
Producers send SQS messages to a queue
Consumers are responsible for retrieving and deleting messages from a queue
Fully managed service
SWF workflow/decider allows an architect/developer to implement distributed, asynchronous application as a workflow.
Coordinates and manages the execution of activities that can be run asynchronously across multiple computing devices
Has consistent execution
Guarantees the order in which tasks are executed
There are no duplicates
A workflow execution can last up to 1 year
Components
Workflows are a sequence of steps required to perform a specific task
A workflow is commonly referred to as a decider
Activities are a single step (or unit of work) in the workflow
Tasks are what interacts with the “workers” that are part of a workflow
Activity task: tells a worker to perform a function
Decision task: tells the decider the state of the workflow execution, which allows the decider to determine the next activity to be performed
Workers are responsible for receiving a task and taking action on it
Basic Monitoring
Data is available automatically in 5-minute periods at no charge.
Detailed Monitoring
Data is available in 1-minute periods
##CloudFormation
Infrastructure as code Cloudformation template can be written as JSON/YAML
Cloudformation stacks are a collection of AWS resources
Only Resources are required
Intrinsic Functions Fn::GetAtt
Fn::Join
Ref
Provides a secure way to store configuration information and sensitive secrets
Traces requests as they move through your applications.
You can use it to gain insights and identify potential optimizations to make inside your application
A managed build service that can compile your source code, run unit tests, and produce deployment artifacts
Automates deployments of your applications to EC2, Lambda and so on
Benefits
Automated deployments
Minimize downtime
Stop and rollback
Centralized rollback
By default, it uses in-place deployment
The existing servers are updated with the new version of an app
Blue/green deployment (EC2)
New app versions are deployed on a new set of instances
Blue/green deployment (Lambda)
Traffic is shifted from one lamba version to another
Canary A percentage of traffic is shifted to the new version
CodeDeploy then waits for a specified time and shifts the rest of the traffic if it sees no errors
Linear
Traffic is shifted in equal increments with an equal number of minutes between each increment
All at once
Traffic is immediately and completely shifted to the new version of the Lambda function
Develop, build and deploy applications