Skip to content

Instantly share code, notes, and snippets.

@khurdz

khurdz/awsome_day_tlse-08-06-2018.md

Created June 12, 2018 08:04
Show Gist options
  • Save khurdz/b6269b3f97ac33b262e0abdb6fd43f10 to your computer and use it in GitHub Desktop.
Save khurdz/b6269b3f97ac33b262e0abdb6fd43f10 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
awsome_day_tlse-08-06-2018.md

Intro

D2SI

Cloud

  • Infrastructure as Code
    • No more configurations by hand
    • No more DAT
    • Code is the documentation
  • Do we create value in managing OS and infrastructures ? If no, go by managed services
  • Monitoring
    • Observability -> Predictivity

AWS Infrastructures

  • Regions
  • AZ (Availibility Zones)
  • Edge locations (CDN, PoP)
  • DDoS protection by default

Services

Core Compute Services

EC2 (Elastic Cloud Computing)

  • Main features
    • Resizable VM
    • Complete control of computing resources
    • Scale capacity as computing requirements change
    • Plan only for capacity that is actually used
    • Linux or Windows
    • Deploy across AWS multiple regions and AZ for reliability
  • AMI (Amazon Machine Image)
    • A template (OS, configurations, applications)
    • Can be built via Packer to specialize/customize images
    • AWS Marketplace is a store of AMIs and IT softwares
  • Instance User Data
    • Upstart script at boot (similar to a container entrypoint)
  • EC2 Lifecycle
    • One EC2 duplicated in several instances (VM)
    • Instances can be linked to EBS (disks, volumes, one per VM)
    • Pay as you go (running instances)
  • Instance Families
    • General purpose
    • Compute optimized
    • Memory optimized
    • Storage optimized
    • GPU instances, FPGAs
  • Naming example: c4.large
    • c: family
    • 4: generation
    • large: ratio CPU/RAM (gabarit)
    • Latest generations are not only less expensive, but also more efficient (newer processors)
  • Instance metadata
  • Purchasing options
    • On-demand instances
    • Reserved instances
    • Schedules instances
    • Spot instances ("trading" model)
    • Dedicated hosts (compliance, licensing...)
    • Choice can be made for any managed services (eg. EKS)

CloudFormation

EKS (Elastic Kubernetes Computing)

  • Management of the master node and the cluster, contrary to k8s installed on EC2
  • Slaves and workers are managed by yourself
  • One endpoint is exposed; from here, you can roll out your deployments as usual

Security

  • Shared Responsibility
    • AWS: security OF the cloud (compute, storage, db, network...)
    • Customers: security IN the cloud (apps, identity, access mgmt, OS, encrytion...)
    • More responsibility for AWS in case of managed services
  • Physical Security
  • Certifications & Accreditations
    • PCI, FIPS, ISO...
    • Global (and some local for US) certifications for every single workload
    • A whole domain of expertise, hard to obtain and to maintain by yourself
  • IAM (Identity and Access Management)
    • Password or access key auth
    • Roles, permissions, groups, users
    • JSON formatted policies, applyable on groups, roles
    • Profiles for instance ACL to resources
    • CloudTrail for audit, security and access monitoring

Network

VPC (Virtual Private Cloud)

  • IPv4/IPv6
  • Across a whole region over all AZ
  • No multicast/anycast
  • No VLAN (L3 only)
  • Subnets
    • Public
    • Private
    • Within a single AZ
  • Gateway
    • Internet Gateway
    • VPC NAT Gateway (if no proxy, enables private subnets to access Internet via a public subnet)
    • Virtual Private Gateway (towards third-party networks, HQ...) with direct connect (private link)
  • Security groups, ACLs
  • VPN IPSec, static or BGP routing

Storage

S3

  • Blob (object) storage
  • No bucket size limit (max one file size 5TB...)
  • Cost calculated with following parameters:
    • Storage size
    • Data out (don't forget to cache your content !)
    • Number of PUT/GET on endpoint
  • Encryption
  • Auditing
  • ACL, policies, IAM
  • Versioning of objects
  • Object lifecycle
    • Moving to S3 infrequent access as data gets cold
    • Archiving with Glacier (very cold data, takes hours to retrieve data)

EBS (Elastic Block Store)

  • Storage volume
  • Can be attached to EC2 instances
  • Possible snapshots to S3
  • SSD/HDD
  • Long-term persistence
  • Possible encryption
  • EBS volumes are in a single AZ
  • Volume data is replicated across multiple servers in an AZ

EC2 Instance Store

  • Local storage
  • No persistence (automatically deleted at VM stop)
  • Useful for big data workloads or high performance compute (up to 3 million IOPS)

Database Managed Services

Data Storage

  • Data requirements (format, size, frequency...)
  • DB engine depends on the latter (there is no such thing as an universal DB engine)

AWS Migration Database Services

Relational DB

  • RDS
    • Community: MySQL, PGSQL, MariaDB
    • Commercial: Oracle, SQL Server
    • Aurora: MySQL, PGSQL
      • High availability (3 AZ by default)
      • Scalability
      • Better performance
      • SQL/transactions are decoupled from storing (caching, logging, data on S3)
      • Multi Master and Serverless (coming soon)
    • Automatic backups, manual snapshots, rewind (up to 72 hours ago)
    • Active/standby on multi-AZ
  • Redshift
    • Data Warehouse (column-oriented)

NoSQL

  • DynamoDB (key/value -> document)
  • ElastiCache (In-memory datastore -> Redis, Memcached)
  • Neptune (Graph)

AWS Lambda: Serverless Compute

  • Compute service + event-driven
  • Lesser costs
  • Warmup (startup, network if ELB)
  • Lot of sources for events
  • Pay as you go (per request)
  • API Gateway

Triad of Services for Production Performance

  • ELB (Elastic Load Balancer)
  • CloudWatch: monitoring system/apps, alerting
  • Auto-scaling
    • Auto-scaling group
    • Auto-scaling triggers:
      • Latency
      • CPU
      • Custom metrics

Miscellaneous

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment