Skip to content

Instantly share code, notes, and snippets.

@baweaver

baweaver/scaling_christmas.md Secret

Created October 5, 2018 06:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save baweaver/512e901196d195c7dda5a3b394bd61d1 to your computer and use it in GitHub Desktop.
Save baweaver/512e901196d195c7dda5a3b394bd61d1 to your computer and use it in GitHub Desktop.
Download ZIP
Scaling Christmas - An Illustrated Adventure
Raw
scaling_christmas.md

Scaling Christmas - An Illustrated Adventure

Very brief outline. Remembering I have 10 minutes I need to trim the heck out of this or really gloss over subject areas.

Outline

  • Introduction
    • Set the scene
    • Christmas story
    • The true Mrs Claus - Master architect of scaling Christmas
  • Oh no!
    • Santa is sick - Can't deliver presents!
    • What to do? - Call in the lemurs, emergency
  • Mrs Claus scales Christmas
    • Logging and Monitoring
      • Statistics on good and bad kids - Regional awareness to predict potential issues (Washington DC will show bright red)
      • Predicted capacity requirements - Base on previous years and current scale
      • Past delivery rates - See previous years rate of response
    • Oncall Rotations
      • Paging to relevant toy makers - Make sure to ping relevant toy makers on breakages
      • Dependency tracking, A down causes B and C down - Tracking upstream toy and component makers and getting them involved earlier.
    • Load Balancing
      • Subsidiary branches request presents from main, lemurs deliver - Main offices dispatch requests to lemurs, lemurs respond with presents
      • Regional availability - US west vs Asia vs European zones for faster present delivery and lower latency
    • Autoscaling
      • Using capacity info to scale up delivery fleet - Scaling up to handle initial traffic, using monitoring to decide to scale up on utilization
      • Scale down as subsidiary branches get presents - Most presents are delivered, cut down on load
    • DDoS Mitigation
      • Reroute suspicious requests from known bad kids - Bad kids want to ruin Christmas! They make bad requests, ignore and send coal.
      • Multi-zone, no direct public API access - No way to directly contact Santa HQ, must go through subsidiary offices
      • Points of presence - If a subsidiary is overrun by bad kids, shut it down until things calm down to ensure eventual delivery and better availability in other zones.
    • Disaster recovery
      • Deployable sleigh configurations - Be able to deploy deterministic sleds and lemur configurations to launch another fleet
      • Santaforming - provision base necessities from a single script
  • Santa is well!
    • Congratulations to the lemurs
    • There's a ton to do to scale all this!
      • Reflect on lessons
  • Merry Christmas and Happy Holidays to all!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment