Carlos Justiniano, Chief Software Architect Flywheel Sports (bio: http://cjus.me)
Building Microservices can be a daunting experience. Small teams often struggle with the dizzying array of tools required to build, test and maintain services. In this talk we'll examine how to address key microservice concerns using Redis as one of the few external infrastructure dependencies!
This talk is for individuals who have started or are about to embark on a journey to build Microservices and are wondering how Redis can help.
Two years ago, at Flywheel Sports, we set out to embrace Microservices. When we started, we had a small team of developers with experience building Node / ExpressJS applications using NoSQL databases and Redis. What we collectively lacked was experience with the overwhelming array of tools used to build and deploy microservices in the cloud. Tools such as etcd, consul, and zookeeper for service discovery, and Docker, Kubernetes for containerization and orchestration. We did have experience with messaging middleware, but not Kafka.
The issue we faced was slowing feature development in order to ramp up on the necessary tools OR "right-size" a solution to meet our needs. We chose the latter and set out to create a small microservices library which wouldn't be overwhelming for Node and ExpressJS developers. Developers who, by the way, lacked extensive experience building and managing microservices at scale. Given our size and lack of DevOp and SREs, we wanted our microservices to be easy to build, maintain and deploy. We knew we would embrace tools as we went along if we could first build a strong foundation.
Our efforts resulted in two libraries and a few supporting tools. At the core of our microservice solution is Redis. By building our Hydra (npmjs.org) library using Redis, we were able to address the following key microservice concerns.
- Service discovery
- Presence management
- Interservice messaging over HTTP REST and/or Redis pub/sub
- Built-in load balancing and routing
- Service-level message queues
- Near zero configuration
Fast forward to today, our use of Redis and our Hydra Microservices library powers our Live video streaming fitness classes via our nationwide FlyAnywhere platform.
- The challenges of building Microservices
- What is and isn't a microservice
- What makes building microservice hard?
- Identifying key microservice concerns
- Teaser for how those concerns will be addressed using Redis
- What we built at Flywheel
- Quick demo of Hydra services
- Distructuring Hydra to see how it leverages Redis
- Redis as a directory service
- Keyname spacing
- The power expirations
- Presence managements using hset / hget / hgetall
- Messaging using pub/sub and key namespacing
- Queue management using rpush / rpoplpush / lmem
- Batching operations
- Using scan instead of keys
- Microservice flight recorder using lpush / ltrim
- Building a similar library in your favorite language
- Hydra is built using NodeJS
- Creating a similar library using Go, Python, C/C++
- Building tools that further extend what we've seen
- A look at how Hydra-Router leverages Redis for service discovery and routing