Presenter: Jay Phelps (@_jayphelps)
- Jay Phelps
- Senior Software Engineer
- InfoSec
- Preventing unauthorized access
- Stopping hackers
- Can block exploits using their gateway proxy
- They need to know if it's working
- They want to watch attackers try
- to learn/refine their defenses
- they want to see it in real-time
- Real-time
- for debugging, infosec, logging
- thousands of servers, millions of devices, all logging
- how do they handle massive amounts of streaming logs?
- RX
- best ideas from observer pattern, iterator pattern, and functional programming
- "lodash for events"
- Available for many languages
- FB, Slack, Microsoft, Netflix, Google, GitHub, Airbnb
- High level intro
- Observable = basic primitive
- Array = collections of item
- Observable = collections of items over time
- Represents a stream
- Marble diagrams are useful for visualizing how thing work with streams (things over time)
- Stream of logs
- Angular 2 uses RxJS
- Logging
- Logging JSON
- Created a SQL like query language to create RxJava
- 8 million messages per second, at peak: how do we scale this?
- Load balancing jobs (autoscaling)
- Need to chain jobs together
- Segment traffic, then query
- High-volume distributed systems have a problem: backpressure
- pressure opposed to the desired flow
- deficit of ability to calculate, causing set of things unable to be handled, growing over time
- Can Buffer (hold and wait) or Drop (discard)
- Job authors get to choose whether to drop or buffer
- they buffer, then autoscale, then drop
- Netflix's lib for this is Mantis
- Where do they actually query this?
- Query builder UI
- Source, Fields, Conditions, and Query Preview
- Problem: can be really high volume (100k+ rps)
- Performance solutions are often driven by UX
- partial Solution: UI virtualization
- still can't update the virutal table 100k/s
- this inability to update is also backpressure
- need to buffer or drop
- UX problem still not resolved; buffer size is unbounded
- Users just want a sample
- See a little bit, in real time, then refine the query
- solution: batch sampling
- Buffer for a bit, then drop after reaching a certain threshold
- 50 was a great number
- How this is implemented: goo.gl/DMOqBA
- Rx enabled making this really easy (and performantly)
- Works for low-volume queries too
- If im a tester, on a device, I just want to see my requests
- I can turn on the firehose if I want
- This tool is called Raven at Netflix
- (It's beautiful) - Ryan
- Can write Javascripts
- Can push transformations to the servers
- Has alerting!
- Has improved debugging, testing, and InfoSec
- Query builder UI
- Netflix loves Rx
- It's powerful, and cross-platform
- Can follow you to basically any language