Kafka Streams is a flexible and powerful framework. The Domain Specific Language (DSL) is an obvious place from which to start, but not all requirements fit the DSL model.
Many people are unaware of the Processor API (PAPI) - or are intimidated by it because of sinks, sources, edges and stores - oh my! But most of the power of the PAPI can be leveraged, simply through the DSL #process method, which lets you attach the general building block Processor interface to your -easy to use- DSL topology, to combine the best of both worlds.
In this talk you’ll get a look at the flexibility of the DSL’s process method and the possibilities it opens up. We’ll use real world use-cases borne from extensive experience in the field with multiple customers to explore power of direct write access to the state stores and how to perform range sub-selects. We’ll also see the options that punctuators bring to the table, as well as opportunities for major latency optimisations.
Key takeaways:
- Understanding of how to combine DSL and Processors
- Capabilities and benefits of Processors
- Real-world uses of Processors