Skip to content

Instantly share code, notes, and snippets.

Avatar

Michael G. Noll miguno

View GitHub Profile
@miguno
miguno / copartitioning.sql
Last active Nov 28, 2019
ksqlDB example: creates a new stream with changed number of partitions and a new field as event key (so that its data can be correctly co-partitioned for joining)
View copartitioning.sql
CREATE STREAM products ...;
CREATE STREAM products_repartitioned
WITH (PARTITIONS=42) AS
SELECT * FROM products
PARTITION BY product_id
EMIT CHANGES;
@miguno
miguno / increased-partitions.sql
Last active Jan 11, 2020
ksqlDB example: Create a new stream with the desired number of partitions.
View increased-partitions.sql
CREATE STREAM products ...;
CREATE STREAM products_repartitioned
WITH (PARTITIONS=30) AS
SELECT * FROM products
EMIT CHANGES;
@miguno
miguno / topic-as-table.java
Last active Jan 9, 2020
Kafka Streams Example: read topic as table
View topic-as-table.java
// Create KTable from Kafka topic.
KTable<String, String> table = builder.table("input-topic", Consumed.with(Serdes.String(), Serdes.String()));
@miguno
miguno / topic-as-table.sql
Last active Jan 9, 2020
ksqlDB example: read topic as table
View topic-as-table.sql
-- Create ksqlDB table from Kafka topic.
CREATE TABLE myTable (username VARCHAR, location VARCHAR)
WITH (KAFKA_TOPIC='input-topic', KEY='username', VALUE_FORMAT='...');
@miguno
miguno / topic-as-stream.sql
Last active Jan 9, 2020
ksqlDB example: read topic as stream
View topic-as-stream.sql
-- Create ksqlDB stream from Kafka topic.
CREATE STREAM myStream (username VARCHAR, location VARCHAR)
WITH (KAFKA_TOPIC='input-topic', VALUE_FORMAT='...');
@miguno
miguno / topic-as-stream.java
Last active Jan 9, 2020
Kafka Streams Example: read a topic as a stream
View topic-as-stream.java
// Create KStream from Kafka topic.
StreamsBuilder builder = new StreamsBuilder();
KStream<String, String> stream =
builder.stream("input-topic", Consumed.with(Serdes.String(), Serdes.String()));
@miguno
miguno / aggregation.java
Last active Jan 9, 2020
Kafka Streams Example: Continuously aggregating a stream into a table
View aggregation.java
// Continuously aggregating a KStream into a KTable.
KStream<String, String> locationUpdatesStream = ...;
KTable<String, Long> locationsPerUser
= locationUpdatesStream
.groupBy((k, v) -> v.username)
.count();
@miguno
miguno / aggregation.sql
Last active Jan 9, 2020
ksqlDB example: Continuously aggregating a stream into a table with a push query
View aggregation.sql
-- Continuously aggregating a stream into a table with a ksqlDB push query.
CREATE STREAM locationUpdatesStream ...;
CREATE TABLE locationsPerUser AS
SELECT username, COUNT(*)
FROM locationUpdatesStream
GROUP BY username
EMIT CHANGES;
@miguno
miguno / midipipe.applescript
Last active Jun 7, 2020
MidiPipe AppleScript for macOS to reconfigure the AKAI APC 40 mk2 MIDI controller to execute Undo (Cmd-Z) and Redo (Shift-Cmd-Z) in Ableton Live with the controller's 'NUDGE -' and 'NUDGE +' buttons, respectively, instead of nudging the tempo.
View midipipe.applescript
# MidiPipe: http://www.subtlesoft.square7.net/MidiPipe.html
#
# The APC 40 out-of-the-box is great for performing, but less useful for producing because it lacks
# frequently used functionality such as Undo/Redo (doh, made a mistake) or deleting clips, for which
# you are forced to go back to mouse and keyboard.
#
# This example AppleScript shows how to use MidiPipe to re-configure the AKAI APC 40 mk2 MIDI controller
# to execute Undo (Cmd-Z) and Redo (Shift-Cmd-Z) in Ableton Live with the controller's 'NUDGE -' and 'NUDGE +'
# buttons, respectively, instead of nudging the tempo.
#
@miguno
miguno / foo.java
Last active Sep 27, 2019
Read KTable / TABLE from beginning (not from 'now')
View foo.java
StreamsBuilder#table("my-topic",
Consumed.with(AutoOffsetReset.EARLIEST));