Skip to content

Instantly share code, notes, and snippets.

View apurvam's full-sized avatar

Apurva Mehta apurvam

View GitHub Profile
@apurvam
apurvam / ksql-print-metrics-example.txt
Last active January 31, 2018 02:13
Example output for the ksql-print-metrics-tool
$ ./bin/ksql-print-metrics
messages-consumed-avg: 96416.96196183885
messages-consumed-min: 88900.3329377909
error-rate: 0.0
num-persistent-queries: 2.0
messages-consumed-per-sec: 193024.78294586178
messages-produced-per-sec: 193025.4730374501
num-active-queries: 2.0
num-idle-queries: 0.0
messages-consumed-max: 103397.81191436431
@apurvam
apurvam / safe-drop-example.sh
Created April 24, 2018 22:15
Example of how KSQL protects against dropping tables and streams which are in use.
ksql> show queries;
Query ID | Kafka Topic | Query String
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CSAS_PAGEVIEWS_ENRICHED | PAGEVIEWS_ENRICHED | CREATE STREAM pageviews_enriched AS SELECT users_original.userid AS userid, pageid, regionid, gender FROM pageviews_original LEFT JOIN users_original ON pageviews_original.userid = users_original.userid;
CSAS_PAGEVIEWS_FEMALE | PAGEVIEWS_FEMALE | CREATE STREAM pageviews_female AS SELECT * FROM pageviews_enriched WHERE gender = 'FEMALE';
CSAS_PAGEVIEWS_FEMALE_LIKE_89 | pageviews_enriched_r8_r9 | CREATE STREAM pageviews_female_like_89 WITH (kaf
@apurvam
apurvam / table-aggregations.sql
Last active April 28, 2018 17:53
Example of table aggregations in KSQL
CREATE TABLE users (userid bigint, zipcode int, created_at bigint) \
WITH (kafka_topic=’users’, value_format=’json’, key=’userid’);
SELECT zipcode, count(*) from users GROUP BY zipcode;
@apurvam
apurvam / timestamp-format.sql
Last active April 28, 2018 17:54
Example of specify TIMESTAMP_FORMAT in KSQL
CREATE STREAM foo (col1 varchar, col2 varchar, timecol varchar) \
WITH (kafka_topic=’foo_topic’, TIMESTAMP=’timecol’, \
TIMESTAMP_FORMAT=’yyyy-MM-ddTHH:mm:SSZ’, value_format=’json’);
@apurvam
apurvam / get_open_prs.py
Last active February 13, 2019 22:05
Get all open PRs in all repositories matching a regex.
#!/usr/bin/python
from urllib2 import Request, urlopen, URLError
import sys
from argparse import ArgumentParser
import json
import re
from dateutil.parser import parse
MAX_RESULTS = 50
@apurvam
apurvam / responsive-kafka-streams.diff
Last active September 19, 2023 05:17
Kafka Streams to Responsive
- KafkaStreams streams = new KafkaStreams(
+ KafkaStreams streams = new ResponsiveKafkaStreams(
builder.build(),
props
);