Skip to content

Instantly share code, notes, and snippets.

@ns-mkusper
Last active July 11, 2019 05:25
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ns-mkusper/cdbfce8e6baaddfb654dd950ee9dc686 to your computer and use it in GitHub Desktop.
Save ns-mkusper/cdbfce8e6baaddfb654dd950ee9dc686 to your computer and use it in GitHub Desktop.
Get Sliding 60-second Owner Count Kinesis Stream Analytics
-- ** Aggregate (COUNT, AVG, etc.) + Sliding time window **
-- Performs function on the aggregate rows over a 10 second sliding window for a specified column.
-- .----------. .----------. .----------.
-- | SOURCE | | INSERT | | DESTIN. |
-- Source-->| STREAM |-->| & SELECT |-->| STREAM |-->Destination
-- | | | (PUMP) | | |
-- '----------' '----------' '----------'
-- STREAM (in-application): a continuously updated entity that you can SELECT from and INSERT into like a TABLE
-- PUMP: an entity used to continuously 'SELECT ... FROM' a source STREAM, and INSERT SQL results into an output STREAM
-- Create output stream, which can be used to send to a destination
CREATE OR REPLACE STREAM "DESTINATION_SQL_STREAM" (owner bigint, owner_count INTEGER);
-- Create a pump which continuously selects from a source stream (SOURCE_SQL_STREAM_001)
-- performs an aggregate count that is grouped by columns ticker over a 10-second sliding window
CREATE OR REPLACE PUMP "STREAM_PUMP" AS INSERT INTO "DESTINATION_SQL_STREAM"
-- COUNT|AVG|MAX|MIN|SUM|STDDEV_POP|STDDEV_SAMP|VAR_POP|VAR_SAMP)
SELECT STREAM "owner", COUNT(*) OVER MINUTE_SLIDING_WINDOW AS owner_count
FROM "SOURCE_SQL_STREAM_001"
-- Results partitioned by owner and a 60-second sliding time window
WINDOW MINUTE_SLIDING_WINDOW AS (
PARTITION BY "owner"
RANGE INTERVAL '60' SECOND PRECEDING);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment