This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from rockset import Client, Q | |
from lambdarest import lambda_handler | |
from credentials import API_KEY | |
import json | |
rs = Client(api_key=API_KEY, | |
api_server='https://api.rs2.usw2.rockset.com') | |
def lambda_handler(event, context): | |
if 'queryStringParameters' in event: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rockset> select "my-field", "my-other-field" | |
from new_collection; | |
+------------+------------------+ | |
| my-field | my-other-field | | |
|------------+------------------| | |
| doc1 | some text | | |
+------------+------------------+ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rockset> select "my-field", "my-other-field" | |
from new_collection; | |
+------------+---------------------------------------------------------------+ | |
| my-field | my-other-field | | |
|------------+---------------------------------------------------------------| | |
| doc1 | some text | | |
| doc2 | [{'c1': 'this', 'c2': 'field', 'c3': 'has', 'c4': 'changed'}] | | |
+------------+---------------------------------------------------------------+ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rockset> select mof.* | |
from new_collection, unnest(new_collection."my-other-field") as mof; | |
+------+-------+------+---------+ | |
| c1 | c2 | c3 | c4 | | |
|------+-------+------+---------| | |
| this | field | has | changed | | |
+------+-------+------+---------+ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rockset> select mof.* | |
from new_collection, unnest(new_collection."my-other-field") as mof; | |
+------------+-------+------+---------+ | |
| c1 | c2 | c3 | c4 | | |
|------------+-------+------+---------| | |
| unexpected | 99 | 100 | 101 | | |
| this | field | has | changed | | |
+------------+-------+------+---------+ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rockset> select typeof(mof.c2) | |
from new_collection, unnest(new_collection."my-other-field") as mof; | |
+-----------+ | |
| ?typeof | | |
|-----------| | |
| int | | |
| string | | |
+-----------+ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import boto3 | |
kinesis = boto3.client('kinesis') # requires AWS credentials to be present in env | |
kinesis.create_stream(StreamName='twitter-stream', ShardCount=5) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# twitter api credentials | |
access_token=... | |
access_token_secret=... | |
consumer_key=... | |
consumer_secret=... | |
class TweetListener(StreamListener): | |
def __init__(self, stream_name): | |
self.kinesis = boto3.client('kinesis') | |
self.stream_name = stream_name |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from rockset import Client, Q, F | |
rs=Client(api_key=...) | |
aws_integration=rs.Integration.retrieve(...) | |
sources=[ | |
rs.Source.kinesis( | |
stream_name="twitter-stream", | |
integration=aws_integration)] | |
twitter_kinesis_demo=rs.Collection.create("twitter-kinesis-demo", sources=sources) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SELECT t.timestamp_ms, | |
t.created_at AS created_at, | |
t.text AS text, | |
t.user.screen_name AS screen_name | |
FROM "twitter-kinesis-demo" t | |
WHERE CAST(timestamp_ms AS INT) > UNIX_MILLIS(current_timestamp() - minutes(1)) | |
ORDER BY timestamp_ms DESC | |
LIMIT 100; |
OlderNewer