Skip to content

Instantly share code, notes, and snippets.

@hectcastro

hectcastro/time-series-riak.md

Last active December 24, 2015 17:19
Show Gist options
  • Save hectcastro/6834619 to your computer and use it in GitHub Desktop.
Save hectcastro/6834619 to your computer and use it in GitHub Desktop.
Download ZIP
General description of an approach to store and retrieve time-series data in Riak.
Raw
time-series-riak.md

First, you need to determine the level of precision you need to query by. In the following example, it ends up being timeboxed into 10 second bins, but in groups of 100 different entities (or meters in this example):

Riak key layout

Then, answering a query like:

Give me <45 minutes> worth of data from <18 hours ago> for entities <1, 2, 226, 301>

Ends up being an aggressive fetch (light pink):

Fetch

And then a filter against that data within the application code:

Filter

Riak handles a large number of keys (billions) pretty well. Things like expiration windows (which are built into Bitcask) can aid in keeping the number of keys managable too.

@hectcastro
Copy link
Author

Shout out to Dietrich Featherston and his presentation on Kobayashi at Boundary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment