Skip to content

Instantly share code, notes, and snippets.

@pauldix
Last active August 29, 2015 14:01
Show Gist options
  • Save pauldix/156c5f7a0549ef88e02d to your computer and use it in GitHub Desktop.
Save pauldix/156c5f7a0549ef88e02d to your computer and use it in GitHub Desktop.
Shard spaces, retention policies, and downsampling
// define these shard spaces
shardSpaces = [
{name: "raw", retention: "4h"},
{name: "week", retention: "7d"},
{name: "month", retention: "30d"},
{name: "year", retention: "365d"},
{name: "keep", retention: "inf"}
]
// apply these rules for mapping data to shard spaces
rules = [{
"database": "metricsDB",
"mappings": [
{"space": "week", "regex": "^week.*"},
{"space": "month", "regex": "^month.*"},
{"space": "year", "regex": "^year.*"},
{"space": "keep", "regex": "^keep.*"},
{"space": "raw", "regex": ".*"}
]
}]
// there can be only one rule set per database
// when data comes into a database, the rules will be
// evaluated in order
// now define some continuous queries for example:
// select percentile(value, 90) from raw.response_times
// group by time(5m), controller_action
// into week.response_times.percentile.90.5m
// select count(user_id), user_id as raw.event_count
// from user_events
// group by time(5m), user_id
// into week.user_events.count.5m
// select sum(event_count) as event_count, user_id
// from week.user_events.count.5m
// group by time(1h), user_id
// into month.user_events.count.1h
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment