Skip to content

Instantly share code, notes, and snippets.

Avatar

Paul Dix pauldix

View GitHub Profile
View influx_enterprise_demo.go
package main
import (
"bufio"
"flag"
"fmt"
"github.com/influxdata/influxdb/client"
"math/rand"
"net/url"
"os"
@pauldix
pauldix / influx_test.sh
Created Feb 25, 2015
InfluxDB test script
View influx_test.sh
printf "creating database\n"
curl -G http://localhost:8086/query --data-urlencode "q=CREATE DATABASE mydb"
printf "creating retention policy\n"
curl -G http://localhost:8086/query --data-urlencode "q=CREATE RETENTION POLICY myrp ON mydb DURATION 365d REPLICATION 1 DEFAULT"
printf "inserting data\n"
curl -d '{"database" : "mydb", "retentionPolicy" : "myrp", "points": [{"name":"cpu","tags":{"region":"useast","host":"server_1","service":"redis"},"fields":{"value":61}},{"name":"cpu","tags":{"region":"useast","host":"server_2","service":"mysql"},"fields":{"value":53}},{"name":"cpu","tags":{"region":"useast","host":"server_3","service":"apache"},"fields":{"value":80}},{"name":"cpu","tags":{"region":"useast","host":"server_4","service":"nginx"},"fields":{"value":86}},{"name":"cpu","tags":{"region":"useast","host":"server_5","service":"paulapp"},"fields":{"value":87}},{"name":"cpu","tags":{"region":"japan","host":"server_6","service":"redis"},"fields":{"value":10}},{"name":"cpu","tags":{"region":"japan","host":"s
@pauldix
pauldix / gist:156c5f7a0549ef88e02d
Last active Aug 29, 2015
Shard spaces, retention policies, and downsampling
View gist:156c5f7a0549ef88e02d
// define these shard spaces
shardSpaces = [
{name: "raw", retention: "4h"},
{name: "week", retention: "7d"},
{name: "month", retention: "30d"},
{name: "year", retention: "365d"},
{name: "keep", retention: "inf"}
]
// apply these rules for mapping data to shard spaces
@pauldix
pauldix / gist:4068851
Created Nov 13, 2012
Using Errplane metrics to set up custom thresholding on exception notifications
View gist:4068851
Errplane.configure do |config|
config.define_custom_exception_data do |black_box|
if black_box.exception.class == ZeroDivisionError
Errplane.report("important_exceptions/#{black_box.exception.class.to_s}")
end
# or just report on all of them and sort it out in the errplane web app
Errplane.report("exception_classes/#{black_box.exception.class.to_s}",
:message => "{Rails.env} - #{black_box.exception.class.to_s} - #{black_box.request_data.to_json}")
@pauldix
pauldix / gist:2830675
Created May 29, 2012
ideas for fetching
View gist:2830675
# set the thread pool size and the timeout in seconds.
fetcher = Feedzirra::Fetcher.new({:thread_pool_size => 100, :timeout => 5})
# some feed data objects. also include feed_entry data objects
feeds = [Feed.new({:entries => [], etag => "..", :last_modified => "...", :url => "...", :feed_url => "...", :title => ""})]
# async style
fetcher.get(feeds, :on_success => lambda {|feed, updated_feed| ...}, :on_failure => lambda {|feed, failure_object| ...})
# that returns before finishing fetching the feeds. just adds them to a thread-safe queue to be processed by a worker pool.
# the failure condition could actually call fetcher.get on the failed feed again if you wanted to retry.
@pauldix
pauldix / gist:1081387
Created Jul 13, 2011
Y U NO WORK?!!!!
View gist:1081387
# here's the dealer socket in one process
require 'ffi-rzmq'
@context = ZMQ::Context.new(1)
@sock = @context.socket(ZMQ::XREQ)
@sock.bind("tcp://*:5550")
count = 0
while true do
count += 1
View gist:1027409
Keyspace: system
Read Count: 11
Read Latency: 1.8718181818181818 ms.
Write Count: 3
Write Latency: 0.021 ms.
Pending Tasks: 0
Column Family: NodeIdInfo
SSTable count: 0
Space used (live): 0
Space used (total): 0
@pauldix
pauldix / gist:1027367
Created Jun 15, 2011
brisk mapred-site.xml
View gist:1027367
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<!-- Auto detect the brisk job tracker -->
<property>
<name>mapred.job.tracker</name>
@pauldix
pauldix / gist:1027361
Created Jun 15, 2011
brisk log - GC thrashing
View gist:1027361
INFO [ScheduledTasks:1] 2011-06-15 15:35:40,474 GCInspector.java (line 128) GC for ParNew: 241 ms, 95072 reclaimed leaving 3569921120 used; max is 10456399872
INFO [IPC Server handler 2 on 41734] 2011-06-15 15:35:40,971 TaskTracker.java (line 2428) attempt_201106151514_0001_r_000000_0 0.11111112% reduce > copy (1 of 3 at 1.03 MB/s) >
INFO [ScheduledTasks:1] 2011-06-15 15:35:41,475 GCInspector.java (line 128) GC for ParNew: 216 ms, 8995440 reclaimed leaving 5737696856 used; max is 10456399872
INFO [ScheduledTasks:1] 2011-06-15 15:35:42,584 GCInspector.java (line 128) GC for ParNew: 280 ms, 48616 reclaimed leaving 7905426312 used; max is 10456399872
INFO [IPC Server handler 1 on 41734] 2011-06-15 15:35:42,778 TaskTracker.java (line 2428) attempt_201106151514_0001_r_000003_0 0.11111112% reduce > copy (1 of 3 at 0.00 MB/s) >
INFO [ScheduledTasks:1] 2011-06-15 15:35:43,585 GCInspector.java (line 128) GC for ConcurrentMarkSweep: 610 ms, 2167917104 reclaimed leaving 2486844944 used; max is 10456399872
INFO [
@pauldix
pauldix / gist:981916
Created May 19, 2011
Redis SORT command examples
View gist:981916
# Optimized for writes, sort on read
# LVC
redis.hset("bonds|1", "bid_price", 96.01)
redis.hset("bonds|1", "ask_price", 97.53)
redis.hset("bonds|2", "bid_price", 95.50)
redis.hset("bonds|2", "ask_price", 98.25)
redis.sadd("bond_ids", 1)
redis.sadd("bond_ids", 2)