Skip to content

Instantly share code, notes, and snippets.

View jjmalina's full-sized avatar

Jeremiah Malina jjmalina

63 followers · 133 following
View GitHub Profile
@jjmalina
jjmalina / main.go
Created December 4, 2013 15:25
make-logs
//
// generates lines of json and sends them to stdout
// usage:
// make-logs -n 10000 | write-or-delete-logs mybucket
package main
import (
"encoding/json"
"flag"
@jjmalina
jjmalina / main.go
Last active December 30, 2015 06:29
//
// insertion or deletion of arbitrary json logs to riak
// usage:
// cat logs | write-or-delete-logs --hosts=127.0.0.1,127.0.0.2 --port=8087 mybucket 2>> write_log &
// OR you can delete EVERYTHING
// write-or-delete-logs --delete --hosts=127.0.0.1,127.0.0.2 --port=8087 mybucket
package main
import (
@jjmalina
jjmalina / console.log
Created December 4, 2013 18:44
Riak console.log during crash
2013-12-03 23:40:00.823 [info] <0.489.0>@mi_scheduler:worker_loop:133 Pid <0.4398.0> compacted 20 segments for 6017383 bytes in 1.420779 seconds, 4.04 MB/sec
2013-12-03 23:40:07.463 [info] <0.489.0>@mi_scheduler:worker_loop:133 Pid <0.4987.0> compacted 12 segments for 28541685 bytes in 6.639782 seconds, 4.10 MB/sec
2013-12-03 23:40:25.105 [info] <0.489.0>@mi_scheduler:worker_loop:133 Pid <0.13536.16> compacted 9 segments for 57840707 bytes in 17.642435 seconds, 3.13 MB/sec
2013-12-03 23:40:33.330 [info] <0.489.0>@mi_scheduler:worker_loop:133 Pid <0.25160.10> compacted 13 segments for 47454983 bytes in 8.225041 seconds, 5.50 MB/sec
2013-12-03 23:40:42.206 [info] <0.489.0>@mi_scheduler:worker_loop:133 Pid <0.6205.0> compacted 15 segments for 50722448 bytes in 8.875925 seconds, 5.45 MB/sec
2013-12-03 23:40:43.327 [info] <0.489.0>@mi_scheduler:worker_loop:133 Pid <0.4593.0> compacted 20 segments for 6306302 bytes in 1.121077 seconds, 5.36 MB/sec
2013-12-03 23:40:44.504 [info] <0.489.0>@mi_scheduler:worker_loop:1
@jjmalina
jjmalina / gist:7793147
Created December 4, 2013 18:45
riakpbc crash stacktrace
2013/12/03 23:40:13 ADDED 2rKAq1snVFT978yI0mHt8XwAOod from line 2362324
2013/12/03 23:40:18 ADDED GMR5zWC2PKhG2Y6TVpdxhhLAO3X from line 2362325
2013/12/03 23:40:23 ERROR from line 2362326
2013/12/03 23:40:23 ERROR from line 2362327
panic: interface conversion: interface is *riakpbc.RpbPutResp, not []uint8
goroutine 11 [running]:
github.com/mrb/riakpbc.(*Node).Ping(0xc200083730, 0xc20008c4b0)
/home/jeremiah/code/go/src/github.com/mrb/riakpbc/node.go:153 +0xde
github.com/mrb/riakpbc.(*Pool).Ping(0xc20008c510)
@jjmalina
jjmalina / gist:8231869
Created January 3, 2014 03:00
WTF
21:57 jmalina in code
λ mkvirtualenv temp
New python executable in temp/bin/python
Installing setuptools, pip...
Complete output from command /Users/jmalina/code/...envs/temp/bin/python -c "import sys, pip; pip...ll\"] + sys.argv[1:])" setuptools pip:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/virtualenv-1.11-py2.7.egg/virtualenv_support/pip-1.5-py2.py3-none-any.whl/pip/__init__.py", line 9, in <module>
File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/virtualenv-1.11-py2.7.egg/virtualenv_support/pip-1.5-py2.py3-none-any.whl/pip/log.py", line 8, in <module>
File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/virtualenv-1.11-py2.7.egg/virtualenv_support/setuptools-2.0.2-py2.py3-none-any.whl/pkg_resources.py", line 2696, in <module>
@jjmalina
jjmalina / enable_yz_search.sh
Created January 16, 2014 14:53
Installing Yokozuna search
RIAK_ADMIN=${RIAK_ADMIN:-riak-admin}
echo Bucket type is ${TYPE:=events}
echo Index is ${INDEX:=$TYPE}
echo Schema name ${SCHEMA:=$INDEX}
echo Schema from ${SCHEMA_SOURCE:=${SCHEMA}_schema.xml}
curl -X PUT -i "http://localhost:8098/search/schema/$SCHEMA" -H "content-type:application/xml" -d @$SCHEMA_SOURCE
curl -X PUT -i "http://localhost:8098/search/index/$INDEX" -H "content-type:application/json" -d "{\"schema\":\"$SCHEMA\"}"
$RIAK_ADMIN bucket-type create $TYPE "{\"props\":{\"n_val\":1,\"last_write_wins\":true,\"search_index\":\"$INDEX\"}}"
$RIAK_ADMIN bucket-type activate $TYPE
@jjmalina
jjmalina / gist:9492428
Last active August 29, 2015 13:57
Chef workflow
# modify the cookbook
knife spork bump <cookbook>
knife cookbook upload <cookbook>
knife spork promote <environment> <cookbook>
knife environment from file <environment>.json
@jjmalina
jjmalina / format_timedelta.py
Created April 23, 2014 20:00
Format a timedelta into the largest possible unit (weeks is the largest in this case)
def format_timedelta(td):
"""formats a timedelta into the largest unit possible
>>> from datetime import timedelta
>>> format_timedelta(timedelta(weeks=2))
'2w'
>>> format_timedelta(timedelta(weeks=2, minutes=2))
'20162m'
>>> format_timedelta(timedelta(days=2))
@jjmalina
jjmalina / datetime_to_epoch.py
Last active August 29, 2015 14:08
Because I keep forgetting this...
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
datetime_to_epoch
~~~~~~~~~~~~~~~~~
Takes a (naive) UTC datetime and converts to epoch
"""
from calendar import timegm
@jjmalina
jjmalina / coinbase.py
Created July 18, 2015 21:35
Real-time Coinbase orders and trades
# -*- coding: utf-8 -*-
"""
coinbase
~~~~~~~~
Real time Coinbase trades and orders using their Websocket Feed
https://docs.exchange.coinbase.com/?python#overview
"""
import asyncio