Skip to content

Instantly share code, notes, and snippets.

View mkocikowski's full-sized avatar

Mik Kocikowski mkocikowski

View GitHub Profile
@mkocikowski
mkocikowski / gist:9283114
Created March 1, 2014 01:00
Take str or unicode in, return utf-8 encoded string, with 'bad' characters stripped
def utf8(data):
"""Takes in str or unicode, returns utf-8 string, stripping invalid chars."""
if type(data) not in [str, unicode]:
raise TypeError("'data' must be str or unicode")
try:
if type(data) is unicode:
s = data.encode('utf-8', 'ignore')
elif type(data) is str:
@mkocikowski
mkocikowski / gist:59971a1bcf207628134c
Created May 9, 2014 03:19
.bash_profile for maven and jdk 7 mavericks
export M2_HOME=/usr/local/share/apache-maven-3.2.1
export PATH=$PATH:$M2_HOME/bin
export JAVA_HOME=`/usr/libexec/java_home -v 1.7`
$ foo='() { echo not patched; }' bash -c foo
bash: foo: command not found
w00t! Not vulnerable.
@mkocikowski
mkocikowski / gist:8fd2728a9c66e5bb9aa2
Created June 3, 2015 14:30
Iterable chunker with no batch size limits
def chunker(iterable=None, chunklen=None):
"""Collects data into fixed-length chunks.
Returns: iterator of iterators. Does not pad the last chunk.
Raises: TypeError on bad iterable or chunklen
Example: chunker("abcde", 2) returns (('a', 'b'), ('c', 'd'), ('e'))
This is better than the functools 'grouper' recipe (using izip on
[iter(iterable)] * n) in that there is no performance penalty for
@mkocikowski
mkocikowski / gist:37114a5232bd3fa2d145
Last active August 29, 2015 14:23
Delete all closed indexes from Elasticsearch cluster
curl -s 'localhost:9200/_cat/indices' | grep close | awk '{ printf "http://localhost:9200/%s\n", $2 }' | xargs -n 1 curl -i -XDELETE
@mkocikowski
mkocikowski / cluster-health.sh
Last active November 10, 2015 21:10
Check elasticsearch cluster status
curl -XGET 'http://localhost:9200/_cluster/health?pretty=true' -s \
| grep 'status' \
| test 'green' == `awk -F '"' '{ print $4 }'` ;
@mkocikowski
mkocikowski / gist:8325452
Last active January 2, 2016 15:48
esbench 100gb macbookpro test dump
curl -XPUT 'http://localhost:9200/esbench_stats/bench/351b0aa5' -d '{"cluster": {"cluster_name": "mik_local", "node_count": 1, "ok": true, "nodes": {"eQDp6TjSQ8C-9_5ktiyLDA": {"thread_pool": {"index": {"max": 8, "queue_size": "200", "type": "fixed", "min": 8}, "search": {"max": 24, "queue_size": "1k", "type": "fixed", "min": 24}, "management": {"max": 5, "keep_alive": "5m", "type": "scaling", "min": 1}, "get": {"max": 8, "queue_size": "1k", "type": "fixed", "min": 8}, "generic": {"keep_alive": "30s", "type": "cached"}, "suggest": {"max": 8, "queue_size": "1k", "type": "fixed", "min": 8}, "refresh": {"max": 4, "keep_alive": "5m", "type": "scaling", "min": 1}, "bulk": {"max": 8, "queue_size": "50", "type": "fixed", "min": 8}, "merge": {"max": 4, "keep_alive": "5m", "type": "scaling", "min": 1}, "snapshot": {"max": 4, "keep_alive": "5m", "type": "scaling", "min": 1}, "percolate": {"max": 8, "queue_size": "1k", "type": "fixed", "min": 8}, "flush": {"max": 4, "keep_alive": "5m", "type": "scaling", "min": 1}, "warm
@mkocikowski
mkocikowski / ip.sh
Last active February 9, 2016 03:46
get IP of eth1 on Vagrant private network (DHCP)
vagrant ssh -c 'ifconfig eth1' | awk '/inet addr/{print substr($2,6)}'
@mkocikowski
mkocikowski / mem.py
Created May 13, 2016 20:27
Check amount of RAM used by current python process
#!/usr/bin/python
import os
# pip install psutil
import psutil
mb = psutil.Process(os.getpid()).memory_info()[0]>>20
print(mb)
@mkocikowski
mkocikowski / ca.cnf
Last active May 19, 2016 19:07
openssl CA configuration file
[ ca ]
default_ca = ca_default
[ ca_default ]
base_dir = $ENV::HOME/.ssh
certificate = $base_dir/datapipe-ca.crt
copy_extensions = copy
database = $base_dir/index.txt
default_days = 365
default_md = sha256