Skip to content

Instantly share code, notes, and snippets.

View RobinUS2's full-sized avatar

Robin Verlangen RobinUS2

18 followers · 12 following
View GitHub Profile
@RobinUS2
RobinUS2 / cleanup_tables.sh
Created December 1, 2015 16:04
Cleanup cassandra tables one by one, starting with the smallest
#!/bin/bash
# This will work in case you have very limited disk space, it will also show more digestable progress compared to nodetool compactionstats
DATA_FOLDER="/var/lib/cassandra/data"
CMD="cleanup"
cd $DATA_FOLDER
# list keyspaces from small to large
KEYSPACES=`du -s * | sort -n | awk '{print $2}'`
# iterate keyspaes
@RobinUS2
RobinUS2 / commands.sh
Created December 1, 2015 13:16
Force cassandra node to join ring after bootstrap failures
#!/bin/bash
# In case a join keeps failing to join the ring but you really want to force it (use with caution), you can do the following
# 1: Shut down node
service cassandra stop
# 2: Backup data
mkdir -p /backup/cassandra/data/system/local
cp -r /var/lib/cassandra/data/system/local/* /backup/cassandra/data/system/local
@RobinUS2
RobinUS2 / list.sh
Created December 1, 2015 08:46
List corrupt sstables / files in Cassandra 2.0.x based on log analysis
sudo grep 'Corrupt' /var/log/cassandra/system.log | grep var | awk '{print \$4}' | grep -v corrupt | cut -c 2- | rev | cut -c 3- | rev | grep -v apache | grep -E "^/" | sort | uniq
@RobinUS2
RobinUS2 / body_reader.go
Created November 17, 2015 09:18
Read Go body bytes with GZIP support
package main
// License: Apache v2
import (
"bytes"
"compress/gzip"
"io/ioutil"
"net/http"
)
@RobinUS2
RobinUS2 / bench-gzip-golang
Created May 9, 2014 17:45
*** NO GZIP
[root@cl025 ~]# ab -n 50000 -c 150 "http://godev.flx1.com/geo"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking godev.flx1.com (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
@RobinUS2
RobinUS2 / cloudpelican-aide-security
Last active August 29, 2015 13:57
CloudPelican AIDE rules
# CloudPelican rules for AIDE ( http://aide.sourceforge.net/ )
# Typically found in /etc/aide.conf
# @author Robin Verlangen
!/usr/local/cloudpelican/queues/.* # ignore queue contents as they change rapidly
!/usr/local/cloudpelican/.filepointers.json$ # ignore filepointers metadata as they change rapidly
!/usr/local/cloudpelican/.initstate$ # ignore init state as this changes every 24 hours
@RobinUS2
RobinUS2 / output-transparent-pixel.go
Created November 25, 2013 11:23
Output a 1x1 transparent gif pixel in Go webserver response. Please note, this is only the relevant code, does not work "out-of-the-box".
import (
"fmt"
"net/http"
"time"
)
const transPixel = "\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x21\xF9\x04\x01\x00\x00\x00\x00\x2C\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02\x44\x01\x00\x3B"
func pixelHandler(w http.ResponseWriter, r *http.Request) {
// Pixel
@RobinUS2
RobinUS2 / anomalies-v2
Created May 1, 2013 14:22
Anomaly detection version 2, better supports recurring events (e.g. hourly spikes)
*** Statistics ***
Count: 100
Max: 107
Min: 102
Avg: 104.0
Variance: 2.46
Standard deviation: 1.5684387141358123
Median: 105.0
Actual: 105 00.00% 106 00.00% 102 00.00% 105 00.00% 107 00.00%
Forecast: 104 00.96% 104 01.90% 104 01.94% 104 00.96% 104 02.84% (MLR15, performance 00.00%)
@RobinUS2
RobinUS2 / gist:5478185
Created April 28, 2013 19:57
Anomaly detection system based on forecasting
run:
*** Statistics ***
Count: 100
Max: 107
Min: 102
Avg: 104.0
Variance: 2.52
Standard deviation: 1.5874507866387544
Median: 104.5
Actual: 104 00.00% 106 00.00% 105 00.00% 107 00.00% 107 00.00%
@RobinUS2
RobinUS2 / hiveserver-check.sh
Created January 9, 2013 09:11
Hive server availability check
#!/bin/bash
STATE=`ps aux | grep "HiveServer" | wc -l`
IP="127.0.0.1"
PORT=10000
function do_restart()
{
echo Restarting
cd /
nohup hive --service hiveserver &
echo $! > /var/run/hiveserver.pid