Skip to content

Instantly share code, notes, and snippets.

View EikeDehling's full-sized avatar

Eike Dehling EikeDehling

  • Textkernel
  • Amsterdam
View GitHub Profile
@EikeDehling
EikeDehling / customer-template.json
Created March 22, 2019 14:17
Interactive demo of template customizations
{
"import": {
"1.5.0": "default-vac-v1.json",
"2.0.0": "default-vac-v2.json"
},
"search_fields": [
{
"name": "status",
"target": "status",
@EikeDehling
EikeDehling / install-kibana.sh
Created September 14, 2017 11:27
Kibana install script debian (as root)
# Install elasticsearch
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list
apt-get update
apt-get install kibana
# Configure elastic
sed -i '/#server.host: 192.168.0.1/c\server.host: 0.0.0.0' /etc/kibana/kibana.yml
@EikeDehling
EikeDehling / install-beats.sh
Created September 14, 2017 11:00
Install filebeat & metricbeat on debian (as root)
# Install filebeat & metricbeat
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-5.x.list
apt-get update
apt-get install filebeat metricbeat
# Configure filebeat
cat >/etc/filebeat/filebeat.yml <<EOL
filebeat.prospectors:
@EikeDehling
EikeDehling / install-elastic-5.6-debian.sh
Last active September 14, 2017 09:45
Shell script to install elastic 5.6 (as root) on debian
# Install elasticsearch
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list
apt-get update
apt-get install elasticsearch
# Configure memory settings
mkdir -p /etc/systemd/system/elasticsearch.service.d
echo -e "[Service]\nLimitMEMLOCK=infinity" > /etc/systemd/system/elasticsearch.service.d/elasticsearch.conf
@EikeDehling
EikeDehling / abusehub-reindex.py
Created September 8, 2017 08:33
Python script to reindex elasticsearch data to monthly indices
#!bin/python
from elasticsearch import Elasticsearch
from datetime import datetime
import time
es = Elasticsearch()
indices_state = es.cluster.state()['metadata']['indices']
@EikeDehling
EikeDehling / titanic.py
Last active March 11, 2017 11:42
Some experiments for kaggle titanic survivors machine learning competition (https://www.kaggle.com/c/titanic)
import pandas
from sklearn import linear_model, svm, tree, naive_bayes
from sklearn.model_selection import cross_val_score
import numpy as np
data = pandas.read_csv('train.csv')
def preprocess(data):
data['Fare'] = data['Fare'].fillna(data['Fare'].mean())
@EikeDehling
EikeDehling / random-apache-log.cmd
Last active March 10, 2017 15:12
Windows batch file for generating a random apache log
@echo off
Setlocal EnableDelayedExpansion
for /L %%n in (1,0,5) do (
SET /A N1=!RANDOM! * 255 / 32768
SET /A N2=!RANDOM! * 255 / 32768
SET /A N3=!RANDOM! * 255 / 32768
SET /A N4=!RANDOM! * 255 / 32768
@EikeDehling
EikeDehling / filebeat.yml
Created February 3, 2017 13:39
Basic filebeat config
filebeat.prospectors:
- input_type: log
paths:
- ./random_apache_log
output.elasticsearch:
hosts: ["127.0.0.1:9200"]
@EikeDehling
EikeDehling / random-apache-log.sh
Last active February 17, 2017 12:51
Bash script to generate a random (apache) log line every random seconds
#!/usr/bin/env bash
while true
do
random_ip=$(dd if=/dev/urandom bs=4 count=1 2>/dev/null | od -An -tu1 | sed -e 's/^ *//' -e 's/ */./g')
random_size=$(( (RANDOM % 65535) + 1 ))
current_date_time=$(date '+%d/%b/%Y:%H:%M:%S %z')
echo "$random_ip - - [$current_date_time] \"GET /data.php HTTP/1.1\" 200 $random_size" | tee -a 'random_log'
@EikeDehling
EikeDehling / install-elasticsearch.sh
Last active May 10, 2017 07:24 — forked from gourneau/ElasticSearch.sh
Script to install elasticsearch (Latest 5.x) on a ubuntu machine (16.04LTS)
# Install latest OpenJDK
sudo apt-get update
sudo apt-get install openjdk-8-jre-headless
# Install elastic
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list
sudo apt-get update
sudo apt-get install elasticsearch