Skip to content

Instantly share code, notes, and snippets.

View jorisdevrede's full-sized avatar

Joris de Vrede jorisdevrede

  • The Netherlands
View GitHub Profile
@jorisdevrede
jorisdevrede / server.py
Created April 29, 2018 12:45
HTTP Server
from argparse import ArgumentParser
from configparser import ConfigParser
from http.server import BaseHTTPRequestHandler
from socketserver import TCPServer
import time
class MyHandler(BaseHTTPRequestHandler):
def do_GET(self):
@jorisdevrede
jorisdevrede / spool.yml
Created January 11, 2018 22:05
ansible spooler
---
- name: Reboot
hosts: all
become: yes
tasks:
- name: reboot system
@jorisdevrede
jorisdevrede / log.py
Last active January 10, 2018 15:29
python logger
#!/usr/bin/env python
import argparse
import logging
def main():
logging.debug('main')
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--log',
@jorisdevrede
jorisdevrede / hlogreader.py
Created December 21, 2017 13:54
log extraction
#!/usr/bin/env python
from __future__ import print_function
import argparse
import re
import os
class HlogReader:
type_principal = 1
@jorisdevrede
jorisdevrede / install.sh
Last active December 6, 2017 20:39
development client installation
sudo apt-get install -y software-properties-common
sudo apt-add-repository -y ppa:ansible/ansible
sudo apt-get update
sudo apt-get install -y ansible
echo 'set nocompatible' > ~/.vimrc
sudo apt-get install -y maven
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
@jorisdevrede
jorisdevrede / config.md
Created December 4, 2017 13:13
Development setup

Java

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
sudo apt-get install oracle-java8-set-default

Java Cryptography Extension (JCE) Unlimited Strength

To utilize 256 bits keys in Java, the following extensions need to be installed: Unlimited Strength Jurisdiction Policy Files (Java 8)

@jorisdevrede
jorisdevrede / connect.py
Last active November 16, 2017 15:36
Elasticsearch optimization
from argparse import ArgumentParser
from logbook import Logger, StreamHandler
import json
import requests
import sys
import uuid
StreamHandler(sys.stdout).push_application()
log = Logger(__name__)
from pyspark import SparkConf, SparkContext
from operator import add
def explode_coords(record):
"""Restructure record into multi (coord, value) tuples
Takes a list of [x-coord, y-coord, value] and explodes that
into inverted tuples with a key (x,y) and a value v for all
@jorisdevrede
jorisdevrede / instructions.md
Last active June 6, 2017 05:34
Spark ML workshop

Import relevant Spark classes

Spark ML and SQL classes are considered optional and therefore not loaded by default. You will need to load these before proceding with ML.

from pyspark.ml import Pipeline
from pyspark.ml.feature import *
from pyspark.ml.classification import *
from pyspark.ml.evaluation import *

Kopieer mij:

diamonds = sqlContext.sql("SELECT * FROM diamonds")
display(diamonds.select("*"))

Kopieer mij:

diamonds = sqlContext.sql("SELECT * FROM diamonds")