Skip to content

Instantly share code, notes, and snippets.

View koljamaier's full-sized avatar

Kolja koljamaier

8 followers · 21 following
  • Hamburg, Germany
View GitHub Profile
@koljamaier
koljamaier / titanicNaiveBayes.py
Last active October 28, 2016 06:56
Naive Bayes in the Kaggle Titanic Competition
from __future__ import division
import numpy as np
from matplotlib import pyplot as plt
from scipy.stats import norm
import pandas as pd
import csv as csv
import seaborn as sns
from numpy import matrix, mat
import re
@koljamaier
koljamaier / texprocessing.py
Created October 28, 2016 07:25
Spark cluster text processing vs. single node
From this little example created by myself I learned, that it is always important to think in the distributed MapReduce paradigm when getting hands on spark.
Operations that are costly calculations in a centralized manner should be avoided on the cluster.
@koljamaier
koljamaier / dataframe.py
Created October 28, 2016 07:32
Spark DataFrame example
from pyspark import SparkContext, SparkConf
from pyspark.sql import SQLContext
from pyspark.sql.types import *
import json
import re
api_pattern = re.compile(r'(POST|HEAD|GET).*HTTP......[0-9]*.[0-9]*')
def matcher( str ):
match = api_pattern.search(str)
@koljamaier
koljamaier / moving_average_vis.py
Created July 25, 2017 10:06
Visualization of a moving average
from __future__ import division
import pylab
import numpy as np
"""
This script visualizes a trend in linear-like-shaped data
generated with random noise.
"""
# generate linear like function (with noise)
@koljamaier
koljamaier / JsonParsing.scala
Created June 10, 2018 12:04
This snippet show the usage of jacksons fasterxml library in scala for my use case
package recfun.week3
import com.fasterxml.jackson.databind.JsonNode
import scala.collection.JavaConversions._
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.databind.ObjectMapper
import scala.util.parsing.json.JSON
object JsonParsing {
@koljamaier
koljamaier / Nqueen.scala
Created June 29, 2018 22:03
Small example from Oderskys "Scala By Example" Book extended by a customized filter-approach. Find the test-file in another Gist
package nqueens
object Nqueens extends App {
println((nQueens(100) map show) mkString "\n")
def isValid(partialSolution: List[Int], col: Int, n: Int): Boolean = {
val partialSolutionCoords = (0 until partialSolution.length) zip partialSolution.reverse
val partial = partialSolutionCoords.toSet
@koljamaier
koljamaier / NqueensTest.scala
Created June 29, 2018 22:04
This is a simple Test suite for scala unit testing
package nqueens
import org.junit.runner.RunWith
import org.scalatest.FunSuite
import org.scalatest.junit.JUnitRunner
@RunWith(classOf[JUnitRunner])
class NqueensTest extends FunSuite {
// test("string take") {
@koljamaier
koljamaier / Multi-output model only evaluating as having one output column.
Created October 20, 2018 21:46 — forked from C4N4D4M4N/Multi-output model only evaluating as having one output column.
package com.IceKontroI.MODEL_TEST;
import org.deeplearning4j.nn.conf.ComputationGraphConfiguration;
import org.deeplearning4j.nn.conf.NeuralNetConfiguration;
import org.deeplearning4j.nn.conf.inputs.InputType;
import org.deeplearning4j.nn.conf.layers.DenseLayer;
import org.deeplearning4j.nn.conf.layers.OutputLayer;
import org.deeplearning4j.nn.graph.ComputationGraph;
import org.nd4j.linalg.api.buffer.DataBuffer;
import org.nd4j.linalg.api.ndarray.INDArray;