Skip to content

Instantly share code, notes, and snippets.

View jaapterwoerds's full-sized avatar

Jaap ter Woerds jaapterwoerds

  • Lambda Lab Software
  • Amsterdam area, the Netherlands
View GitHub Profile
@phobos182
phobos182 / kafka_consumer.py
Created October 22, 2012 15:07
Kafka Graphite Consumer
#!/usr/bin/env python
from optparse import OptionParser
from brod.zk import *
import pickle
import struct
import socket
import sys
import time
@markhibberd
markhibberd / characters.md
Last active January 14, 2021 18:32
Haskell Character Data

Overview

There are lots of representations for strings. In most languages they pick one set of tradeoffs and run with it. In haskell the "default" implementation (at least the one in the prelude) is a pretty bad choice, but unlike most other languages (really) good implementations exist for pretty much every way you can twist these things. This can be a good thing, but it also leads to confusion, and frustration to find the right types and how to convert them.

Types

@Fristi
Fristi / Aggregate.hs
Last active November 6, 2022 20:50
DDD/Event Sourcing in Haskell. Implemented an aggregate as a type class and type families to couple event, command and error types specific to the aggregate. Errors are returned by using Either (Error e) (Event e). Applying Applicative Functors fits here well to sequentially check if the command suffice.
{-# LANGUAGE TypeFamilies #-}
import Data.Function (on)
import Control.Applicative
data EventData e = EventData {
eventId :: Int,
body :: Event e
}
@jboner
jboner / akka-cluster-implementation-notes.md
Last active March 4, 2023 22:30
Akka Cluster Implementation Notes

Akka Cluster Implementation Notes

Slightly disorganized but reasonably complete notes on the algorithms, strategies and optimizations of the Akka Cluster implementation. Could use a lot more links and context etc., but was just written for my own understanding. Might be expanded later.

Links to papers and talks that have inspired the implementation can be found on the 10 last pages of this presentation.

Akka Gossip

Gossip state

This is the Gossip state representation:

@debasishg
debasishg / gist:8172796
Last active July 5, 2024 11:53
A collection of links for streaming algorithms and data structures

General Background and Overview

  1. Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
  2. Models and Issues in Data Stream Systems
  3. Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
  4. Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
  5. [Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep=rep1&t
@colestanfield
colestanfield / gist:fac042d3108b0c06e952
Created August 8, 2014 22:55
sbt-assembly merge strategy for aop.xml files
// Create a new MergeStrategy for aop.xml files
val aopMerge: MergeStrategy = new MergeStrategy {
val name = "aopMerge"
import scala.xml._
import scala.xml.dtd._
def apply(tempDir: File, path: String, files: Seq[File]): Either[String, Seq[(File, String)]] = {
val dt = DocType("aspectj", PublicID("-//AspectJ//DTD//EN", "http://www.eclipse.org/aspectj/dtd/aspectj.dtd"), Nil)
val file = MergeStrategy.createMergeTarget(tempDir, path)
@Fristi
Fristi / FreeActor.scala
Last active April 25, 2019 13:12
Free Monad operating on Akka actors
package nl.mdj.fpinscala
import akka.actor.{Actor, ActorRef, ActorSystem, Props}
import akka.pattern.ask
import akka.util.Timeout
import nl.mdj.fpinscala.KeyValueStore.{KvS, ListKeys}
import scala.concurrent.duration._
import scala.concurrent.{ExecutionContext, Await, Future}
import scalaz.std.list._
@awesomebytes
awesomebytes / ipython_to_file.md
Created March 16, 2016 10:38
Save iPython session to a python file as code

Save an iPython session commands/code to a file

You must use the magic method %save:

In [1]: %save?
Type:       Magic function
String Form:<bound method CodeMagics.save of <IPython.core.magics.code.CodeMagics object at 0x7fb5d25bb1d0>>
Namespace:  IPython internal
File: /usr/lib/python2.7/dist-packages/IPython/core/magics/code.py
@thbkrkr
thbkrkr / list.rb
Last active July 25, 2023 07:54
Kafka JMX MBeans list
com.sun.management:type=DiagnosticCommand
com.sun.management:type=HotSpotDiagnostic
java.lang:name=CodeCacheManager,type=MemoryManager
java.lang:name=Code Cache,type=MemoryPool
java.lang:name=Compressed Class Space,type=MemoryPool
java.lang:name=G1 Eden Space,type=MemoryPool
java.lang:name=G1 Old Generation,type=GarbageCollector
java.lang:name=G1 Old Gen,type=MemoryPool
java.lang:name=G1 Survivor Space,type=MemoryPool
java.lang:name=G1 Young Generation,type=GarbageCollector
@OneCricketeer
OneCricketeer / kafka-jmx-exporter.yml
Created March 2, 2018 05:24
Kafka JMX Patterns
# https://github.com/rama-nallamilli/kafka-prometheus-monitoring
# https://github.com/Yolean/kubernetes-kafka/blob/master/prometheus/10-metrics-config.yml
lowercaseOutputName: true
rules:
- pattern : kafka.network<type=Processor, name=IdlePercent, networkProcessor=(.+)><>Value
- pattern : kafka.network<type=RequestMetrics, name=RequestsPerSec, request=(.+)><>OneMinuteRate
- pattern : kafka.network<type=SocketServer, name=NetworkProcessorAvgIdlePercent><>Value
- pattern : kafka.server<type=ReplicaFetcherManager, name=MaxLag, clientId=(.+)><>Value
- pattern : kafka.server<type=BrokerTopicMetrics, name=(.+), topic=(.+)><>OneMinuteRate