Skip to content

Instantly share code, notes, and snippets.

View obohrer's full-sized avatar

Olivier Bohrer obohrer

View GitHub Profile
@pakt
pakt / rdwr.py
Created August 15, 2015 10:59
Direct read/write access to Python's memory
#
# read/write access to python's memory, using a custom bytearray.
# some code taken from: http://tinyurl.com/q7duzxj
#
# tested on:
# Python 2.7.10, ubuntu 32bit
# Python 2.7.8, win32
#
# example of correct output:
# inspecting int=0x41424344, at 0x0228f898
; compilers are cool
(when-let [missing-methods (->> possible-dispatch-values
(map #(when-not (get-method multimethod %) %))
(filter identity)
seq)]
(throw (ex-info "Missing methods for `multimethod`" {:dispatch-vals missing-methods})))
@rbranson
rbranson / gist:038afa9ad7af3693efd0
Last active September 29, 2016 17:44
Disaggregated Proxy & Storage Nodes

The point of this is to use cheap machines with small/slow storage to coordinate client requests while dedicating the machines with the big and fast storage to doing what they do best. I found that request coordination was contributing to about half the CPU usage on our Cassandra nodes, on average. Solid state storage is quite expensive, nearly doubling the cost of typical hardware. It also means that if people have control over hardware placement within the network, they can place proxy nodes closer to the client without impacting their storage footprint or fault tolerance characteristics.

This is accomplished in Cassandra by passing the -Dcassandra.join_ring=false option when the process is started. These nodes will connect to the seeds, cache the gossip data, load the schema, and begin listening for client requests. Messages like "/x.x.x.x is now UP!" will appear on the other nodes.

There are also some more practical benefits to this. Handling client requests caused us to push the NewSize of the heap up

@kristopherjohnson
kristopherjohnson / pipe-forward.swift
Last active March 29, 2024 19:44
Swift: define F#-style pipe-forward (|>) operator that evaluates from left to right.
// F#'s "pipe-forward" |> operator
//
// Also "Optional-chaining" operators |>! and |>&
//
// And adapters for standard library map/filter/sorted
infix operator |> { precedence 50 associativity left }
infix operator |>! { precedence 50 associativity left }
infix operator |>& { precedence 50 associativity left }
infix operator |>* { precedence 50 associativity left }
@jkreps
jkreps / benchmark-commands.txt
Last active June 17, 2024 03:54
Kafka Benchmark Commands
Producer
Setup
bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test-rep-one --partitions 6 --replication-factor 1
bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test --partitions 6 --replication-factor 3
Single thread, no replication
bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance test7 50000000 100 -1 acks=1 bootstrap.servers=esv4-hcl198.grid.linkedin.com:9092 buffer.memory=67108864 batch.size=8196
@justenwalker
justenwalker / unsafeAssassinateEndpoint.md
Created January 9, 2014 17:33
Cassandra unsafe assassinate - removing a dead node from the cluster

Unsafe Node Assassinate

On Cassandra 1.1.x, nodetool removetoken on a dead node hangs. The only way to reliably remove the node from the cluster is to use an undocumented JMX command to force the remove of the node from gossip.

Upgrading to 1.2.x and above should obviate these steps in favor of nodetool removenode - See 1.2 Docs

It's probably also a good idea to run a nodetool repair after this operation - See wiki

Prerequisites

@puredanger
puredanger / gist:6250711
Created August 16, 2013 15:05
bash replacement for rm to prevent shooting yourself in the face.
function rm () {
local path
for path in "$@"; do
# ignore any arguments
if [[ "$path" = -* ]]; then :
else
local dst=${path##*/}
# append the time if necessary
while [ -e ~/.Trash/"$dst" ]; do
dst="$dst "$(date +%H-%M-%S)
@ryankearney
ryankearney / ComcastInject.html
Last active June 10, 2023 14:40
This is the code Comcast is injecting into its users web traffic.
<script language="JavaScript" type="text/javascript">
// Comcast Cable Communications, LLC Proprietary. Copyright 2012.
// Intended use is to display browser notifications for critical and time sensitive alerts.
var SYS_URL='/e8f6b078-0f35-11de-85c5-efc5ef23aa1f/aupm/notify.do';
// var image_url='http://servicealerts.comcast.net:8080/images/mt';
var image_url='http://xfinity.comcast.net/constantguard/BotAssistance/notice/images';
var headertext1='<strong>Comcast Courtesy Notice</strong>';
var textline1='You have reached 90% of your <b>monthly data usage allowance</b>.';
var textline2='Please sign in for more information and to remove this alert.';
var acknowledgebutton='<a href=\"#\" onClick="document.location.href=\''+SYS_URL+'?dispatch=redirect&redirectName=login&paramName=bmUid\'" title="Sign in to acknowledge" style="color: #FFFFFF;"><img alt="Sign in to acknowledge" src="'+image_url+'/mt_signin.png"/></a>';
@adeel
adeel / push-notifications.clj
Created July 26, 2011 01:03
Example of sending Apple push notifications with Clojure.
; Depends on [com.notnoop.apns/apns "0.1.6"].
(:import ('com.notnoop.apns APNS))
(defn send-push-notification [device-token message]
(let [service (.build (.withSandboxDestination
(.withCert (APNS/newService) "resources/apns-dev-cert.p12" "password")))
payload (.build (.alertBody (APNS/newPayload) message))]
(.push service device-token payload)))
@rcampbell
rcampbell / s3.clj
Created May 11, 2011 10:14
Storing and retrieving Clojure data structures as GZIP compressed JSON in Amazon S3
(ns aws.s3
(:refer-clojure :exclude [get])
(:use [clojure.walk :only (keywordize-keys stringify-keys)]
[clojure.contrib.def :only (defonce-)]
[clojure.contrib.json :only (read-json write-json)])
(:import [java.io PrintWriter InputStreamReader ByteArrayInputStream ByteArrayOutputStream]
[java.util.zip GZIPInputStream GZIPOutputStream]
[com.google.common.base Charsets]
[com.amazonaws.services.s3 AmazonS3Client]
[com.amazonaws.services.s3.model Region CreateBucketRequest ObjectMetadata