Skip to content

Instantly share code, notes, and snippets.

View daniel-j-h's full-sized avatar

Daniel J. H. daniel-j-h

View GitHub Profile
daniel-j-h /
Created April 20, 2014 16:50
C++ Intel Threading Building Blocks palindrome example, showing reduction vs. combinable -- note: memory bound
#include <iostream>
#include <string>
#include <vector>
#include <iterator>
#include <algorithm>
#include <fstream>
#include <functional>
#include "tbb/task_scheduler_init.h"
#include "tbb/blocked_range.h"
daniel-j-h /
Created May 6, 2014 12:49
ceph/ceph @24c5ea8: -Weverything

A plea for more tooling usage


I tried to build Ceph with the Clang (3.4) compiler. The only issue preventing the build to finish was a "VLA of non-POD type" usage which I fixed here:


daniel-j-h / blocking_queue.hpp
Last active August 29, 2015 14:02
blocking queue, non-blocking queue (Sean Parent's list splice trick), monitor<T>, concurrent<T> (Herb Sutter's primitives)
#include <list>
#include <mutex>
#include <condition_variable>
#include <iterator>
template <typename T>
daniel-j-h /
Last active August 29, 2015 14:04
ceph/ceph @27f6dbb: mapping all objects to a single pg

Forcing Ceph into mapping all objects to a single PG


Recently I was wondering how Ceph (i.e. Rados in particular) maps object to Placement Groups (PGs).

It basically works like the following:

pg = hash(object)

osd = crush(pg)

daniel-j-h /
Created July 30, 2014 13:00
Preimage Attack on RADOS
#!/usr/bin/env python2
# -*- coding: utf-8 -*-
# I used this in order to insert the preimages into a local ceph cluster,
# started in my ceph dev directory via src/ Adapt paths if needed.
import sys, os
srcdir = '/home/daniel/cn-work/ceph-dev/src'
pool = 'collisions'
daniel-j-h /
Last active August 29, 2015 14:04
Ceph: assert uniform object to placement group distribution using a Chi-Squared Hypothesis Test
#!/usr/bin/env python2
# -*- coding: utf-8 -*-
import sys, json
from collections import defaultdict
from scipy.stats import chisquare
# usage: ceph pg dump_json | ./
if __name__ == '__main__':
daniel-j-h / thread_guard.hpp
Created August 5, 2014 17:37
Thread Guards: The first one takes ownership over the thread, the second one only ensures exception-safe join.
#include <thread>
#include <utility>
#include <stdexcept>
// takes ownership of the thread
class scoped_thread {
explicit scoped_thread(std::thread t_) : t(std::move(t_)) {
if(!t.joinable()) throw std::logic_error{"no thread"};
daniel-j-h / spinlock.hpp
Created August 5, 2014 17:55
Spinlock: It spins and locks!
#include <atomic>
class spinlock_mutex {
spinlock_mutex() : flag(ATOMIC_FLAG_INIT) {}
void lock() { while(flag.test_and_set(std::memory_order_acquire)); }
void unlock() { flag.clear(std::memory_order_release); }
daniel-j-h / core.hy
Last active August 29, 2015 14:12
Hy: python+clojure (see:
#!/usr/bin/env hy
(import [operator [eq]]
[itertools [chain count]]
[functools [partial reduce]])
(defn map-indexed [f coll]
(map f (count) coll))
daniel-j-h /
Last active August 29, 2015 14:14
Geospatial Analysis, KSIT workshop on 2015-2-11