Skip to content

Instantly share code, notes, and snippets.

View tnachen's full-sized avatar

Timothy Chen tnachen

210 followers · 19 following
View GitHub Profile
@tnachen
tnachen / Distributed systems meta-list.md
Created November 3, 2013 07:47 — forked from macintux/Distributed systems meta-list.md
@tnachen
tnachen / gist:366efb2ae2ea3f5f2cfd
Created July 12, 2014 07:32
Drill Window Phase 2
Drill Window Notes:
New Function Scope for Window queries
Function rename in DrillOptiq for window functions, add "_win" perhaps
Let's focus on these window functions:
---------------------------------------
lead
rank
lag
percentile
@tnachen
tnachen / gist:cac461f0285acc5eefd5
Created September 2, 2014 22:38
sysinfo
#include <stdio.h>
#include <stdlib.h>
#include <iostream>
#include <sys/sysinfo.h>
#include <errno.h>
int main(int argc, char** argv)
{
struct sysinfo info;
if (sysinfo(&info) != 0) {
@tnachen
tnachen / gist:5aab48f2437786530f49
Created September 3, 2014 19:12
FROM stackbrew/ubuntu:13.10
MAINTAINER Mesosphere support@mesosphere.io
RUN ["env", "DEBIAN_FRONTEND=noninteractive", "apt-get", "update"]
RUN ["env", "DEBIAN_FRONTEND=noninteractive", "apt-get", "install", "-y", "--fix-missing", "--force-yes", "python2.7", "libsasl2-2", "libcurl3", "curl", "wget", "unzip"]
RUN ["env", "DEBIAN_FRONTEND=noninteractive", "apt-get", "install", "-y", "--fix-missing", "--force-yes", "python", "python-setuptools", "python-pip", "python-protobuf"]
RUN ["env", "DEBIAN_FRONTEND=noninteractive", "apt-get", "install", "-y", "--fix-missing", "--force-yes", "default-jre-headless"]
ADD http://downloads.mesosphere.io/master/ubuntu/13.10/mesos_0.19.0-xcon2_amd64.deb /tmp/mesos.deb
RUN ["env", "DEBIAN_FRONTEND=noninteractive", "dpkg", "-i", "/tmp/mesos.deb"]
@tnachen
tnachen / gist:1cedbd760e44f4f423b4
Created September 23, 2014 19:01
Docker talk + bio
Bio
Timothy Chen is a Apache Mesos PMC and committer, and lead the development for the Docker on Mesos integration at Mesosphere. He's also a Apache Drill committer and contributed to Apache Kafka, Spark and Hive. His previous work experiences includes PaaS CloudFoundry at VMWare and Cloud services for the game industry including the Halo series.
Abstract
"Scale your Docker containers with Mesos"
Docker provided an awesome way for developers to isolate and ship dependencies of their production applications, and there is a vibrant community that provides lots of different images. However, to deploy and scale these docker containers as distributed systems still remains a challenge for the users. Apache Mesos is a resource management system that provides the primitives to scale and operate distributed systems. With the latest release Docker is integrated into Mesos and in this talk I'll talk about the motivation and architecture of the integration, and how to scale Docker containers and tap into the rich fe
@tnachen
tnachen / gist:50f3f376af9a0c158aa8
Created October 6, 2014 18:29
SVN Diff benchmark
struct Size {
int diff;
int orig;
Duration duration;
};
const int STEP = 16 * 1024;
static void benchDiff(int size, double diffPercent, bool silent)
{
@tnachen
tnachen / gist:8a33ff23f260d5ab479f
Created November 12, 2014 22:02
FutureDispatch bug
Matching: )^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@, Event method: )^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
Matched!
Update Future finished in test.
Method: )^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
0 libmesos-0.22.0.dylib 0x000000010d9cf42e _ZN7process8internal8dispatchERKNS_4UPIDERKNSt3__110shared_ptrINS4_8functionIFvPNS_11ProcessBaseEEEEEERKNS4_12basic_stringIcNS4_11char_traitsIcE\
ENS4_9allocatorIcEEEE + 238
1 libmesos-0.22.0.dylib 0x000000010cefa65f _ZN7process8dispatchIN5mesos8internal5slave10LimitationENS3_15IsolatorProcessERKNS1_11ContainerIDES6_EENS_6FutureIT_EERKNS_3PIDIT0_EEMSD_FSB_T1_\
@tnachen
tnachen / gist:e97cb242d31fb2f281e4
Created January 13, 2015 19:20
https://reviews.apache.org/r/29809/
diff --git a/src/launcher/fetcher.cpp b/src/launcher/fetcher.cpp
index 4bda91c..bcd90ae 100644
--- a/src/launcher/fetcher.cpp
+++ b/src/launcher/fetcher.cpp
@@ -89,8 +89,7 @@ Try<bool> extract(const string& filename, const string& directory)
// Attempt to get the uri using the hadoop client.
Try<string> downloadWithHadoopClient(
const string& uri,
- const string& directory,
- const string& basename)
@tnachen
tnachen / gist:15fd6131b9751e9393c2
Created June 27, 2015 00:43
docker launch failed
I0627 00:04:39.045922 24200 slave.cpp:620] Successfully attached file '/tmp/mesos/slaves/20150623-193734-918679562-5050-12782-S22/frameworks/20150623-193734-918679562-5050-12782-004│·····················
9/executors/test-task/runs/08285ac6-ad2f-42e8-8edd-0a80b8a9e828' │·····················
I0627 00:04:39.052626 24208 store.cpp:200] Stored layer with hash: 511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158 │·····················
I0627 00:04:39.053170 24208 store.cpp:200] Stored layer with hash: df7546f9f060a2268024c8a230d8639878585defcc1bc6f79d2728a13957871b │·····················
I0627 00:04:39.053649 24208 store.cpp:200] Stored layer with hash: ea13149945cb6b1e746bf28032f02e9b5a793523481a0a18645fc77ad53c4ea2 │·····················
I062
@tnachen
tnachen / gist:105b81d05b33feb07224
Created July 30, 2015 23:51
sample traces
53a5f78a-f0f3-4273-8ba1-f29ac11341b5,mesos.internal.log.PromiseRequest,ff2f5c26-e4cc-4626-bf06-3e27b2bdbdec,,(3)@10.0.77.237:5050,log-replica(1)@10.0.77.237:5050,1438288302620932096,message_outbound_queued
53a5f78a-f0f3-4273-8ba1-f29ac11341b5,mesos.internal.log.PromiseResponse,c75db6b9-7066-4ecc-a19d-3fe5f9c07702,ff2f5c26-e4cc-4626-bf06-3e27b2bdbdec,log-replica(1)@10.0.77.237:5050,(3)@10.0.77.237:5050,1438288302626352128,message_outbound_queued
f95fac79-d1ed-4e17-9c71-6b9a19594cab,mesos.internal.log.WriteRequest,a2cc030c-7128-4f06-a828-60d8789f6d3b,,(5)@10.0.77.237:5050,log-replica(1)@10.0.77.237:5050,1438288302656142080,message_outbound_queued
f95fac79-d1ed-4e17-9c71-6b9a19594cab,mesos.internal.log.WriteResponse,f27a49ef-f14e-4c75-8f2a-d8d2142d6282,a2cc030c-7128-4f06-a828-60d8789f6d3b,log-replica(1)@10.0.77.237:5050,(5)@10.0.77.237:5050,1438288302663973120,message_outbound_queued
9f6bb86d-7d3d-4b7b-93eb-842b35411d45,mesos.internal.log.LearnedMessage,36b1e706-b33c-482e-8694-28ec66168792,,@0.0.0.0:0,log-replic