- Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
- Models and Issues in Data Stream Systems
- Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
- Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
- [Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep=rep1&t
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
curl -s https://api.github.com/orgs/twitter/repos?per_page=200 | ruby -rubygems -e 'require "json"; JSON.load(STDIN.read).each { |repo| %x[git clone #{repo["ssh_url"]} ]}' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ּ_בּ | |
בּ_בּ | |
טּ_טּ | |
כּ‗כּ | |
לּ_לּ | |
מּ_מּ | |
סּ_סּ | |
תּ_תּ | |
٩(×̯×)۶ | |
٩(̾●̮̮̃̾•̃̾)۶ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Producer | |
Setup | |
bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test-rep-one --partitions 6 --replication-factor 1 | |
bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test --partitions 6 --replication-factor 3 | |
Single thread, no replication | |
bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance test7 50000000 100 -1 acks=1 bootstrap.servers=esv4-hcl198.grid.linkedin.com:9092 buffer.memory=67108864 batch.size=8196 |
The prep-script.sh
will setup the latest Node and install the latest perf version on your Linux box.
When you want to generate the flame graph, run the following (folder locations taken from install script):
sudo sysctl kernel.kptr_restrict=0
# May also have to do the following:
# (additional reading http://unix.stackexchange.com/questions/14227/do-i-need-root-admin-permissions-to-run-userspace-perf-tool-perf-events-ar )
sudo sysctl kernel.perf_event_paranoid=0
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Thee will be more information here when I share the entire problem space I'm working on, but | |
# in short, this is preview material for my second talk in a series called "What Computer Scientists Know". | |
# The first talk is on recursion, and goes through several examples., leading up to a problem based | |
# on a simple puzzle that initial estimates based on performance of a previous puzzle would take years | |
# to solve on modern computers with the techniques shown in Ruby. That sets the stage for improving the | |
# performance of that problem with threading, concurrency, and related tuning. | |
# | |
# The second talk is on threading and concurrency, touching on algorithmic performance as well. | |
# Using some knowledge of the problem (board symmetry, illegal moves, etc), we reduce the problem space | |
# to about .5% of what we initially thought it was. Still, the initial single threaded solution took more |
ror, scala, jetty, erlang, thrift, mongrel, comet server, my-sql, memchached, varnish, kestrel(mq), starling, gizzard, cassandra, hadoop, vertica, munin, nagios, awstats
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
import os | |
import mmap | |
import struct | |
import signal | |
import optparse | |
import cql | |
try: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
location /webhook/17cbb307-94ec-446b-a17b-ab82594c974c { | |
if ($request_method != 'POST') { | |
return 405; | |
} | |
content_by_lua 'ngx.print(io.popen("/path/to/script.sh"):read("*a"))'; | |
} |
NewerOlder