Skip to content

Instantly share code, notes, and snippets.

@edwardw
edwardw / gist:1046499
Created June 25, 2011 13:39
Recipe to install System.Console.Readline hackage on Snow Leopard

Following works for me:

$ brew install readline
$ cabal install readline --configure-option=--with-readline-includes=/usr/local/Cellar/readline/6.2.1/include --configure-option=--with-readline-libraries=/usr/local/Cellar/readline/6.2.1/lib
@edwardw
edwardw / blog.md
Created June 27, 2011 15:50
Haskell surprised me again

This time it is case expression. I was looking for a way in haskell to define constants so I wrote this:

one = 1
two = 2
three = 3

which n =
  case n of
 one -> "one"
@edwardw
edwardw / ablog.md
Created July 3, 2011 08:58
Haskell operator section

In this wonderful tutorial "Write Yourself a Scheme in 48 Hours", this line puzzles me for a while:

apply :: String -> [LispVal] -> LispVal
apply func args = maybe (Bool False) ($ args) $ lookup func primitives

What does ($ args) really mean? Turns out it is called operator section, a special syntax which LYSH fails to mention. This thread did a great job on explaining it:

http://www.haskell.org/pipermail/beginners/2009-January/000776.html

@edwardw
edwardw / ablog.md
Created July 6, 2011 05:22
Parallel programming in Haskell

Simon Peyton Jones gave a talk on parallel programming in Haskell. A must to watch: http://skillsmatter.com/podcast/scala/talk-by-haskell-expert-simon-peyton-jones/js-1434

His notion of task, semi-implicit and data parallelism is very inspiring. And here are things interest me the most from his talk (in no particular order):

  • Erlang-style messaging in Haskell (cloud haskell)
  • parMap, a.k.a Control.Parallel.Strategies
  • Data.Array.Repa
  • Data.Array.Accelerate, i.e. GPU in Haskell

Both Repa and Accelerate come from Manuel Chakravarty. His homepage: http://www.cse.unsw.edu.au/~chak/

@edwardw
edwardw / gist:1517994
Last active September 29, 2015 00:17
Install zookeeper on illumos and supervise it using SMF

Two interesting findings:

  • A utility called manifold that helps creating SMF manifest;
  • An annoying platform compatibility issue in zookeeper startup script.

Fist, install zookeeper:

$ wget -c http://www.takeyellow.com/apachemirror//zookeeper/stable/zookeeper-3.3.4.tar.gz
$ tar xvzf zookeeper-3.3.4.tar.gz
@edwardw
edwardw / gist:1518116
Last active June 8, 2016 15:52
Install and start storm nimbus and supervisor

First, install storm dependencies. How do I compile jzmq for ZeroMQ on OSX? helps.

$ brew install zeromq
$ git clone https://github.com/nathanmarz/jzmq.git
$ cd jzmq
$ sudo vim /usr/share/aclocal/dirlist (sudo cat > /usr/share/aclocal/dirlist permission denied?)
/usr/local/share/aclocal
/usr/local/Cellar/pkg-config/0.25/share/aclocal/
$ export JAVA_HOME=/Library/Java/Home
@edwardw
edwardw / gist:1518437
Created December 24, 2011 22:00
Run a storm topology
$ mkdir ~/.storm
$ cp conf/storm.yaml ~/.storm
$ ...
$ bin/storm jar word-count.jar WordCountTopology

Storm is designed to be robust, e.g. being able to survive nimbus or supervisor restart. It does accomplish that, except one caveat. Say, I kill nimbus process and alter the content of storm.local.dir, in where nimbus has stored some states. Then I restart nimbus but it can't run! I guess it's because nimbus' state dir doesn't match those in zookeeper server's. By restarting zookeeper server process, everything works again.

@edwardw
edwardw / gist:1528044
Created December 28, 2011 14:07
Index wikipedia using ElasticSearch
$ bin/plugin -install elasticsearch/elasticsearch-river-wikipedia/1.0.0
$ bin/elasticsearch
$ curl -XPUT 192.168.1.6:9200/_river/
$ curl -XPUT 192.168.1.6:9200/_river/wikipedia/_meta -d '
{
    "type" : "wikipedia"
}'
@edwardw
edwardw / gist:1540780
Created December 30, 2011 17:49
Hush, not Harsh

HBase: The Definitive Guide has a sample application called hush. But it can't run as is. As this writing, the latest release of HBase is 0.90.5, which lacks certain features the book needs, such as coprocessor. So hush even can't compile against 0.90.5 HBase jar. It requires HBase 0.91.0-SNAPSHOT. Revision 1130916, to be precise, according to the book's website. Lars George, the author, kindly provides such a distribution in his apache site. But hush still need some minor tweaks to function properly as advertised.

That's a lot of rough edges. The book should've done better in regard to explain how to run its own sample application. Or it might not be the book's fault. I read somewhere that a team picked another NoSQL database over HBase because they felt HBase has 'too many moving parts'. This is probably still true. Since Hadoop reaches 1.0 several days ago, HBase should be able to impr

@edwardw
edwardw / gist:1609770
Created January 14, 2012 01:31
OpenIndiana Miscellaneous

Tunnel both TCP and DNS traffic over SSH

$ ssh -D 1080 -fN user@server

autossh seems to be more suitable for this task:

$ autossh -M 20000 -fN -D 1080 -i /path/to/key user@server