Skip to content

Instantly share code, notes, and snippets.

@edwardw
edwardw / gist:1046499
Created Jun 25, 2011
Recipe to install System.Console.Readline hackage on Snow Leopard
View gist:1046499

Following works for me:

$ brew install readline
$ cabal install readline --configure-option=--with-readline-includes=/usr/local/Cellar/readline/6.2.1/include --configure-option=--with-readline-libraries=/usr/local/Cellar/readline/6.2.1/lib
@edwardw
edwardw / blog.md
Created Jun 27, 2011
Haskell surprised me again
View blog.md

This time it is case expression. I was looking for a way in haskell to define constants so I wrote this:

one = 1
two = 2
three = 3

which n =
  case n of
 one -> "one"
@edwardw
edwardw / ablog.md
Created Jul 3, 2011
Haskell operator section
View ablog.md

In this wonderful tutorial "Write Yourself a Scheme in 48 Hours", this line puzzles me for a while:

apply :: String -> [LispVal] -> LispVal
apply func args = maybe (Bool False) ($ args) $ lookup func primitives

What does ($ args) really mean? Turns out it is called operator section, a special syntax which LYSH fails to mention. This thread did a great job on explaining it:

http://www.haskell.org/pipermail/beginners/2009-January/000776.html

@edwardw
edwardw / ablog.md
Created Jul 6, 2011
Parallel programming in Haskell
View ablog.md

Simon Peyton Jones gave a talk on parallel programming in Haskell. A must to watch: http://skillsmatter.com/podcast/scala/talk-by-haskell-expert-simon-peyton-jones/js-1434

His notion of task, semi-implicit and data parallelism is very inspiring. And here are things interest me the most from his talk (in no particular order):

  • Erlang-style messaging in Haskell (cloud haskell)
  • parMap, a.k.a Control.Parallel.Strategies
  • Data.Array.Repa
  • Data.Array.Accelerate, i.e. GPU in Haskell

Both Repa and Accelerate come from Manuel Chakravarty. His homepage: http://www.cse.unsw.edu.au/~chak/

@edwardw
edwardw / gist:1517994
Last active Sep 29, 2015
Install zookeeper on illumos and supervise it using SMF
View gist:1517994

Two interesting findings:

  • A utility called manifold that helps creating SMF manifest;
  • An annoying platform compatibility issue in zookeeper startup script.

Fist, install zookeeper:

$ wget -c http://www.takeyellow.com/apachemirror//zookeeper/stable/zookeeper-3.3.4.tar.gz
$ tar xvzf zookeeper-3.3.4.tar.gz
@edwardw
edwardw / gist:1518116
Last active Jun 8, 2016
Install and start storm nimbus and supervisor
View gist:1518116

First, install storm dependencies. How do I compile jzmq for ZeroMQ on OSX? helps.

$ brew install zeromq
$ git clone https://github.com/nathanmarz/jzmq.git
$ cd jzmq
$ sudo vim /usr/share/aclocal/dirlist (sudo cat > /usr/share/aclocal/dirlist permission denied?)
/usr/local/share/aclocal
/usr/local/Cellar/pkg-config/0.25/share/aclocal/
$ export JAVA_HOME=/Library/Java/Home
@edwardw
edwardw / gist:1518437
Created Dec 24, 2011
Run a storm topology
View gist:1518437
$ mkdir ~/.storm
$ cp conf/storm.yaml ~/.storm
$ ...
$ bin/storm jar word-count.jar WordCountTopology

Storm is designed to be robust, e.g. being able to survive nimbus or supervisor restart. It does accomplish that, except one caveat. Say, I kill nimbus process and alter the content of storm.local.dir, in where nimbus has stored some states. Then I restart nimbus but it can't run! I guess it's because nimbus' state dir doesn't match those in zookeeper server's. By restarting zookeeper server process, everything works again.

@edwardw
edwardw / gist:1528044
Created Dec 28, 2011
Index wikipedia using ElasticSearch
View gist:1528044
$ bin/plugin -install elasticsearch/elasticsearch-river-wikipedia/1.0.0
$ bin/elasticsearch
$ curl -XPUT 192.168.1.6:9200/_river/
$ curl -XPUT 192.168.1.6:9200/_river/wikipedia/_meta -d '
{
    "type" : "wikipedia"
}'
@edwardw
edwardw / gist:1540780
Created Dec 30, 2011
Hush, not Harsh
View gist:1540780

HBase: The Definitive Guide has a sample application called hush. But it can't run as is. As this writing, the latest release of HBase is 0.90.5, which lacks certain features the book needs, such as coprocessor. So hush even can't compile against 0.90.5 HBase jar. It requires HBase 0.91.0-SNAPSHOT. Revision 1130916, to be precise, according to the book's website. Lars George, the author, kindly provides such a distribution in his apache site. But hush still need some minor tweaks to function properly as advertised.

That's a lot of rough edges. The book should've done better in regard to explain how to run its own sample application. Or it might not be the book's fault. I read somewhere that a team picked another NoSQL database over HBase because they felt HBase has 'too many moving parts'. This is probably still true. Since Hadoop reaches 1.0 several days ago, HBase should be able to impr

@edwardw
edwardw / gist:1609770
Created Jan 14, 2012
OpenIndiana Miscellaneous
View gist:1609770

Tunnel both TCP and DNS traffic over SSH

$ ssh -D 1080 -fN user@server

autossh seems to be more suitable for this task:

$ autossh -M 20000 -fN -D 1080 -i /path/to/key user@server