russelldb

How to create, build and run an OTP application using Rebar

Get Rebar from bitbucket.org

hg clone http://hg.basho.com/rebar/

Build Rebar

  ./bootstrap 
==> rebar (compile)
Congratulations! You now have a self-contained script called "rebar" in
your current working directory. Place this script anywhere in your path

russelldb

Setting Up Clojure on OS X

I spent a lot of time trying to find a pretty optimal (for me) setup for Clojure… at the same time I was trying to dive in and learn it. This is never optimal; you shouldn't be fighting the environment while trying to learn something.

I feel like I went through a lot of pain searching Google, StackOverflow, blogs, and other sites for random tidbits of information and instructions.

This is a comprehensive "what I learned and what I ended up doing" that will hopefully be of use to others and act as a journal for myself if I ever have to do it again. I want to be very step-by-step and explain what's happening (and why) at each step.

Step 1: Getting Clojure (1.3)

russelldb

alias focus="sudo sh -c \"echo '127.0.0.1 www.facebook.com twitter.com mail.google.com # aab6de513ab5de9359809f3cdb62d352' >> /etc/hosts\""
alias unfocus='sudo sed -i "/aab6de513ab5de9359809f3cdb62d352/d" /etc/hosts'

russelldb

on the guest SmartOS VM

pkg_add http://pkgsrc.joyent.com/sdc6/2012Q1/i386/All/smtools
# note this is going to shut down the VM
sm-prepare-image

on the global zone

russelldb

How to stream the London 2012 Olympics

There have been several HOWTOs posted regarding streaming the 2012 Olympics using HTTP / SOCKS proxies via SSH and other similar methods. None of these actually work using the latest Flash on Mountain Lion (with Firefox, Chrome or Safari). Additionally, the third-party streaming sites don't provide BBC's amazing interface, which lets you quickly skip to individual competitors and events. However, setting up an OpenVPN server does work, with some tweaks. You'll get the exact same UX that people in England receive.

• Get a Linode VM in the UK. The 512MB server for $20 works just fine. (If you want to use my referral link, go for it: http://bit.ly/OuzdVe)

• Follow the standard OpenVPN installation documentation. (Basically, 'apt-get install openvpn' or 'yum install openvpn' and then follow these docs: http://openvpn.net/index.php/open-source/documentation/howto.html). For an OS X client, I prefer Viscosity: http://www.thesparklabs

russelldb

(riak@127.0.0.1)1> {ok, C} = riak:local_client().
{ok,{riak_client,'riak@127.0.0.1',undefined}}
(riak@127.0.0.1)2> Object = riak_object:new(<<"testbucket">>, <<"testkey">>, <<"testvalue">>).
{r_object,<<"testbucket">>,<<"testkey">>,
          [{r_content,{dict,0,16,16,8,80,48,
                            {[],[],[],[],[],[],[],[],[],[],[],[],[],[],...},
                            {{[],[],[],[],[],[],[],[],[],[],[],[],...}}},
                      <<"testvalue">>}],
          [],
          {dict,1,16,16,8,80,48,

russelldb

Locate the section for your github remote in the .git/config file. It looks like this:

[remote "origin"]
	fetch = +refs/heads/*:refs/remotes/origin/*
	url = git@github.com:joyent/node.git

Now add the line fetch = +refs/pull/*/head:refs/remotes/origin/pr/* to this section. Obviously, change the github url to match your project's URL. It ends up looking like this:

russelldb

CvRDTs are (almost?) as general as they can be

What are you talking about, and why should I care?

Now that we live in the Big Data, Web 3.14159 era, lots of people want to build databases that are too big to fit on a single machine. But there's a problem in the form of the CAP theorem, which states that if your network ever partitions (a machine goes down, or part of the network loses its connection to the rest) then you can keep consistency (all machines return the same answer to

russelldb

Here's where I understand the state of the art to be:

• In this INRIA tech report, Shapiro, Preguiça, Baquero and Zawirski (SPBZ) prove, amongst other things, that a sufficient condition for CRDTs to achieve eventual consistency on networks which may reorder and duplicate packets (which I'll call flaky networks, henceforth) is that
  1. the underlying datatype forms a semilattice,
  2. messages are full states,
  3. incoming messages are combined with the node's current state using the least-upper-bound operation in the semilattice.
• It's possible to relax condition 2 and still achieve eventual consistency over flaky networks by fragmenting the state into independent parts and transmitting updates to each part separately. For instance, in the G-Set CRDT (an add-only bitset) one can transmit only the index of the element to be added.
• In [these slides from a talk at Dagstuhl](http://www.dagstuhl.de/mat/Files/13/13081/13081.BaqueroCarlos.Sl

russelldb

Step 1: From you logs I have decoded the bucket/key

binary_to_term(<<131,109,0,0,0,9,85,114,108,84,111,83,99,97,110>>).
<<"UrlToScan">>
binary_to_term(<<131,109,0,0,0,61,104,116,116,112,37,51,97,37,50,102,37,50,102,119,119,119,46,97,110,104,114,105,46,110,101,116,37,50,102,37,51,102,102,101,101,100,37,51,100,114,115,115,50,37,50,54,97,109,112,37,51,98,112,37,51,100,55,51,49,49,48>>).
<<"http%3a%2f%2fwww.anhri.net%2f%3ffeed%3drss2%26amp%3bp%3d73110">>

Step 2: Please run the below snippits from riak attach to identify the owning partitions of this bucket/key