lenary

I'm going travelling this summer. Here's to where:

• Washington DC (~ 3 nights)
• Austin, TX (~ 3 nights)
• San Diego (~ 3 nights)
• San Francisco (~ 1 week)
• Portland (~ 3 nights)
• Seattle (~ 3 nights)
• NYC (more than 3 nights, less than a week)
• Boston (~ 1.5 weeks)

lenary

                  --   (constraints on k and v, defined somehow)
class Map (impl k v ~> constraints) where
  new :: impl k v
  insert :: (constraits) => k -> v -> impl k v -> impl k v
  get :: (constraints) => k -> impl k v -> Maybe v

-- then somehow this
instance Map (HashMap k v ~> (Hash k, Eq k)) where
  new = ...
  insert = ...

lenary

I've had one question about subpar performance in Riak 2.0's CRDTs. I thought I'd write this so that people can more easily diagnose these issues, without the CRDT team having to step in every time.

An Example: A Client was having problems with the performance fetching and updating sets. The issue manifested itself with poor fetch performance.

So, how do you go about debugging/diagnosing this?

lenary

$ erl
Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [dtrace]

Eshell V5.9.3.1  (abort with ^G)
1> <<99,111,110,118,101,114,115,97,116,105,111,110>>.
<<"conversation">>
2> <<100,107,114,100,54,58,104,107,110,120,110,114,115,49>>.
<<"dkrd6:hknxnrs1">>

lenary

bind-key -temacs-copy c copy-selection
bind-key -temacs-copy x copy-pipe "reattach-to-user-namespace /usr/bin/pbcopy"
bind-key -temacs-copy z copy-pipe "mate"

lenary

Sooo, CRDTs.

They broadly fall into two groups, Commutative and Convergent.

• Commutative Replicated Data Types (aka CmRDTs): These are also referred to as op-based CRDTs. Essentially, updates cause commutative operations to be broadcast. However, this broadcast channel is required to be reliable and have an order (see §2.2.2 of [1] below).
• Convergent Replicated Data Types (aka CvRDTs): These are also referred to as state-based CRDTs. Essentially they consist of a data structure with a commutative merge operation which will deterministically take two of these structures and merge them into one, preserving updates. These don't require a reliable broadcast system, instead can rely on the liveness properties of an Eventually Consistent (EC) system normally.

Why is the reliable broadcast thing an issue? To have it you require strong consistency. CRDTs are supposed to be designed for Highly Available, EC systems, which by definition (and CAP theorem) can't have strong consistency[+].

lenary

Counters are expected to land in Riak 1.4! See this pull-request.

h/t to @russelldb for the demo code.

lenary

%%%-------------------------------------------------------------------
%%% @author Heinz Nikolaus Gies <heinz@licenser.net>
%%% @copyright (C) 2013, Heinz Nikolaus Gies
%%% @doc
%%% An Implementation of the OR Set (Observe Remove Set) CvRDT. It
%%% is optimized over the default implementation. Details about the
%%% optimization are based on the paper:
%%% http://www.cmi.ac.in/~madhavan/papers/mss-tr-2013.html
%%% @end
%%% Created :  1 Jun 2013 by Heinz Nikolaus Gies <heinz@licenser.net>

lenary

# ALL destructive actions still need to be confirmed manually by the user. 
# It's also worth pointing out that in special cases, things may not work. 
# Most of this info is gleaned from https://devcenter.heroku.com/articles/migrating-from-shared-database-to-heroku-postgres

set -e
APP=$1

echo "APP = $APP"

heroku addons:add heroku-postgresql:dev -a $APP

lenary

-module(call_servers).

% External Methods
-exports([call_servers/3]).

% Internal Methods that are needed anyway (I think)
-exports([call_server/3, wait_loop/2]).

call_servers(Servers, Message, Timeout) ->
  Receiver = spawn(?MODULE, wait_loop, [self(), Servers, []]),