vinoski

When you build Erlang/OTP on OS X, it unfortunately defaults to handling a maximum of 1024 file descriptors. You can get around this limitation with the right combination of configuration options and manual changes to a generated config file.

First, go into your unpacked Erlang/OTP source directory and run the following command, replacing the value 10000 with whatever value you want for max file descriptors:

perl -i -pe 's/(define\s+FD_SETSIZE\s+)\d+/\1 10000/' erts/config.h.in 

Next, when you run configure in your Erlang/OTP source directory, be sure to include the right CFLAGS setting, as shown below:

CFLAGS='-DREDEFINE_FD_SETSIZE -DFD_SETSIZE=15000 -D_DARWIN_UNLIMITED_SELECT' ./configure --enable-kernel-poll <other options>

cmeiklejohn

(defn threshold
  ([thr call-fn]
   #(threshold thr 1 call-fn))
  ([thr cur call-fn]
   (if (> cur thr)
     (do (apply call-fn [])
       #(threshold thr call-fn))
     #(threshold thr (inc cur) call-fn))))

kellymclaughlin

PBStatusFun = fun() ->
                      VnodePids = [Pid || {_, Pid} <- riak_core_vnode_manager:all_index_pid(riak_kv_vnode)],
                      Links = [process_info(Pid, [links]) || Pid <- VnodePids],
                      WorkerPoolPids = [WPPid || [{links,[_, WPPid]}] <- Links],
                      WorkerPoolLinks = [process_info(Pid, [links]) || Pid <- WorkerPoolPids],
                      PoolboyPids = [PoolboyPid || [{links,[_, PoolboyPid]}] <- WorkerPoolLinks],
                      [poolboy:status(Pid) || Pid <- PoolboyPids]
              end.

PBStatusFun = fun(Index) ->

clr

Rolling with Eventual Consistency

Keywords: riak, ripple, statistics, eventually consistent, race condition, CRDT, map/reduce, analytics

In a previous post I wrote about the different mindset that a software engineer should have when building for a key-value database as opposed to a relational database. When working with a relational database, you describe the model first and then query the data later. With a key-value database, you focus first on what you want the result of the query to look like, and then work backward toward a model.

As an example, I described averaging some value:

"In a SQL database, we can just call average() on that column, and it will compute the answer at query time. In a key-value store, we can add logic in the application layer to catch the object before it enters Riak, fetch the average value and number of included elements from Riak, compute the new rolling average, and save that answer back in Riak

bookshelfdave

# Dependencies:
# OCaml findlib 
# http://piqi.org/

git clone git@github.com:basho/riak_pb.git

cd src
rm -f *.proto
rm -f *.piqi
rm -f *.cm*

bobby

(ns datomic-helpers
  (:require [datomic.api     :as d]))

;;; Expose Datomic vars here, for convenience

;;; Ring middleware

(defn wrap-datomic
  "A Ring middleware that provides a request-consistent database connection and
  value for the life of a request."

sadache

If your csv doesn't contain escaped newlines then it is pretty easy to do a progressive parsing without putting the whole file into memory. The iteratee library comes with a method search inside play.api.libs.iteratee.Parsing :

def search (needle: Array[Byte]): Enumeratee[Array[Byte], MatchInfo[Array[Byte]]]

which will partition your stream into Matched[Array[Byte]] and Unmatched[Array[Byte]]

Then you can combine a first iteratee that takes a header and another that will fold into the umatched results. This should look like the following code:

// break at each match and concat unmatches and drop the last received element (the match)

ckirkendall

(use '[clojure.core.match :only [match]])

(defn evaluate [env [sym x y]]
  (match [sym]
    ['Number]   x
    ['Add]      (+ (evaluate env x) (evaluate env y))
    ['Multiply] (* (evaluate env x) (evaluate env y))
    ['Variable] (env x)))
    
(def environment {"a" 3, "b" 4, "c" 5})

reiddraper

Riak Users,

I've been working on some new features for Riak's secondary indexes (2i), and I wanted to get some feedback from the community before proceeding much further.

How does 2i work currently?

2i lets you query index values with

swannodette

(ns datomic-play.core
  (:use [datomic.api :only [db q] :as d])
  (:require [clojure.core.logic :as l]
            [clojure.pprint :as pp]))

(def uri "datomic:dev://localhost:4334/hello")

(defprotocol IUnifyWithDatum
  (unify-with-datum [u v s]))