Skip to content

Instantly share code, notes, and snippets.

@PharkMillups
Created August 18, 2010 16:46
Show Gist options
  • Save PharkMillups/535377 to your computer and use it in GitHub Desktop.
Save PharkMillups/535377 to your computer and use it in GitHub Desktop.
07:04 <darrik> i'm trying to add custom map and reduce functions to riak. the docs say add them
to rel/files or rel/overlay (depending on which doc you look at), but i'm not sure how to get
rebar to include them.
07:05 <drev1> javascript or erlang functions?
07:05 <darrik> erlang
07:06 <darrik> can you store erlang functions in a bucket the way you can with js?
that would be ideal
07:06 <drev1> no
07:06 <drev1> Erlang functions need to be in the lib path of the node
07:07 <drev1> which means the Erlang functions need to be compiled and included with
the release
07:07 <darrik> right. the docs suggest if you put them in rel/overlay or rel/files, rebar will include
them, but perhaps i'm missing a rebar configuration step?
07:07 <drev1> including them in rel/files or rel/overlay would not work
07:08 <drev1> where does it say that in thedocs?
07:08 <darrik> https://wiki.basho.com/display/RIAK/MapReduce
07:09 <darrik> "Any modules and functions you use in your Erlang MapReduce calls must be
available on all nodes in the cluster. You can add them in Erlang applications in
the rel/overlay/lib directory, by specifying the -pz option in vm.args, or by adding
the path to the add_paths setting in app.config."
07:11 <drev1> there are more steps involved than described in that note; also, the change
in overlay structure makes that note obsolete
07:12 <darrik> this explains my failed attempts :)
07:12 <drev1> that note referred to a process of packaging a custom beam file in the overlay structure
07:15 <drev1> if you are looking to have rebar compile the code for you, you can create a
separate rebar project for your custom map/reduce functions and include that as a dep in rebar.config
07:15 <darrik> right on, thank you
07:15 <drev1> or you can add your functions to riak_kv_mapreduce.erl
07:15 <drev1> but I would go with the separate project route
07:16 <darrik> can you field a rebar question?
07:16 <drev1> sure
07:16 <darrik> rebar docs are so sparse :(
07:17 <darrik> all the rebar projects i've looked at with deps use {hg, ...}. is there a {git, ...} equivalent,
and is there somewhere that describes the fields for the deps?
07:17 <drev1> yes, there is a git equivalent
07:18 <drev1> looking for an example
07:18 <darrik> superb. is it {dep_name, dep_version, {vcs, url, name_plus_version}} ?
07:19 <darrik> or is that last param the source control tag or branch ?
07:19 <drev1> tag or branch
07:19 <darrik> fantastic! thank you kindly!
07:22 <drev1> {riak_kv, "0.12.0", {git, "git@github.com:basho/riak_kv.git", "master"}}
07:22 <darrik> perfect
07:31 <darrik> ok, last question, i promise
07:32 <darrik> it's telling me that dependency directory does not satisfy version regexp. what
precisely is it testing?
07:34 <drev1> good question
07:34 <drev1> one sec
07:34 <drev1> the vsn entry in the .app file
07:37 <drev1> darrik: does that make sense?
07:37 <darrik> yep! didn't think of that. thought maybe it was checking tags
07:38 <darrik> misleading because it specifically says the directory doesn't satisfy, so i
thought it was regexp'ing the directory, which seemed odd
07:38 <darrik> thank you for all your help, @drev1
07:50 <darrik> well, so far so good, but now i'm not sure how to get that code into the
running release. ./rebar generate doesn't include it, even though the dep is fetched and compiled.
07:50 <darrik> i tried adding it to rel/reltool.config
07:52 <drev1> where in reltool.config did you add it?
07:54 <darrik> i tried adding an {app, } section, and tried adding it under {rel, "riak", "0.12.0", [ ** ] }
07:54 <darrik> no combination of those worked :)
07:55 <drev1> hmm, adding it to the rel file should include it
07:55 <darrik> assuming i did all this, the proper sequence is ./rebar get-deps, ./rebar compile, ./rebar generate ?
07:56 <drev1> yes
07:59 <darrik> where is the proper place to specify it in reltool.config?
08:00 <drev1> the locations you have specified are correct
08:07 <darrik> @drev1: i figured it out. no other application was using it (obviously) and
i left off {incl_cond, include}
08:08 <darrik> @drev1: it's working perfectly now, so thank you for your considerable assistance
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment