Skip to content

Instantly share code, notes, and snippets.

@PharkMillups
Created September 13, 2010 22:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save PharkMillups/578168 to your computer and use it in GitHub Desktop.
Save PharkMillups/578168 to your computer and use it in GitHub Desktop.
12:20 <drewr> what do people do to manage complexity in their mapred functions?
12:20 <drewr> trying to do anything nontrivial results in an inordinate amount of time troubleshooting
12:21 <seancribbs> drewr: write smaller functions, first of all
12:22 <drewr> what's the way to modularize? not quite sure what's in scope when
12:22 <seancribbs> build objects that contain the functions you need
12:22 <seancribbs> then call them by name (instead of specifying the contents in your job)
12:23 <seancribbs> also, extract pieces that are complicated into 'helper' functions
12:24 <drewr> so you store small functions in riak and call out to those?
12:25 <drewr> how are the riak builtins.js wired in?
12:25 <drewr> iow, how Riak.reduceSum available as a function name?
12:27 <seancribbs> there's a configuration option for where to look
for your custom JS files:
http://wiki.basho.com/display/RIAK/Configuration+Files#ConfigurationFiles-jssourcedir
12:28 <drewr> I'm assuming each node would need the files, correct?
12:28 <seancribbs> yes
12:28 <seancribbs> you don't need to shutdown a running node to load them, you can use `riak-admin js_reload`
12:29 <drewr> ok, cool
12:29 <seancribbs> the other option is to store them in a bucket, but that doesn't solve
the modularity problem
12:38 <drewr> ok, trying out js_source_dir, took the same code that
was working and now get {error,{error,{case_clause,error},....}}
12:38 <drewr> is that just what we're going to get any time there's a problem?
12:38 <seancribbs> no, it should be giving you more useful errors
12:59 <drewr> js_source_dir works well, thanks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment