gistfile1.txt

8:19 <micheil> with riak's map/reduce, does that emit results over the network as they are found? 

08:20 <seancribbs> micheil: map results are sent back to the coordinating node, yes 

08:20 <micheil> seancribbs: I'm meaning if I'm connecting over the binary protocol from say, 
node.js, and my riak db has say 10,000 items, that's going to take sometime to iterate over 

08:21 <micheil> so, will my client connecting over the binary protocol receive the 
items as they get matched? 

08:21 <seancribbs> yes, if you're using protobuffs, they'll be streamed 

08:21 <seancribbs> on HTTP, you have to tell it to stream 

08:22 <micheil> okay 

08:22 <seancribbs> IIRC 

08:22 <micheil> and if as this map/reduce is running, if new items that match get 
added, will those end up coming back? 

08:23 <seancribbs> until the job finishes 

08:23 <seancribbs> it's not a "continuous query" 

08:23 <micheil> okay 

08:24 <seancribbs> however, if you wanted to set up something like that, you could 
make a postcommit hook that posts to a message queue 

08:24 <micheil> hmm.. I'll have to read the documentation on that 

08:25 <seancribbs> arg gave a demo of something like that at the latest riak meetup in SF 

08:25 <micheil> I've got something of a community events stream, similar to the 
twitter streaming api, and as data comes in, I want to write it to database for later 
back querying, but I also want to stream it down to a client that can filter the 
results or something 

08:26 <seancribbs> yeah, a message queue or XMPP alongside riak would be good for that 

08:26 <arg> micheil: this may be useful: 
https://bitbucket.org/argv0/riak_rabbit/src/tip/apps/rabbit_publisher/src/ 

08:26 <micheil> well, to say my full stack, it's node.js + websockets + something. 

08:27 <bingeldac> /dev/mull? 

08:27 <bingeldac> heh 

08:27 <bingeldac> null 

08:27 <micheil> so as results come in I'm streaming them down, but it'd 
be great to be able to filter the data 

08:27 <seancribbs> /dev/mullet? 

08:27 <micheil> bingeldac: fastest datastore around, right? 

08:27 <bingeldac> webscale it is 

08:27 <seancribbs> micheil: yeah, your post-commit hook could do that 

08:27 <micheil> seancribbs: okay, cool. 

08:27 <micheil> and riak, it can store json data like mongodb, right? 

08:28 <bingeldac> yes 

08:28 <bingeldac> and you can parse the json in m/r functions 

08:28 <micheil> nice. 

08:29 <seancribbs> micheil: yes JSON. not like mongodb. 

08:29 <micheil> okay 

08:29 <micheil> I've never done much with riak 

08:29 <seancribbs> micheil: one more data point about the streaming map-reduce. 
the last message you get will have the "done" flag set 

08:29 <seancribbs> so you know when your query is done 

08:29 <micheil> okay 

08:29 <micheil> neat 

08:29 <seancribbs> forgot about that