Created
September 22, 2010 14:37
-
-
Save PharkMillups/591780 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
14:42 <timmorgan> Is there any way to do bulk insert into Riak? | |
14:43 <seancribbs> timmorgan: no, one key at a time, but it's pretty easy to script | |
14:44 <misaka> seancribbs - Is that a design decision? ie: is it ever likely to change? | |
14:44 <timmorgan> ok thanks -- inserting 14K records is taking 10 mins or so | |
14:44 <seancribbs> timmorgan: if you can send it via Protocol Buffers, it'll be faster | |
14:45 <timmorgan> is that an alternative to the REST interface? | |
14:45 <seancribbs> yes | |
14:46 <timmorgan> ok I'll check it out! | |
14:46 <seancribbs> it's not supported in all languages right now, unfortunately | |
14:46 <timmorgan> no Ruby I gather? | |
14:47 <seancribbs> no… state of protobuffs in ruby is rather sad | |
14:47 <seancribbs> in general, not just with riak | |
14:47 <timmorgan> oh I see | |
14:49 <timmorgan> I'm not familiar with it, but would this be a good start? | |
http://github.com/macks/ruby-protobuf | |
14:51 <seancribbs> yes, I've looked at that. a little circumstantial, but someone made a protobuf | |
client using it and found it was slower than HTTP with curb | |
14:51 <seancribbs> my guess is that it's a combination of slow pack/unpack and bitshifting in Ruby | |
14:52 <seancribbs> which is pretty typical for these binary protocol things, witness the mongodb driver | |
14:55 <timmorgan> bummer | |
14:55 <seancribbs> timmorgan: in the long-term, we intend to have a pure C driver that can | |
link with many languages | |
14:56 <timmorgan> cool | |
14:56 <timmorgan> what's the state of affairs in Node.js land? | |
14:56 <timmorgan> I don't know Node, but am thinking of picking it up for a project | |
14:56 <timmorgan> thinking that Node + Riak could be awesome | |
14:57 <seancribbs> Node has nice support for both interfaces | |
14:57 <seancribbs> see riakjs.org | |
14:57 <technoweenie> welllll | |
14:57 <technoweenie> the coffeescript rewrite isnt done yet :) | |
14:57 <technoweenie> the current version only uses http. frank and i are still finishing | |
things up | |
14:57 <seancribbs> timmorgan: technoweenie is your man on that ;) and frank06 | |
14:57 <seancribbs> technoweenie: please to be stealing my test server | |
14:58 <timmorgan> I was trying to compare Node+Riak to Node+Riak, but couldn't find a | |
working MySQL driver for Node | |
14:59 <timmorgan> Still, Riak seems freaking fast (as does everything) on Node | |
14:59 <timmorgan> oops I mean Node+MySQL to Node+Riak | |
15:01 <technoweenie> timmorgan: http://github.com/felixge/node-mysql | |
15:02 <timmorgan> pretty sure I tried that one | |
15:02 <timmorgan> maybe it was just me being dumb | |
15:02 <timmorgan> I will try again | |
15:03 <timmorgan> oh right, I tried masuidrive's fork, per Google's recommendation -- | |
hasn't been updated since April | |
15:03 <timmorgan> I will try felixge's | |
15:03 <technoweenie> hah | |
15:03 <technoweenie> well he's a prolific and badass node author. ping him if | |
there are bugs | |
15:35 <timmorgan> I got the mysql driver to work; thanks technoweenie | |
15:35 <technoweenie> rad | |
15:36 <timmorgan> I think I'm doing something wrong with Riak though -- I get | |
about 475 reqs/s with Node+Riak, and over 1300 with Node+MySQL | |
15:36 <timmorgan> microbenchmark, I know, but that difference is striking | |
15:37 <timmorgan> I installed the Inno store | |
15:37 <amerine> timmorgan: seems odd. | |
15:38 <amerine> timmorgan: I use bitcask for almost everything riak+node though. | |
15:40 <seancribbs> timmorgan: depending on what you're doing and how many nodes you have, | |
that's to be expected | |
15:40 <seancribbs> HTTP is much more heavyweight than MySQL binary protocol | |
15:41 <timmorgan> I guess so | |
15:42 <technoweenie> yea you'll want protobufs for speed | |
15:43 <technoweenie> the win with riak comes from having lots of nodes for handling | |
writes. whereas a single mysql box will hit a limit and start deadlocking (depending on | |
indexes and crap like that) | |
15:43 <technoweenie> well, one of the wins, i guess | |
15:44 <timmorgan> right | |
15:44 <technoweenie> are those reads or writes though? | |
15:44 <timmorgan> reads | |
15:44 <technoweenie> just in a little test of mine, i noticed a quick 2x speedup just moving | |
to protobufs | |
15:44 <seancribbs> it's generally in the range of 2-10x speedup | |
15:45 <seancribbs> depending on many factors | |
15:45 <timmorgan> ok cool | |
15:45 <timmorgan> riak.js doesn't support them as of today? | |
15:45 <timmorgan> (I thought that's what I understood) | |
15:45 <seancribbs> i thought technoweenie got it working | |
15:45 <seancribbs> ;-) | |
15:45 <technoweenie> its not released | |
15:46 <timmorgan> can't wait to try it | |
15:46 <technoweenie> well you can try it now but its only coffeescript. i'm waiting on frank to | |
finish rewriting the http stuff |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment