gistfile1.txt

11:53 <cagdas> I've send a message to the riak list for some time ago 
with the subject "Some Feature Ideas: Lists and Twin Keys" 

11:53 <cagdas> I wonder if this was a stupid message or not :) 

11:54 <benblack> cagdas: what is a twin key? 

11:54 <cagdas> benblack: it's just a made-up name 

11:54 <cagdas> what I mean is 

11:55 <cagdas> two keys referring to the same value 

11:55 <benblack> so, aliases 

11:55 <cagdas> yes 

11:55 <cagdas> does riak already support this? 

11:55 <pharkmillups> benblack: http://riak.markmail.org/search/?q=twin+key#query:twin%20key+page:1+mid:pdonq27ciizc4tcp+state:results 

11:55 <benblack> not really, but you could get almost exactly the same 
thing with links 

11:55 <benblack> and link walking 

11:55 <pharkmillups> for reference if you need it 

11:56 <benblack> pharkmillups: did you just tell me to rtfm? 

11:56 <pharkmillups> ;) 

11:57 <roidrage> haha, pharkmillups, that needs to go into the recap :) 

11:57 <benblack> cagdas: list structures present challenges in a system with no 
multi-node synchronization  mechanisms. 

11:59 <cagdas> I see, how redis do that (or says doing that) 

11:59 <jdmaturen> redis is a single process on a single server 

11:59 <benblack> redis is not distributed 

12:00 <benblack> and redis cluster has a single master for every key 

12:00 <benblack> if that master is unavailable, you can't write to that key 

12:02 <cagdas> I see but what is the difference between setting a key to 
{val1, val2} which was {val1} and saying add val2 to the end of {val1}? 

12:03 <benblack> what if you say "add val2 to the end" at the same time someone 
else says "add val3 to the 
end"? 

12:03 <benblack> what is the right result? 

12:04 <benblack> {val1, val2}? {val1, val3}? {val1, val2, val3}? {val1, val3, val2}? 

12:04 <cagdas> ideally the last two 

12:04 <cagdas> oh I see 

12:05 <benblack> ok, and how do you get there? 

12:05 <benblack> i have {val1, val2} and {val1, val3} as siblings now 

12:05 <benblack> how do i know it should be {val1, val2, val3} vs just replacing one 
2 element list with the other? 

12:06 <cagdas> what you know is that val2 and val3 share the same order although 
they are different 

12:06 <cagdas> another challenge will be if we add val3 2 times 

12:06 <benblack> what is someone else removed val2 and added val3? 

12:06 <cagdas> I see 

12:06 <benblack> so {val1, val3} really is what they intended 

12:07 <benblack> not {val1, val2, val3} 

12:07 <benblack> without inter-node coordination mechanisms, this is not 
generally possible 

12:08 <cagdas> I see, any plans to have this mechanism, or is this not the 
path riak is wanted to go? 

12:08 <benblack> there has been some discussion of integrating a paxos 
variant, but nothing firm 

12:10 <technowe_> cagdas: theres no law that says you cant use redis + riak 

12:11 <technowe_> shit i must still be logged in with my imac 

12:11 <cagdas> sure, I am planning that but... 

12:12 <cagdas> synchronizing redis and riak looks like an issue, still 

12:13 <johnae_> I have an issue proxying riak through nginx or more specifically 
luwak, anyone have experience with that? 

12:13 <pharkmillups> cagdas: might want to ask siculars about that next time he 
shows up. I know he is using riak + redis 

12:14 <cagdas> nice, I'll do 

12:14 <pharkmillups> cagdas: he 

12:14 <jdmaturen> which part are you trying to synchronize? 

12:14 <cagdas> say I've a list of comments 

12:14 <cagdas> adding new comment is easy on redis 

12:14 <pharkmillups> is active on the mailing list, too, so if you sent something along 
he (and others) would probably get back to you 

12:14 <cagdas> but I want to be sure that it's also saved to riak 

12:15 <technowe_> add it to riak, then store the riak key in some redis list 

12:15 <jdmaturen> post-commit hook indexing ftw 

12:15 <cagdas> I also want the list itself to be stored 

12:15 <jdmaturen> and redis persistence is not reliable enough? 

12:16 <cagdas> I don't want the list to stay on redis forever, redis has a limit 
at the end right? 

12:17 <jdmaturen> at the end? 

12:19 <hoodoos> guys, how to turn javascript cache off? 

12:20 <jdmaturen> cagdas: you could use the redis list operations, then every X seconds 
read the full list and save to riak 

12:20 <cagdas> redis stores everything in memory? 

12:20 <jdmaturen> gets trickier if lists are falling out of redis 

12:20 <jdmaturen> correct 

12:20 <technowe_> cagdas: redis persists to disks, so list data will be saved if it goes down 

12:21 <cagdas> jdmaturen: of course maybe it's everengineering but I was planning to 
create some queues from recent changes so a consumer will check whihc lists it should 
update from redis to riak etc. 

12:21 <jdmaturen> *falling out = running out of memory and pushing items out 
of cache 

12:21 <cagdas> technowe_: as far as I know, redis is still limited with the 
memory you have? 

12:21 <technowe_> redis 2.0+ has vm support 

12:21 <jdmaturen> cagdas: event queues aren't overengineering 

12:22 <jdmaturen> ;) 

12:22 <technowe_> maybe you want to go to #redis :) 

12:22 <cagdas> technowe_: Using vm is the start of the disaster 
I guess 

12:22 <cagdas> technowe_: :) 

12:22 <jdmaturen> vm = disaster, yes 

12:22 <technowe_> why's the vm a disaster? did i miss something 

12:23 <technowe_> oh i mean "redis vm" not your system vm 

12:23 <technowe_> http://code.google.com/p/redis/wiki/VirtualMemorySpecification 

12:24 <roidrage> the vm is happily humming along in production over here 

12:24 <roidrage> *knocks on wood* 

12:24 <cagdas> I see