gistfile1.txt

14:43 <Gerald1> Anyone know how bitcask handles replication? 

14:43 <benblack> bitcask doesn't handle replication 

14:43 <benblack> riak does 

14:43 <Gerald1> so upper level in Riak handles replication? 

14:44 <Gerald1> ok 

14:44 <Gerald1> bitcask does write through? or write-back caching? 

14:45 <Gerald1> I read the design doc, it seems that bitcask does write through. 
So every time a value is updated, the data file on disk is appended. 

14:45 <benblack> bitcask keeps the key index in memory, the rest relies on OS 
buffer cache, last i looked. 

14:46 <Gerald1> Is key index synced to disk? 

14:46 <Gerald1> or just in memory 

14:46 <benblack> both. the hint files are exactly that 

14:46 <Gerald1> The hint file is generated only when data files are merged if I 
understand it right. 

14:47 <benblack> yes 

14:48 <benblack> given your definition for write-through above, yes, bitcask 
is write-through 

14:50 <Gerald1> So hint file is associated with a merged data file 

14:50 <benblack> the contents of a cask file can be generated from the cask 
file, which is how that part works 

14:50 <benblack> so the index is not critical to keep around 

14:50 <benblack> s/contents/index/ 

14:51 <Gerald1> yes, index can be rebuilt from those data files. 

14:51 <Gerald1> but it may be slow 

14:52 <benblack> you plan on crashing all the time? 

14:52 <Gerald1> not really, but quick recovery/restart is good 

14:53 <benblack> it'd only be restart and only for cask files that had not been 
previously merged 

14:53 <benblack> which will usually be a small fraction of your data 

14:55 <Gerald1> I agree. 

14:56 <Gerald1> I assume value update is atomic, is that right? 

14:56 <benblack> hrm 

14:56 <benblack> that's kind of an odd question 

14:57 <benblack> across multiple nodes it isn't (that's inherent in the design) 

14:57 <benblack> and on a single node, the files are append only, 
and merge eliminates mutations/deletions later 

14:58 <Gerald1> right, my last question is stupid :-( 

14:59 <benblack> not stupid, just not applicable to riak/bitcask 

14:59 <benblack> using innostore as a backend, atomicity of udpates does matter 
since inno is not append-only 

14:59 <benblack> and it handles that atomicity on its own 

15:00 <Gerald1> I see. 

15:01 <Gerald1> so semantics of operation atomicity (e.g. getset, increment) 
is guaranteed by upper level code 

15:02 <benblack> increment is not a supported operation 

15:03 <benblack> and get/put concurrency involves vector clocks 

15:04 <benblack> the backends do guarantee atomicity for updates to them, 
but how that is done is predicated on the higher level mechanisms for concurrency 

15:05 <Gerald1> I see. 

15:07 <Gerald1> Thank you very much, Benjamin! 

15:07 <benblack> np