edocabinet.txt

= "edo cabinet" talk RubyKaigi2009 (20090719)

03:08:46 <takesako> ３日目～★
09:27:11 <tmaeda> 中会議場：都合によりセッションの順序を変更します。最初はartonさんの予定でしたが田中哲さんにします。
09:27:55 <muscovyduck> edo cabinet
09:28:18 <muscovyduck> もうすぐ始まるよ
09:30:11 <tdtds> test
09:33:15 <muscovyduck> 朝は人が少ないなあ
09:33:48 <muscovyduck> 始まりました
09:33:52 <muscovyduck> 日本語だ
09:34:07 <muscovyduck> http://tinyurl.com/edocabinet
09:34:17 <muscovyduck> ruby and tokyo cabinet
09:34:40 <muscovyduck> スイス人、横浜に住んでます
09:36:19 <muscovyduck> Rubyforgeにtokyocabinetのコードがない
09:36:29 <muscovyduck> ちょっとCっぽいので使いづらい
09:36:40 <muscovyduck> ハッシュのように使いたい
09:36:48 <leonardchin> Hi
09:36:52 <leonardchin> The oriiginal bindings were way too C-like
09:37:01 <leonardchin> Way too many thigns to write yourself (boilerplate)
09:37:09 <leonardchin> so I made it simpler.
09:37:16 <leonardchin> And reduced the amount you oneed to write
09:37:21 <leonardchin> rufus is latin for red?
09:37:33 <leonardchin> I like C, but I'm not very good at it
09:37:46 <leonardchin> At the end of last year, Charles Nutter, I heard about FFI for Ruby
09:38:00 <leonardchin> via Charles Nutter's blog
09:38:04 <leonardchin> I was very taken by it
09:38:16 <leonardchin> Foreign Function Interface
09:38:22 <leonardchin> the first FFI was for common list
09:38:31 <leonardchin> and after that, many other languages got an FFI
09:38:37 <leonardchin> even Haskell has one
09:38:41 <muscovyduck> Foreign Function Interface
09:38:52 <leonardchin> In ruby, the first FFI implementation was in Rubinius
09:38:56 <leonardchin> it was a nice ruby DSL
09:39:23 <leonardchin> I felt that the implementation had lots of potential
09:39:40 <leonardchin> I'm going to introduce 4 examples of FFI
09:39:42 <leonardchin> first, crypt
09:39:45 <leonardchin> lets look at man crypt
09:39:48 <leonardchin> simple
09:39:53 <leonardchin> here's my example
09:40:08 <leonardchin> Simple isn't it
09:40:12 <leonardchin> Next example
09:40:14 <leonardchin> Uuid
09:40:26 <leonardchin> THere are already various implementations
09:40:38 <leonardchin> Unfortunately this example only works on mac os x
09:40:40 <leonardchin> and not on linux
09:40:45 <leonardchin> I asked for help on the FFI mailing list
09:40:50 <leonardchin> but noone responded
09:40:50 <leonardchin> :(
09:41:06 <leonardchin> This is how the FFI implementation in ruby is used
09:41:15 <leonardchin> first attach C functinos, then call them
09:41:24 <leonardchin> Next is an example using libevent
09:41:36 <leonardchin> Its a little big, so i've split it into 2 pages
09:41:43 <leonardchin> first, i attach lots of libevent functinos
09:41:55 <leonardchin> and i link the dylib with ffi_lib
09:42:01 <leonardchin> then i call it
09:42:09 <leonardchin> This is a simple web server
09:42:18 <leonardchin> unfortunately, it only ever responds with 'hello world'
09:42:28 <leonardchin> here is a sample performance test on my computer
09:42:43 <leonardchin> its just a toy, but it gives you a taste of potential performance
09:42:50 <leonardchin> here is an example using Lua
09:43:01 <leonardchin> lua is a language from brazil, and is really good for embedding
09:43:17 <leonardchin> this program uses Lua tables
09:43:28 <leonardchin> Lua-Ruby bridge built with ruby ffi
09:43:56 <leonardchin> there is also an ffi-inliner library
09:44:09 <leonardchin> you can inline c code, and then call it later
09:44:16 <leonardchin> oops, i forgot about tokyo cabinet
09:44:23 <leonardchin> back to Tokyo Cabinet/Tyrant
09:44:32 <leonardchin> Its a photo of Shinjuku, but this is about TOkyo ;)
09:44:54 <leonardchin> Its by Mikio Hirabayashi, who works for mixi
09:45:06 <leonardchin> The biggest source of information is in the mixi community
09:45:20 <leonardchin> there is also a sourceforge group, but Mikio only rarely replies there ;)
09:45:36 <leonardchin> This is the mixi engineers blog where mikio often writes about tokyo tyrant/cabinet
09:45:42 <leonardchin> I don't read it much, but i like the pictures
09:45:55 <leonardchin> Tokyo *** products are GPL licensed
09:46:01 <leonardchin> and the code is publicly available
09:46:05 <leonardchin> but the repository is not
09:46:12 <leonardchin> But that's ok, i can read the soruce
09:46:18 <leonardchin> it is very clean and readable source
09:46:39 <leonardchin> So, rufus-tokyo was initially just for using tokyo cabinet
09:46:45 <leonardchin> but i soon added support for tokyo tyrant too
09:46:52 <leonardchin> tokyo cabinet is local only
09:47:00 <leonardchin> tokyo tyrant lets you use it remotely
09:47:19 <leonardchin> Tokyo tyrant can be used with many programs over the network
09:47:32 <leonardchin> I use the word "remote", but i mean here tcp or unix sockets
09:47:51 <leonardchin> it speaks tt (tokyo tyrant) , memchached and http protocols
09:48:07 <leonardchin> rufus-tokyo speaks tokyo tyrants protocol
09:48:30 <leonardchin> the key/value api is abstract, the native api is key/hash
09:48:37 <leonardchin> i.e. a table
09:48:55 <leonardchin> tokyo tyrant only has the table and abstract apis
09:49:03 <leonardchin> rufus-tokyo sticks to this philosophy
09:49:09 <leonardchin> next, i'll show some code
09:49:30 <leonardchin> the top snipped is a sample of cabinet usage, the bottom snippet is tyrant
09:49:50 <leonardchin> It works just like a hash
09:50:01 <leonardchin> Pretty trivial, so i'll show something more interesting
09:50:04 <leonardchin> the table API
09:50:36 <leonardchin> its quite simple code
09:50:45 <leonardchin> looks a little like an sql table
09:50:58 <leonardchin> but it has no schema, and there are no columns
09:51:02 <leonardchin> its just a key-value hash
09:51:21 <leonardchin> its also easy to use tyrant tables
09:51:29 <leonardchin> the only difference is the connection
09:51:33 <leonardchin> everything else is the same
09:51:44 <leonardchin> local/remote only differs in the setup code
09:52:15 <leonardchin> so Lua has reappeared in the bottom right
09:52:25 <leonardchin> So I talked about Lua before
09:52:30 <leonardchin> its a very easy to embed runtime
09:52:40 <leonardchin> Mikio embedded Lua into Tokyo Tyrant
09:52:57 <leonardchin> its really easy to recompile it with embedded lua
09:53:10 <leonardchin> the left hand side is the increment fucnction in lua
09:53:21 <leonardchin> the code on the right is calling the increment function
09:53:26 <leonardchin> inside tokyo cabinet
09:54:14 <leonardchin> If you're interested in using Tokyo Tyrant, check out some of Ilya's blog posts.
09:54:48 <leonardchin> Finally
09:54:51 <leonardchin> Rufus::Edo
09:55:07 <leonardchin> The first 20 slides of this presentation were about FFI
09:55:21 <leonardchin> unfortunately, FFI is a lot slower than regular extensions for ruby
09:55:38 <leonardchin> the default ruby bindings are much faster than rufus-tokyo
09:55:55 <leonardchin> so I created rufus::edo as a C-extension for ruby
09:56:28 <leonardchin> it has the same interface as Rufus::Tokyo
09:56:48 <leonardchin> this is the same code as before, with just the require statements and connections changed
09:57:00 <leonardchin> it also uses the same rufus namespace
09:57:08 <leonardchin> What you get is the speed of a real C extension
09:57:30 <leonardchin> You should choose Rufus ::Edo if you want speed
09:57:46 <leonardchin> the pure ruby part is referring to Mikio's tyrant library
09:58:04 <leonardchin> its a bit slo
09:58:04 <leonardchin> w
09:58:22 <leonardchin> but you don't need to install the tyrant library locally
09:58:57 <leonardchin> ruby-tokyotyrant is another implementation, and is probably the fastest client for ruby
09:59:34 <leonardchin> ruby-tokyotyrantという新しいライブラリがあって、rufus-tokyoと同じAPI
09:59:57 <leonardchin> rubyでもっとも高速なtokyo tyrantクライアント実装
10:00:06 <leonardchin> there are lots of projects rufus tokyo in the wild
10:00:13 <leonardchin> some of them are here
10:00:40 <leonardchin> Thank you