Erlang Study Meeting Tokyo 2012.09 @kuenishi

gistfile1.rst

MessagePack <3 Cowboy

@kuenishi

Erlang Study Meeting Tokyo 2012.09

9/28

Who are you?

• @kuenishi
• GitHub, Twitter, Facebook
• "Class Change"
• Basho Japan KK

MessagePack

• Serialisation Protocol
• Efficient without compression
• easy to compute

{"compact" : true}

81 A7 compact C3

• msgpack.org
• almost compatible with JSON

Why so compact?

• Kind of a Huffman coding
• Shorter code for smaller value
• First byte sometimes include binary length

nil

16#C0

false

16#C2

true

16#C3

float

16#CA

double

16#CB

0..127

2#0

-32..-1

2#111

2#101

raw bytes up to 31 bytes

2#1001

array up to 15 elements

2#1000

map up to 15 entries

MessagePack RPC

• RPC Protocol

[0, X, "ping", []]

=> [1, X, nil, "pong"]

• replacable Transport layer
• TCP, UDP, UDS, SCTP, TCP/gz, (ZeroMQ)
• Short request after long-time request can overcome and return earlier

Other implementations

• C++, Java
• Ruby, Python, PHP, Perl, etc....

MessagePack Erlang

• github.com/msgpack/msgpack-erlang
• So small because of binary syntax
• 280 LoC

0.0.3: before cowboy

• basic inet_async TCP server
• also has UDP transport
• rather simple process tree

$ wc *.erl                                             [tmp ~/src/msgpack_rpc/src]
   249     868    8845 gen_msgpack_rpc.erl
   210     736    8822 gen_msgpack_rpc_srv.erl
   146     481    5350 mprc.erl
   161     522    5398 mprc_tcp.erl
   160     522    5525 mprc_udp.erl
    63     160    2075 mprs.erl
   227     748    8020 mprs_tcp.erl
   225     740    8148 mprs_udp.erl
   363    1415   13413 msgpack.erl
    47     102    1301 msgpack_util.erl
  1851    6294   66897 total

0.0.4: after cowboy

$ wc *.erl
    55     199    2054 msgpack_rpc_client.erl
   208     632    7552 msgpack_rpc_connection.erl
   187     553    6105 msgpack_rpc_protocol.erl
   450    1384   15711 total

How to run

• Start server which exports the function

-export([hello/1]).
hello(_Argv)-> <<"hello">>.

...

  {ok, _Pid} = cowboy:start_listener(testlistener, 3,
                                    cowboy_tcp_transport, [{port, 9199}],
                                    msgpack_rpc_protocol, [{module, msgpack_rpc_test}]),

• Start client

{ok, Pid} = msgpack_rpc_client:connect(tcp, "localhost", 9199, []),
{ok, <<"hello">>} = msgpack_rpc_client:call(Pid, hello, [<<"hello">>]),

Be careful

• No string list
• Use binary string for "string" in other languages
• JSON object (Python dict) is: {proplist()}

Why cowboy?

• Implementing transport is difficult even in Erlang
• Replacable transport layer
• Stable

T H A N K S

One more thing...

• msgpack-rpc-erlang 0.4.0 doesn't work with cowboy 0.7.0 !!!!!!

SO I UPDATED IN 20 MINUTES -------------------------

• let's do the fxxking REVIEW.
• still needs fix in deps/cowboy/rebar.config
• and just released 0.5.0 (master).