unlocking-the-superpowers-of-the-erlang-vm.org

unlocking-the-superpowers-of-the-erlang-vm.txt

* Unlocking the super-powers of the Erlang VM


 Lars Hesel Christensen
 VerneMQ Engineer

 twitter: @larshesel
 ✉ lars@vernemq.com
 ✉ lhc@larshesel.dk
 github.com/larshesel

* The Erlang VM

A virtual machine built to support Erlang and applications in the
Telco domain.

Other languages on the Erlang VM:

- Elixir
- LFE
- Efene
- Alpaca
- ...

* Erlang VM Features

- Hot code (re)loading
- Millions of processes
- Pattern matching
- Links and monitors
- Supervisors
- Logging
- Metrics
- Tracing

- Different schedulers available
- Async threads for IO
- Different memory allocators
- Thread pinning

* What are the super-powers?

- Hot code (re)loading
- Processes
- Links and monitors
- Pattern matching
- Tracing

* Tracing

- Allows to instrument the VM to emit messages on certain events at
  run-time.

Traceable events:

- function calls (and returns)
- process events (messages, exits)
- ...

* Tracing support

Built-in:

- `erlang:trace_pattern/3` and `erlang:trace/3`
- `dbg`

Community:

- erlyberly (github.com/andytill/erlyberly)
- redbug (github.com/massemanet/redbug)
- recon (github.com/ferd/recon)

* The `dbg` module

Function call example

#+begin_src erlang
(algol@algol)1> dbg:tracer(). %% Start a tracer process
{ok,<0.67.0>}
(algol@algol)2> dbg:p(all, c). %% Trace calls on all processes
{ok,[{matched,algol@algol,32}]}
(algol@algol)3> dbg:tp(lists, seq, x).
{ok,[{matched,algol@algol,2},{saved,x}]}
(algol@algol)4> lists:seq(1,10).
(<0.65.0>) call lists:seq(1,10)
(<0.65.0>) returned from lists:seq/2 -> [1,2,3,4,5,6,7,8,9,10]
[1,2,3,4,5,6,7,8,9,10]
#+end_src

* The `dbg` module

Trace process events

#+begin_src erlang
18> Pid = spawn(fun() -> receive {From,Msg} -> From ! Msg end end).
<0.121.0>
19> dbg:tracer().
{ok,<0.123.0>}
20> dbg:p(Pid,[m,procs]).
{ok,[{matched,nonode@nohost,1}]}
21> Pid ! {self(),hello}.
(<0.121.0>) << {<0.95.0>,hello}
(<0.121.0>) <0.95.0> ! hello
(<0.121.0>) exit normal
{<0.95.0>,hello}
#+END_SRC

* Using `recon`

#+begin_src erlang
1> recon_trace:calls({lists,seq, fun(_) -> return_trace() end}, 10).
2
2> lists:seq(1,10).

17:38:10.194721 <0.71.0> lists:seq(1, 10)
[1,2,3,4,5,6,7,8,9,10]

17:38:10.195357 <0.71.0> lists:seq/2 --> [1,2,3,4,5,6,7,8,9,10]
3>
#+end_src


#+begin_src erlang
recon_trace:calls({lists, seq, fun(_) -> return_trace() end},
                  {10,100}, [{scope, local}]).
#+end_src

* Building an MQTT session tracer

In VerneMQ we wanted to log specific events for an MQTT session. We
needed:

- Something fast
- Something safe
- Something where we could control formatting

Options:

- Try to use lager metadata
- Add log statements
- Use the tracing subsystem ...

* The architecture

 Trace System (VM) ->

      Rate Limiter (rate_tracer process) ->

              Trace Formatter (vmq_tracer process)

* Show code and demo




* The end

Tracing is a fantastic tool. Can completely replace `printf` type
debugging. It is invaluable for determining issues in running systems.