public
Last active

using rebar to create an erlang app with a dependency, and then starting a release created with reltool

  • Download Gist
rebar-reltool-relase-notes.md
Markdown

Summary

I'm trying to figure out how to use rebar to:

  • create erlang project
  • add a dependency on an erlang module from github
  • start the app via the erl console
  • create a release and start the app from the release (via the generated scripts)

History

  • What's the best way to declare dependencies for erlang? I would have thought using rebar's transitive dependency management would be the way to go... put external dependencies in rebar.config, run 'rebar get-deps && rebar compile' and you're off to the races. But this thread indicates that this could break apps with shared dependencies. If both apps A and B depend on C, and C hoarks, then both apps A and B die, too.

[stack overflow question] http://stackoverflow.com/questions/11141198/when-to-use-erlang-applicationstart-or-included-applications-and-a-supervisor)

I realize this is an extension to the above question, but I'm hoping that we can get some more discussion going by re-posting the question.

  • Assuming that this thread was incorrect, and that rebar dependencies are the way to go, I'm having a teensy problem. I was following Dizzy's 2011 presentation (http://vimeo.com/22700527) and thought, sweet, I'll create a sample app and have it depend on ibrowse (very cool app).

App Creation

make basic app

$ mkdir foo
$ cd foo
$ rebar create-app appid=foo
$ find .
.
./src
./src/foo_sup.erl
./src/foo.app.src
./src/foo_app.erl

verify compiles

$ rebar compile
==> foo (compile)
Compiled src/foo_app.erl
Compiled src/foo_sup.erl

add dependency

rebar.config (add this at project root)

{deps, [
       {ibrowse, ".*", {git, "git://github.com/cmullaparthi/ibrowse.git", {tag, "v4.0.1"}}}
   ]
}.

get deps

$ rebar get-deps
==> foo (get-deps)
Pulling ibrowse from {git,"git://github.com/cmullaparthi/ibrowse.git",
                         {tag,"v4.0.1"}}
Cloning into 'ibrowse'...
==> ibrowse (get-deps)

verify compiles

$ rebar compile
==> ibrowse (compile)
Compiled src/ibrowse_lib.erl
Compiled src/ibrowse_lb.erl
Compiled src/ibrowse_app.erl
Compiled src/ibrowse_test.erl
Compiled src/ibrowse_sup.erl
Compiled src/ibrowse.erl
/home/todd/projects/erlang/foo/deps/ibrowse/src/ibrowse_http_client.erl:999: Warning: variable 'State' is unused
Compiled src/ibrowse_http_client.erl
==> foo (compile)

Add the dependency

Add 'ibrowse' to my foo application's list of dependencies...(per instructions here http://www.erlang.org/doc/man/app.html).

deps/ibrowse/ebin/ibrowse.app shows:

Application = ibrowse

Ok, so I'll update my src/foo.app.src from:

{application, foo,
[
 {description, ""},
 {vsn, "1"},
 {registered, []},
 {applications, [
                 kernel,
                 stdlib
                ]},
 {mod, { foo_app, []}},
 {env, []}
]}.

to

{application, foo,
[
 {description, ""},
 {vsn, "1"},
 {registered, []},
 {applications, [
                 kernel,
                 stdlib,
<<< added >>>      ibrowse
                ]},
 {mod, { foo_app, []}},
 {env, []}
]}.

compile

$ rebar compile

verify compiles ok

Check that the ebin/foo.app has been updated to include the ibrowse dependency...

{application,foo,
            [{description,[]},
             {vsn,"1"},
             {registered,[]},
             {applications,[kernel,stdlib,ibrowse]},
             {mod,{foo_app,[]}},
             {env,[]},
             {modules,[foo_app,foo_sup]}]}.

Ok, looks good.

start the app, foo

$ erl -pa ebin/ -pa deps/ibrowse/ebin/
Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:6:6] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.2  (abort with ^G)

1> application:start(foo).
{error,{not_started,ibrowse}}

Ok, that didn't work...

try starting ibrowse first, then foo

$ erl -pa ebin/ -pa deps/ibrowse/ebin/
Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:6:6] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.2  (abort with ^G)
1> ibrowse:start().
{ok,<0.34.0>}
2> application:start(foo).
{error,{not_started,ibrowse}}

Seek advice...

Ok, I contacted some friends and erlang experts to figure out what I was doing wrong. The word from on-high goes like this:

  • get the dependency (ibrowse, in this case), into the project using rebar.config as in #3 above
  • from within the erlang shell, start the dependency (ibrowse) before this app (below)

Let's try it...

$ erl -pa ebin/ -pa deps/ibrowse/ebin/
Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:6:6] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.2  (abort with ^G)
1> application:start(ibrowse).
ok
2> application:start(foo).    
ok
3> regs().

** Registered procs on node nonode@nohost **
Name                  Pid          Initial Call                      Reds Msgs
application_controlle <0.6.0>      erlang:apply/2                    6091    0
code_server           <0.19.0>     erlang:apply/2                  113604    0
erl_prim_loader       <0.3.0>      erlang:apply/2                  182938    0
error_logger          <0.5.0>      gen_event:init_it/6                265    0
file_server_2         <0.18.0>     file_server:init/1                 113    0
foo_sup               <0.44.0>     supervisor:foo_sup/1                41    0
global_group          <0.17.0>     global_group:init/1                 59    0
global_name_server    <0.12.0>     global:init/1                       50    0
ibrowse               <0.38.0>     ibrowse:init/1                     962    0
ibrowse_sup           <0.37.0>     supervisor:ibrowse_sup/1           104    0
inet_db               <0.15.0>     inet_db:init/1                     220    0
init                  <0.0.0>      otp_ring0:start/2                 4278    0
kernel_safe_sup       <0.28.0>     supervisor:kernel/1                 58    0
kernel_sup            <0.10.0>     supervisor:kernel/1              34950    0
rex                   <0.11.0>     rpc:init/1                          35    0
standard_error        <0.21.0>     erlang:apply/2                       9    0
standard_error_sup    <0.20.0>     supervisor_bridge:standar           41    0
user                  <0.24.0>     group:server/3                      36    0
user_drv              <0.23.0>     user_drv:server/2                 1017    0

** Registered ports on node nonode@nohost **
Name                  Id              Command                                 
ok
4> 

Release Goodness

now, how about this release business

Advice from friends suggests: for releases, use reltool and it will "calculate the load/start order for you and encode it into the boot script

Following instructions from:

Create the release

$ mkdir rel
$ cd rel
$ rebar create-node nodeid=foo

Here's what we have

├── deps
│   └── ibrowse
├── ebin
│   ├── foo.app
│   ├── foo_app.beam
│   └── foo_sup.beam
├── rebar.config
├── rel
│   ├── files
│   ├── foo
│   └── reltool.config
└── src
    ├── foo_app.erl
    ├── foo.app.src
    └── foo_sup.erl

Update the reltool.config,

$ cat foo/rel/reltool.config

{sys, [
       {lib_dirs, []},
       {erts, [{mod_cond, derived}, {app_file, strip}]},
       {app_file, strip},
       {rel, "foo", "1",
        [
         kernel,
         stdlib,
         sasl,
         ibrowse,
         foo
        ]},
       {rel, "start_clean", "",
        [
         kernel,
         stdlib
        ]},
       {boot_rel, "foo"},
       {profile, embedded},
       {incl_cond, derived},
       {mod_cond, derived},
       {excl_archive_filters, [".*"]}, %% Do not archive built libs
       {excl_sys_filters, ["^bin/.*", "^erts.*/bin/(dialyzer|typer)", "^erts.*/(doc|info|include|lib|man|src)"]},
       {excl_app_filters, ["\.gitignore"]},
       {app, ibrowse, [{mod_cond, app}, {incl_cond, include}, {lib_dir, "../deps/ibrowse"}]},
       {app, foo, [{mod_cond, app}, {incl_cond, include}, {lib_dir, ".."}]}
      ]}.
{target_dir, "foo"}.
{overlay, [
           {mkdir, "log/sasl"},
           {copy, "files/erl", "\{\{erts_vsn\}\}/bin/erl"},
           {copy, "files/nodetool", "\{\{erts_vsn\}\}/bin/nodetool"},
           {copy, "files/foo", "bin/foo"},
           {copy, "files/foo.cmd", "bin/foo.cmd"},
           {copy, "files/start_erl.cmd", "bin/start_erl.cmd"},
           {copy, "files/install_upgrade.escript", "bin/install_upgrade.escript"},
           {copy, "files/sys.config", "releases/\{\{rel_vsn\}\}/sys.config"},
           {copy, "files/vm.args", "releases/\{\{rel_vsn\}\}/vm.args"}
          ]}.

Generate the release:

$ cd foo
$ rebar compile generate

[foo]$ rebar compile generate
==> ibrowse (compile)
Compiled src/ibrowse_lb.erl
Compiled src/ibrowse_app.erl
Compiled src/ibrowse_lib.erl
Compiled src/ibrowse_test.erl
Compiled src/ibrowse_sup.erl
Compiled src/ibrowse.erl
/home/todd/projects/erlang/foo_project/foo/deps/ibrowse/src/ibrowse_http_client.erl:999: Warning: variable 'State' is unused
Compiled src/ibrowse_http_client.erl
==> rel (compile)
==> foo (compile)
Compiled src/foo_app.erl
Compiled src/foo_sup.erl
==> rel (generate)

Start the release:

[foo]$ sh rel/foo/bin/foo console
Exec: /home/todd/projects/erlang/foo_project/foo/rel/foo/erts-5.9.2/bin/erlexec -boot /home/todd/projects/erlang/foo_project/foo/rel/foo/releases/1/foo -mode embedded -config /home/todd/projects/erlang/foo_project/foo/rel/foo/releases/1/sys.config -args_file /home/todd/projects/erlang/foo_project/foo/rel/foo/releases/1/vm.args -- console
Root: /home/todd/projects/erlang/foo_project/foo/rel/foo
Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:6:6] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.2  (abort with ^G)
(foo@127.0.0.1)1> 
(foo@127.0.0.1)1> application:which_applications().
[{foo,[],"1"},
 {ibrowse,"Erlang HTTP client application","4.0.1"},
 {sasl,"SASL  CXC 138 11","2.2.1"},
 {stdlib,"ERTS  CXC 138 10","1.18.2"},
 {kernel,"ERTS  CXC 138 10","2.15.2"}]


(foo@127.0.0.1)2> regs().

** Registered procs on node 'foo@127.0.0.1' **
Name                  Pid          Initial Call                      Reds Msgs
alarm_handler         <0.44.0>     gen_event:init_it/6                 28    0
application_controlle <0.6.0>      erlang:apply/2                    3608    0
auth                  <0.19.0>     auth:init/1                         43    0
code_server           <0.25.0>     erlang:apply/2                   34428    0
erl_epmd              <0.18.0>     erl_epmd:init/1                    131    0
erl_prim_loader       <0.3.0>      erlang:apply/2                  164809    0
error_logger          <0.5.0>      gen_event:init_it/6               2513    0
file_server_2         <0.24.0>     file_server:init/1                 274    0
foo_sup               <0.57.0>     supervisor:foo_sup/1                41    0
global_group          <0.23.0>     global_group:init/1                 59    0
global_name_server    <0.12.0>     global:init/1                       50    0
ibrowse               <0.52.0>     ibrowse:init/1                    1121    0
ibrowse_sup           <0.51.0>     supervisor:ibrowse_sup/1            97    0
inet_db               <0.15.0>     inet_db:init/1                     261    0
init                  <0.0.0>      otp_ring0:start/2                25837    0
kernel_safe_sup       <0.34.0>     supervisor:kernel/1                 58    0
kernel_sup            <0.10.0>     supervisor:kernel/1             385824    0
net_kernel            <0.20.0>     net_kernel:init/1                  428    0
net_sup               <0.17.0>     supervisor:erl_distributi          265    0
overload              <0.45.0>     overload:init/1                     39    0
release_handler       <0.46.0>     release_handler:init/1            4137    0
rex                   <0.11.0>     rpc:init/1                          28    0
sasl_safe_sup         <0.43.0>     supervisor:sasl/1                  156    0
sasl_sup              <0.42.0>     supervisor:sasl/1                  151    0
standard_error        <0.27.0>     erlang:apply/2                       9    0
standard_error_sup    <0.26.0>     supervisor_bridge:standar           41    0
user                  <0.30.0>     group:server/3                      20    0
user_drv              <0.29.0>     user_drv:server/2                  814    0

** Registered ports on node 'foo@127.0.0.1' **
Name                  Id              Command                                 
ok

I had some great advice from the erlang mailing list, and this advice caused me to go and re-read some documentation:

This has the stock info you'd expect: http://www.erlang.org/doc/apps/reltool/index.html

But the pdf is actually more useful, as it has some examples showing multiple {app...} stanzas: http://www.erlang.org/doc/apps/reltool/reltool.pdf

For example, from the pdf, on page 10:

{sys, [{escript,
"examples/display_args",
[{incl_cond, include}]},
{app, inets, [{incl_cond, include}]},
{app, mnesia, [{incl_cond, exclude}]},
{app, ssl, [{incl_cond, exclude}]},
{app, runtime_tools, [{incl_cond, exclude}]},
{app, syntax_tools, [{incl_cond, exclude}]}]}.
{sys,[{escript,"examples/display_args",[{incl_cond,include}]},
{app,inets,[{incl_cond,include}]},
{app,mnesia,[{incl_cond,exclude}]},
{app,ssl,[{incl_cond,exclude}]},
{app,runtime_tools,[{incl_cond,exclude}]},
{app,syntax_tools,[{incl_cond,exclude}]}]}

The key thing I was getting hung up on was how to include apps in the release configuration file, reltool.config:

First, make sure foo and it's dependency, ibrowse, are listed as part of the release:

       {rel, "foo", "1",
        [
         kernel,
         stdlib,
         sasl,
         ibrowse,
         foo
        ]},

Second, add the apps to the last part of {sys...

       {app, ibrowse, [{mod_cond, app}, {incl_cond, include}, {lib_dir, "../deps/ibrowse"}]},
       {app, foo, [{mod_cond, app}, {incl_cond, include}, {lib_dir, ".."}]}

Clearly I don't fully understand all this....I'll also be re-reading this page a few more times:

http://learnyousomeerlang.com/release-is-the-word

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.