Publishing of Erlang Libraries with Elixir Bindings

index.md

This post explores how you can write and publish a library that can be used directly in Erlang without the need to install Elixir, but also provide convenient Elixir bindings.

Should you create a multi-language library?

Writing a multi-language library comes with some complexity. It therefore makes sense to ask yourself first, if the additional work & complexity is worth the effort.

Any library written in Erlang and published to Hex, can also be used in Elixir. Instead of calling SomeModule.function(...), you can also call :some_module.function(...) directly. Erlang modules can also be aliased by the user to align the Developer Experience with Elixir modules:

alias :some_module, as: SomeModule

SomeModule.function(...)

Child Specifications

Elixir workers & supervisors come with a predefined child_specification/1 function. This function allows a supervisor to start your worker / supervisor without crafting a child specification by hand.

This function can also be implemented directly in Erlang to allow the same behavior.

Records / Structs

Erlang does not have structs built-in. Instead, often records are used to provide a data structure with a defined shape, which is also checked at compiletime.

Records can also be used in Elixir. They are however a lot less common and not recommended in most use-cases.

It can therefore make sense, to provide an Elixir struct conversion fr Erlang records.

Other Data Structures

Elixir provides a whole collection of data structures like Stream, Range & MapSet. Those data strcutres do not exist in Erlang. To provide an as convenient as possible developer experience, it can make sense to expose functionality differently in Elixir.

Writing your libary

Erlang Core

I recommend writing your library first in Erlang only. To do so, start your rebar3 project as usual and implement the functionality in Erlang.

Elixir Bindings

To add elixir bindings, create a new mix project in a separate directory. Make sure to use the same application name as defined in src/[NAME].app.src. Copy all relevant files into your main repository:

• mix.exs - Mix Config
• .gitignore - Merge with existing .gitignore
• .formatter.exs - Elixir Formatter Config
• test/test_helper.exs - ExUnit Test Initialization

When running Mix tasks, Mix will automatically compile both the Elixir files (in lib) and the Erlang files (in src). All your Erlang modules are therefore available in your search path.

Mix Project Settings

Some settings in mix.exs like description and lcenses are already defined in src/[NAME].app.src. Instead of repeating the same values and possibly having diverging values, I recommend reading the Erlang application settings in your mix.exs:

{:ok, [{:application, :app_name, props}]} = :file.consult(~c"src/[NAME].app.src")
@props Keyword.take(props, [:description, :env, :mod, :licenses, :vsn])

You can then replace the staic values like the version with the dynamically loaded ones in various places.

Dependencies

Runtime dependencies of your application will have to be listed in each managers configuration. Make sure to put the same version requirements to both.

Test / Dev Dependencies can be added separately and only have to be added in the context they re used in. If for example meck is used in EUnit tests and mock in ExUnit tests, meck only needs to be in the rebar.config and mock in the mix.exs.

Documentation using ex_doc

Unfortunately as of Nov 2023, Erlang documentation and Elixir documentation do not work entirely the same. Elixir will include documentation information directly in the compiled module. Erlang does not include the information, but provides the functionality to write the documentation into files so that it can be picked up by ex_doc.

To add bot Elixir & Erlang modules into the same documentation, your mix.exs will have to be adjusted slightly to make it work:

• Add aliases: [docs: ["compile", &edoc_chunks/1, "docs"]] to your mix.exs / project
• Add the following function to your mix.exs:

  defp edoc_chunks(_args) do
    base_path = Path.dirname(__ENV__.file)
    doc_chunk_path = Application.app_dir(:app_name, "doc")
  
    :ok =
      :edoc.application(:app_name, String.to_charlist(base_path),
        doclet: :edoc_doclet_chunks,
        layout: :edoc_layout_chunks,
        preprocess: true,
        dir: String.to_charlist(doc_chunk_path)
      )
  end

When you're now calling mix docs, the alias will first generate the erlang documentation chunks and then generate the documentation for both languages.

Test Coverage

EUnit, Common Test and ExUnit can all emit a test coverage. To combine all of them into one, you can use the following steps:

• rebar3 eunit --cover --cover_export_name eunit
• rebar3 ct --cover --cover_export_name ct
• mix test --cover --export-coverage mix_test
• cp _build/test/cover/{eunit,ct}.coverdata cover/
• mix test.coverage

Publishing

Rebar3, Mix & Hex integration

A package can be published to Hex with multiple supported managers. To do so, the project should include configuration files for each supported manager.

When publishing, one manager has to be chosen to do the actual publishing. All other managers files will be included in the package, but are not part of the actual publication command invocation.

I recommend using mix for this.

Setup:

• Add package to mix.exs / project

  [
    build_tools: ["rebar3", "mix"],
    files: [
      "include",
      "lib",
      "LICENSE*",
      "mix.exs",
      "README*",
      "rebar.config",
      "src"
    ],
    licenses: ["YOUR_LICENSE"],
    links: %{"GitHub" => "https://github.com/organisation/repo"}
  ]

When executing mix hex.publish, your package should now be published as normal. You should see both rebar3 and mix listed in the Build Tools section on hex.pm.

Working Example

The described methods were used to publish oidcc. Have a look to find all the described steps implemented.