elixir-huff-part-1.md

The free lunch is over.

Eight years ago, Herb Sutter wrote a very compelling article, titled The Free Lunch is Over detailing how the major processors and architectures were turning to hypethreading and multicore.

The majorify of developers at the time, including myself, were all writing software designed to run on single cores machines. What those news meant to us was that, if we actually want our software to run faster in the upcoming years, we would have to effectively change the way we write software!

At the time (and still today), many developers were writing object oriented software, which focus on encapsulating state and changing this state. When it comes to multi-core software, mutating this state becomes trickier because two different cores may try to mutate the same state at the same time. Multi-core software requires concurrency and concurrency introduces the need for coordination.

We have all been there. Organizing a whole party on your own can be a lot of work. Organizing a party in a group is much more fun but without coordination it becomes a recipe for disaster, as we'd find out on the day of the party that nobody sent the invites because everyone assumed someone else has done so. The tools we use for coordination also affect directly the fun while organizing and the success of the party.

Multi-core programming is not much different. Concurrency can be a lot of fun, as long as we have the proper tools available to us. The issue is that, historically, those proper tools were not available in mainstream languages.

Today, eight years after the article, many things have changed. First off, hyperthreading and multicore have indeed become the reality. Even the cheapest CPUs today run on a multicore architecture, providing at least two cores. In fact, I am writing this article in a computer running on a Intel® Core™ i5, that has four cores, each of them with 2.80GHz. Fun fact: the 2.80GHz clock rate was reached commercially by Intel® back in 2002 with Pentium® 4.

Second of all, we have seen a great growthy in the commercial use and search for functional programming languages, like F#, Scala, Clojure, Haskell and more. And the reason for such is because functional languages provides a better foundation for thinking concurrently and the proper tools for writing concurrent software.

This article is about one of those functional languages, called Elixir, that runs on the Erlang Virtual Machine.

Elixir and Concurrency

When writing Elixir software, we organize our code into tiny processes, let's see an example:

# You can put this snippet in a file named: sample.exs
# And run it with: elixir sample.exs

# 1) Spawns a new process
ponger = Process.spawn(fn ->
  receive do
    { :ping, sender } ->
      sender <- :pong
  end
end)

# 2) Sends a :ping message to the new process
ponger <- { :ping, self() }

# 3) Receives a :pong message back
receive do
  :pong ->
    IO.puts "Got a pong back!"
after
  1000 ->
    IO.puts "One second passed and no pong :("
end

In the example above, we have created a new process using Process.spawn, passing a function, a block of code that will be executed in this new process. This block of code will simply wait for a :ping message to arrive. Once the message arrives, it sends a reply, labelled :pong.

After the process is created, we send it a message labelled :ping, also passing self(), which is the current process, in the message. Finally, the same process that sent the :ping waits for a :pong message back or prints a failure message if such message does not arrive in 1 seconds (1000 miliseconds).

Although a very simple example, this code snippet reflects a lot on how Elixir code works and runs:

1. Your code is organized into processes and creating a process is very cheap;
2. Those processes do not share any state, instead they communicate via messages;
3. As processes do not share any state, they run concurrently and use receive blocks for receiving messages (coordination);
4. Those processes are isolated. If something goes wrong in a particular process, other processes will continue running;
5. Processes can fail as well as the communication in between them. Using after inside a receive block allows us to react to those scenarios;

By not sharing any state and communicating via messages, processes are the building block for concurrency in Elixir, bringing the joy for building concurrent systems. This is so spread throughout the language and the ecossytem that it can be verified in your first experience with Elixir, which is installig Elixir itself. When installing Elixir, you should see all cores on your machines being used, the same for running the Elixir language test suite.

This abstraction deeply affects how we write software and none of this is novel. All those features are provided by the Erlang Virtual Machine, and both Erlang and Elixir languages leverage its power.

The Erlang Virtual Machine

Erlang is a language and a Virtual Machine developed by Ericsson. The first version developed internally at Ericsson in 1986 was designed with the aim of improving the development of telephony applications which have a very specific set of requirements:

1. Fault-tolerant: a telephony system should continue operating properly even in the event of the failure of some of its components;

2. Non-stop: such applications should work non-stop and avoid downtimes. In fact, it is possible to perofrm hot-code swaps, which allows developers to upgrade code in production without shutting off the system;

3. Distributed: different machines in the same telephone system should be able to easily exchange information in between them;

Let's see how those three charactistics maps to the processes we have just discussed:

1. Fault-tolerant: our code is built with processes that are isolated and if a process fails, other processes in the system can continue running. Furthermore, we often define supervisors, which is a particular kind of processes that watch over other proceses and restart them when something goes wrong. In fact, "let it crash" is a common philosphy when writing software for the Erlang VM, since supervisors are able to restart processes so they can go back to a known, working state;

2. Non-stop: processes do not share any state, instead each process has its own state and information is exchanged via message passing. This means that, if a process is acting poorly, we can update its code in production (and its state), without affecting the remaining of the system as long as we keep sending and receiving such messages properly. If instead processes shared state, such code upgrades would be much harder because we would need to consider how changing the state would affect all other components in the system;

3. Distributed: because the system is designed around message passing, the Erlang VM abstracts the message passing in a such a way that it doesn't matter if the recipient of the message is in the same machine or in another node in the same network;

In other words, while I have lured you into this article with promises of concurrency, we are actually getting much more than concurrency. We are getting a whole framework for building fault-tolerant, concurrent and distributed applications! And all this made available by the Erlang VM.

Everything I have talked so far applies to all languages that run on the Erlang VM. So, why Elixir? We [explore what Elixir brings to the Erlang VM and discuss Elixir's goals in the second part of this article](LINK TO SECOND PART OF THE ARTICLE).

elixir-huff-part-2.md

In the [first part of this article](LINK TO THE FIRST PART OF THE ARTICLE), we have discussed the need for concurrency in today's software and how the Erlang VM not only provides the proper tooling and abstractions for writing concurrent software but also for building distributed, fault-tolerant applications.

In this second part, we will explore what Elixir brings to the table and its main goals.

Why Elixir?

The better way to answer this question is to look at Elixir's three main goals: productivity, extensibility and compatibility.

Productivity is, in general, a hard goal to measure. A language productive for creating desktop applications may not be productive for mathematical computing. Productivity depends directly on the field in which you intend to use the language, the available tools in the ecosystem and how easy it is to create and extend those tools.

For example, Elixir attempts to eliminate many steps in between starting with the language and writing production ready code. That's why Elixir ships with a unit test framework named ExUnit and build called Mix. With simple three commands, you can create a complete Elixir application, including a supervisor chain, compile and test it:

$ mix new sample
* creating README.md
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/sample.ex
* creating lib/sample
* creating lib/sample/supervisor.ex
* creating test
* creating test/test_helper.exs
* creating test/sample_test.exs

$ cd sample

$ mix test
Compiled lib/sample/supervisor.ex
Compiled lib/sample.ex
Generated sample.app
.

Finished in 0.06 seconds (0.06s on load, 0.00s on tests)
1 tests, 0 failures

Our second goal, extensibility is directly tied to productivity. To quote Guy Steele:

Now we need to go meta. We should now think of a language design as being a pattern for language designs. A tool for making more tools of the same kind. [...] A language design can no longer be a thing. It must be a pattern, a pattern for growth. A pattern for growing a pattern, for defining the patterns that programmers can use for their real work and main goals.

• Guy Steele, keynote at the 1998 ACM OOPSLA conference on "Growing a Language"

For this reason, we have opted for a small language core. For example, while some languages have if, case, try and so on as language keywords, each with its own rules in the parser, in Elixir they are just macros. This allows us to implement most of Elixir in Elixir and also allows developers to extend the language using the same tools we used to build the language itself, often extending the language to the specific domains they are working on.

Here is an example of how someone would implement unless, which is a keyword in many languages, in Elixir:

defmacro unless(expr, opts) do
  quote do
    if(!unquote(expr), unquote(opts))
  end
end

unless true do
  IO.puts "this will never be seen"
end

Since a macro receives the code representation as arguments, we can simply convert an unless into an if at compile time.

Macros are also the base construct for meta-programming in Elixir: the ability to write code that generates code. Meta-programming allows developers to easily get rid of boilerplate and create powerful tools. ExUnit, that test framework that ships with Elixir, uses macros for expressiveness. Let's see an example:

ExUnit.start

defmodule MathTest do
  use ExUnit.Case, async: true

  test "adding two numbers" do
    assert 1 + 2 == 4
  end
end

The first thing to notice is the async: true option. When your tests do not have any side-effects, you can run them concurrently by passing the async: true option.

Next we define a test case and we do an assertion with the assert macro. Writing tests by simply using assert would be a bad practice in many languages as it would provide a poor error report. In such languages, functions/methods like assertEqual or assert_equal would be the recommended way of performing such assertion.

In Elixir, however, assert is a macro and as such it can look into the code being asserted and infer that a comparison is being made. This code is then transformed to provide a detailed error report when the test runs:

1) test adding two numbers (MathTest)
   ** (ExUnit.ExpectationError)
                expected: 3
     to be equal to (==): 4
   at test.exs:7

This simple example illustrates how a developer can leverage macros to provide a concise but powerful API. Macros have access to the whole compilation environment, being able to check the imported functions, macros, defined variables and more. And those examples are just scratching the surface of what can be achieved with macros in Elixir.

Finally, the last goal is compatibility. Compatibility is about being compatible with the Erlang VM and the whole existing ecossystem, which allows developers to benefit from all the features we have described in the previous section. Elixir can call Erlang code, and vice-versa, with no conversion cost at all.

Elixir and the Erlang VM are here to change the way you write software and prepare you for the next years in computing. By using Elixir, you can join open source projects like CouchDB, Riak, RabbitMQ, Chef11 and companies like Ericsson, Heroku, Facebook, Basho, Klarna and Wooga which already enjoying the benefits provided by the Erlang VM, some of them for quite a long time.

So give Elixir a try! We have a getting started guide online, books being written (Programming Elixir and Introducing Elixir), plus many others community resources.