introducing_with.md

Introducing with

with is for's younger brother. Imagine we have two functions:

def ok(x), do: {:ok, x}
def error(x), do: {:error, x}

While for is used to match on values out of a collection:

for {:ok, x} <- [ok(1), error(2), ok(3)], do: x
#=> [1, 3]

with is used to match on values directly:

with {:ok, x} <- ok(1),
     {:ok, y} <- ok(2),
     do: {:ok, x + y}
#=> {:ok, 3}

Because all values matched, the do block was executed, returning its result. If a value does not match, it will abort the chain:

with {:ok, x} <- ok(1),
     {:ok, y} <- error(2),
     do: {:ok, x + y}
#=> {:error, 2}

Since error(2) did not match {:ok, y}, the with chain aborted, returning {:error, 2}.

There are many different scenarios on every day Elixir code that we can use with. For example, it is useful to avoid nesting "case"s:

case File.read(path) do
  {:ok, binary} ->
    case :beam_lib.chunks(binary, :abstract_code) do
      {:ok, data} ->
        {:ok, wrap(data)}
      error ->
        error
    end
  error ->
    error
end

Can now be rewritten as:

with {:ok, binary} <- File.read(path),
     {:ok, data} <- :beam_lib.chunks(binary, :abstract_code),
     do: {:ok, wrap(data)}

Another example is Plug itself. Plug will only call the next plug if the :halted field in the connection is false. Therefore a whole plug pipeline can be written as:

with %{halted: false} = conn <- plug1(conn, opts1),
     %{halted: false} = conn <- plug2(conn, opts2),
     %{halted: false} = conn <- plug3(conn, opts3),
     do: conn

If any of them does not match, because :halted is true, the pipeline is aborted.

Similarly to for, variables bound inside with won't leak. Also similar to for, with allows "bare expressions". For example, imagine you need to calculate a value before calling the next match, you may write:

with {:ok, binary} <- File.read(path),
     header = parse_header(binary),
     {:ok, data} <- :beam_lib.chunks(header, :abstract_code),
     do: {:ok, wrap(data)}

That's pretty much the gist of it. Thoughts?

FAQ

Q: Why with instead of one of the operators proposed on other discussions?

The operators in the other discussions were always bound to some particular shape. For example, it would always match on {:ok, _}, which is limitting. with explicitly lays out the pattern, which means you can match on any value you want.

Q: Will with work with the pipe operator?

The answer is no. with, for, the pipe operator are implementations of different monads and we know from other comunities monads do not compose well. Even without a type system, the only way to introduce with that works with pipe is by defining something like withpipe, and maybe withfor, which obviously wouldn't scale because there are too many combinations. Furthermore, we really appreciate that with makes all the patterns explicit, instead of hiding it behind a pipe operator. with also gives us full control on how the arguments are forwarded.

José goes crazy

To mimic my ElixirConf keynotes, where the talk starts with reasonable proposals and ends-up with me babbling stuff that may not ever see the day of light, let's do it a bit in written form too. :)

In 2014, I had proposed stream for and parallel for support alongside for comprehensions. Could we have similar modifiers for with too? I am glad you asked!

We could at least introduce async with:

async with part1 <- fetch_part("foo"), # async
           part2 <- fetch_part("bar"), # async
           {:ok, page1} <- fetch_page(part2), # await part2, async
           {:ok, page2} <- fetch_page(part1), # await part1, async
           do: {page1, page2}

The right side of <- is always executed inside a new task. As soon as any of the parts finish, the task that depends on the previous one will be resolved. In other words, Elixir will solve the dependency graph for us and write this in the most performant way as possible. It will also ensure that, if a clause does not match, any running task is cancelled.

That's not the only example. Ecto could introduce transactional with, where we wrap the whole with chunk in a transaction and rollback if it does not match. Imagine you want to introduce a User model followed by a Log, here is how it would be written today:

user_changeset = User.changeset(%User{}, params)
Repo.transaction fn ->
  case Repo.insert(user_changeset) do
    {:ok, user} ->
      log_changeset = Log.changeset(%Log{user_id: user.id}, params)
      case Repo.insert(log_changeset) do
        {:ok, log} -> {user, log}
        {:error, log_changeset} -> Repo.rollback log_changeset
      end
    {:error, user_changeset} ->
      Repo.rollback user_changeset
end

Now with transactional with:

user_changeset = User.changeset(%User{}, params)
transactional with user <- Repo.insert(user_changeset),
                   log_changeset = Log.changeset(%Log{user_id: user.id}, params),
                   log <- Repo.insert(log_changeset),
                   do: {user, log}

Note: we can't use transaction as a name because it is already part of the Ecto.Repo API. If someone knows a shorter word than transactional, I would appreciate it. Please send me a personal e-mail. :)

You can see a discussion about this Ecto example in particular here: elixir-ecto/ecto#1009. Of course, there is a lot of prior art on all of this, but those last two examples, async and transactional, are heavily inspired by computation expressions from F#.

While this "José goes crazy" section is meant to highlight how we could extend this feature, let's focus on the first part of this issue: the barebone with. It is definitely useful enough to stand on its own.

** REMINDER: KEEP THE DISCUSSION IN THE MAILING LIST **

Link to the Discussion on the mailing list.

But why we can only use variables mentioned in with block only

with ... do
 # here_only 
else 
#why not here
end 

How we can mitigate this?