for!.md

Proposal for strict matching comprehensions

Background

for comprehensions are one of the most powerful features in Elixir. It supports both enumerable and bitstring generators, filters through boolean expressions and pattern matching, collectibles with :into and folding with :reduce.

One of the features are automatic filtering by patterns in generators:

list = [{:ok, 1}, {:ok, 2}, {:error, :fail}, {:ok, 4}]
for {:ok, num} <- list, do: num
#=> [1, 2, 4]

Generator filtering is very powerful because it allows you to succinctly filter out data that is not relevant to the comprehension in the same expression that you are generating elements out of your enumerable/bitstrings. But the implicit filtering can be dangerous because changes in the shape of the data will silently be removed which can cause hard to catch bugs.

The following example can show how this can be an issue when testing Posts.create/0. If a change causes the function to start returning {:ok, %Post{}} instead of the expected %Post{} the test will pass even though we have a bug.

test "create posts" do
  posts = Posts.create()
  for %Post{id: id} <- posts, do: assert is_integer(id)
end

The example uses a test to highlight the issue but it can just as well happen in production code, specially when refactoring in other parts of the code base than the comprehension.

Elixir is a dynamically typed language but dynamic typing errors are less of an issue compared to many other dynamic languages because we are usual strict in the data we accept by using pattern matching and guard functions. for is by design not strict on the shape of data it accepts and therefor loses the nice property of early failure on incorrect data.

Proposal

I propose an alternative comprehension macro called for! that has the same functionality as for but instead of filtering on patterns in generators it will raise a MatchError.

posts = [{:ok, %Post{}}]
for! %Post{id: id} <- posts, do: assert is_integer(id)
#=> ** (MatchError) no match of right hand side value: {:ok, %Post{}}

Pattern matching when not generating values with = remains unchanged.

for! gives the developer an option to be strict on the data it accepts instead of silently ignoring data that does not match.

Other considerations

You can get strict matching with for today by first assigning to a variable. This way you can also mix filtering and strict matching patterns.

posts = [{:ok, %Post{}}]
for post <- posts,
    %Post{id: id} = post,
    do: assert is_integer(id)
#=> ** (MatchError) no match of right hand side value: {:ok, %Post{}}

Another alternative is to introduce a new operator such as <<- (the actual token can be anything, <<- is only used as an example) for raising pattern matches instead of introducing a completely new macro.

posts = [{:ok, %Post{}}]
for %Post{id: id} <<- posts, do: assert is_integer(id)
#=> ** (MatchError) no match of right hand side value: {:ok, %Post{}}

A downside of adding new functions or macros is that it doesn't compose as well compared to adding options (or operators) to existing functions. If we want to add another variant of comprehensions in the future we might be in the position that we need 4 macros, and then 8 and so on.

Another benefit of adding an operator is that you can mix both <- and <<- in a single comprehension.

The downside of an operator is that it adds more complexity for the language user. We would also need an operator that is visually close to <- but still distinctive enough that they are easy to separate since their behavior are very difference.