Currently the implementation of is_record(data)
is the original instance of code in Elixir core that we want defguard
to generate, so it should be a good thing to tackle first. If we can use is_record(data)
in a custom guard generated by defguard
we're pretty much done. In fact, we can already generate the correct AST just fine in Macro.guard/2
, but we need to raise errors if we happen to know the expansion of is_record(data)
isn't valid for use in guards. To do this we need to implement the blank functions above:
Macro.expand_guard(Macro.t, Macro.Env.t) :: Macro.t
to expand it until it can be validatedMacro.guard_validity_check(Macro.t) :: boolean
to walk through the expanded ast making sure it all looks good.
Ideally, the implementations will be in terms of calls to :erlang
and :elixir
so that it evolves with the languages. That is, we want to ask erlang and Elixir what's allowed in a guard, instead of just telling the user what's allowed in guards.
This is non-trivial, especially the second one. So we'll do the expansion first.
Let's convince ourselves that we can't progress any further as is. We'll force the calling env to report as actually Macro.Env.in_guard?
, but ignore it for the raw ast.
expand_guard = fn (ast, _env) -> ast end
import Record
env = Map.put(__ENV__, :context, :guard)
expand_guard.((quote do: is_record(data)), env)
#=> {:is_record, [context: Elixir, import: Record], [{:data, [], Elixir}]}
IO.puts Macro.to_string expand_guard.((quote do: is_record(data)), env)
#=> is_record(data)
Obviously we lack the information about is_record
to write a guard_validity_check
function.
Our options for expansion are:
Macro.expand_once(ast, env)
Macro.expand(ast, env)
Macro.prewalk(ast, &(Macro.expand &1, env))
The first will handle a top level AST transformations: imports, aliases, the like. The second will iteratively perform all possible transformation on the top-level piece of AST. The final one will do that, recursively, through the whole AST. This is the one we want.
expand_guard = fn (ast, env) -> Macro.prewalk(ast, &(Macro.expand &1, env)) end
import Record
env = Map.put(__ENV__, :context, :guard)
expand_guard.((quote do: is_record(data)), env)
#=> {{:., [], [:erlang, :andalso]}, [],
#=> [{{:., [], [:erlang, :andalso]}, [],
#=> [{:is_tuple, [context: Record, import: Kernel], [{:data, [], Elixir}]},
#=> {:>, [context: Record, import: Kernel],
#=> [{:tuple_size, [context: Record, import: Kernel], [{:data, [], Elixir}]}, 0]}]},
#=> {:is_atom, [context: Record, import: Kernel],
#=> [{:elem, [context: Record, import: Kernel], [{:data, [], Elixir}, 0]}]}]}
IO.puts Macro.to_string expand_guard.((quote do: is_record(data)), env)
#=> :erlang.andalso(:erlang.andalso(is_tuple(data), tuple_size(data) > 0), is_atom(elem(data, 0)))
That looks fun! And it gives us our penultimate piece of the puzzle: how to expand the guard.
@doc false
# Expands a guard `expr` in `env` for evaluation in `Macro.validate_guard/1`.
@spec expand_guard(Macro.t, Macro.Env.t) :: Macro.t
defp expand_guard(expr, env) do
Macro.prewalk(ast, &(Macro.expand &1, Map.put(env, :context, :guard)))
end
But how do we tell if it's production ready? What is our Guard.guard_validity_check(Macro.t) :: bool
? How can we walk through each segment of the ast and know if it's valid, before someone tries to use our guard?
From here on out we'll assume that the variable full_ast
is set to:
import Record
full_ast = Macro.prewalk((quote do: Record.is_record(data)), &(Macro.expand &1, Map.put(__ENV__, :context, :guard)))
#=> {{:., [], [:erlang, :andalso]}, [],
#=> [{{:., [], [:erlang, :andalso]}, [],
#=> [{:is_tuple, [context: Record, import: Kernel], [{:data, [], Elixir}]},
#=> {:>, [context: Record, import: Kernel],
#=> [{:tuple_size, [context: Record, import: Kernel], [{:data, [], Elixir}]}, 0]}]},
#=> {:is_atom, [context: Record, import: Kernel],
#=> [{:elem, [context: Record, import: Kernel], [{:data, [], Elixir}, 0]}]}]}
We can see what the Erlang Expressions guide says is valid in guards:
- The atom true
- Other constants (terms and bound variables), all regarded as false
- Calls to the BIFs specified in table Type Test BIFs below
- Term comparisons
- Arithmetic expressions
- Boolean expressions
- Short-circuit expressions (andalso/orelse)
Lets build up a series of function clauses that will tell if any piece of AST fits these categories.
These are said to be valid in guards:
- The atom true
- Other constants (terms and bound variables), all regarded as false
We can isolate terms through our Type Test BIFs and what we know about the AST: anything that isn't a tuple in a Macro.prewalk
should be good.
term when not is_tuple(term) -> true
That doesn't cover variables, though.
- Other constants (terms and bound variables), all regarded as false
When we generate our macro through Guard.guard/2
we're effectively eliminating unbound variables, that's the whole point of this. Any other unbound variables will show up as issue when that macro is compiled. That leaves us with bound variables. These are easy to cover, take our data
example in full_ast
from is_record(data)
:
{:data, [], Elixir}
{name, meta, atom} when is_atom(atom)
For our purposes any remaining bound variable reference is valid in a guard, even if the compiler chokes on it later:
{_, _, atom} when is_atom(atom) -> true
Let's look at our root expression in full_ast
next:
{{:., [], [:erlang, :andalso]}, [], [...]}
{{:., dotmeta, [module, fun]}, meta, args} when is_list(args)
How do we decide if a remote call is valid? In a guard, they generally aren't, except for ones to :erlang
mentioned in the Erlang Expressions guide. So we can target a subset of remote calls:
{{:., _, [:erlang, call]}, _, args} when is_list(args)
Frustratingly, :erl_internal
almost gives us all the tools we need to evaluate these.
:erl_internal.op_type
will help us find the candidates mentioned here:
- Term comparisons:
:erl_internal.comp_op/2
- Arithmetic expressions:
:erl_internal.arith_op/2
- Boolean expressions
:erl_internal.bool_op/2
The docs fail to mention the list expressions we can find in :erl_internal.list_op/2
. And even the documentation for :erl_internal
doesn't make it clear that there's a 5th op category, send operations, which are not allowed in guards.
Even with all that, we haven't covered the example from full_ast
, which :erl_internal
can't help us with:
- Short-circuit expressions (andalso/orelse)
As it turns out, this is already taken care of for us in :elixir_utils.guard_op/2
:
guard_op('andalso', 2) ->
true;
guard_op('orelse', 2) ->
true;
guard_op(Op, Arity) ->
try erl_internal:op_type(Op, Arity) of
arith -> true;
list -> true;
comp -> true;
bool -> true;
send -> false
catch
_:_ -> false
end.
This leaves just one category left from the guard expressions docs not accounted for, which we can use :erl_internal.guard_bif/2
for:
This gives us two clauses, one to determine if a remote call is valid:
{{:., _, [:erlang, call]}, _, args} when is_list(args) -> :erl_internal.guard_bif(call, length(args)) or :elixir_utils.guard_op(call, length(args))
And one to let remote call heads through successfully, since their validity is determined at a higher level:
{:., _, [:erlang, _call]} -> true
We now have a full specification of what erlang knows is good in guards:
@doc """
Determines if an expanded guard `expr` is valid.
"""
@spec validate_guard(Macro.t) :: :ok | {:error, term}
def validate_guard(expr) do
validate expr, fn
# handle good remote calls to :erlang
{{:., _, [:erlang, call]}, _, args} when is_list(args)
-> :erl_internal.guard_bif(call, length(args)) or :elixir_utils.guard_op(call, length(args))
# and their constituent call subexprs, if they're not valid they'll be caught during the walk above
{:., _, [:erlang, _call]}
-> true
# let variables fly, the compiler will complain after this is all over if they're really no good
{_, _, atom} when is_atom(atom)
-> true
# let general terms through for further iterations
term when not is_tuple(term)
-> true
# anything else is invalid
_ -> false
end
end
We should be finished, right? Not quite.
So far things are going pretty well. We're dipping into internal :erl_internal
and :elixir_utils
, but really that's a good thing--if any of our assumptions about AST change, they will be kept up to date as these modules will change upstream. No sketchy hard-coded AST checks, no error trying and catching! But there's still a big issue.
Even though we've expanded our AST, and encoded whether or not each element is valid in guards, most elements of our test case fail:
import Record
full_ast = Macro.prewalk((quote do: is_record(data)), &(Macro.expand &1, Map.put(__ENV__, :context, :guard)))
#=> {{:., [], [:erlang, :andalso]}, [],
#=> [{{:., [], [:erlang, :andalso]}, [],
#=> [{:is_tuple, [context: Record, import: Kernel], [{:data, [], Elixir}]},
#=> {:>, [context: Record, import: Kernel],
#=> [{:tuple_size, [context: Record, import: Kernel], [{:data, [], Elixir}]}, 0]}]},
#=> {:is_atom, [context: Record, import: Kernel],
#=> [{:elem, [context: Record, import: Kernel], [{:data, [], Elixir}, 0]}]}]}
Many of these terms are technically not valid according to erlang (ex. Elixir.Kernel.is_tuple/2
) because the Elixir compiler will rewrite that under the covers into :erlang.is_tuple/2
, which is pretty slick, but makes it hard to tell in userland if they are acceptible.
What we need is a way to expand these rewrites, so we need to teach Macro.expand/2
a new trick. Let's start by adding a flag to the expand family that will let us tell Elixir to handle rewrites:
module Guard do
@doc """
Macro.expand, with the option expand rewrites a la `rewrite: true`
"""
def expand(tree, env, opts \\ [])
def expand(tree, env, opts) do
expand_until({tree, true}, env, opts)
end
defp expand_until({tree, true}, env, opts) do
expand_until(do_expand_once(tree, env, opts), env, opts)
end
defp expand_until({tree, false}, _env, _opts) do
tree
end
def expand_once(ast, env, opts \\ [])
def expand_once(ast, env, opts) do
elem(do_expand_once(ast, env, opts), 0)
end
end
Again, we'll steal the tests from Macro.expand to ensure what we're doing doesn't break the previous behaviour.
I'm going to drop out of the markdown format and back into code files. First I'l post this updated spec file, then add the full and final code-commented version of our Guard spec.
You can sroll down to the bottom of zzz_guard.ex
until you see
##############
# Resume here!
##############
This is where we will address our implementation of defp Guard.do_expand_one(Macro.t, Macro.Env.t, Keyword.t) :: Macro.t
.