Eugene (>):
Hey Carl,
Just read your blog post about quasiquotes in Perl. Got some questions:
Cool! I had a look through your work on Scala's macros last time you wrote. Quite probably your email today will trigger another exploration.
- Why do you need different AST flavors? How do they differ?
This is the deepest of your four questions, and one I do not answer fully in the post. It's directly connected to the problem that took me months to solve. I will attempt to give a satisfactory answer here. Feel free to follow up with more questions if anything is unclear.
A macro definition must out of necessity come textually before a corresponding macro invocation. Even so macro, arguments are built very "early" in the compile-run cycle, whereas quasi ASTs are built relatively "late". This is because as a macro is invoked, the parser essentially pauses and allows the runtime to execute the macro routine, in what is called BEGIN time in Perl. (BEGIN time is any runtime activity which happens ASAP during parsing.)
So this is what distinguishes quasi ASTs from argument ASTs: the different kinds of environments in which they are constructed. Quasi ASTs have a full runtime context to adopt as theirs, whereas the argument ASTs only have the static compilation context that exists during parsing.
Here's an example to highlight the difference between static lexpads and runtime lexpads.
<masak> r: my $i = 42; BEGIN { say $i }; say $i
<p6eval> rakudo 170c90: OUTPUT«Any()42»
Since BEGIN runs ASAP during parsing, you're seeing "Any()" which is a lookup of $i in the static lexpad. Parsing finishes, the program as such starts running, the assignment happens, and the second say gives "42" back from the runtime lexpad.
Things involving the static lexpads don't happen a lot... except when you're mucking around running code ASAP, like BEGIN blocks and macros. Another case where it happens is role composition. At the time a role is composed into a class, which happens during parsing, the role block runs, and risks binding things to the static environment, which is the only one that exists during parsing.
<masak> r: my $x = 42; role R { method foo { $x } }; class C is R {}; say C.new.foo
<p6eval> rakudo 7c0820: OUTPUT«42»
(Editor's note: meant to do does R above, not is R.)
At the time R is composed into C, there's only the static lexpad containing $x with the value Any. Generally these things just sort themselves out by ordinary closure cloning by the runtime, but it doesn't here, because C has forced R to evaluate during parsing. Rakudo makes sure things like R.foo get the right scope eventually by inserting "fixups", code that runs whenever the block containing the declaration of R (the mainline block, in this case) is entered.
Now, the components of the above challenging situation are exactly the ones we find when expanding a macro: BEGIN-time evaluation of code, forced binding of static contexts, and a wish to re-bind later to the actual runtime contexts.
<masak> r: my $x = 42; macro foo($value) { quasi { say {{{$value}}} } }; foo $x
<p6eval> rakudo 7c0820: OUTPUT«42»
{{{$value}}} expands to $x, yes, but it needs to expand to $x with the context being the mainline. The only such context that exists at the time of parsing is the static context, in which $x is Any. So we need the context to be re-bound to the mainline runtime context as soon as it exists, because that's where the 42 is.
So that's why quasi ASTs and argument ASTs are different and why where they're coming from matters quite a bit. The latter have to worry about static contexts.
(I'm still a bit shocked at the simplicity of the eventual solution: we simply dump a fully-formed closure into the argument AST, along with an instruction to call it. The closure gets automatically cloned by regular mechanisms, and so the context ends up being the right one without any magic.)
- Can "quasi" blocks be written outside of macros? E.g. in helper functions? Does hygiene still work then?
Yes and yes.
<masak> r: sub foo { my $greeting = "OH HAI"; quasi { say $greeting } }; BEGIN my $gast = foo; macro bar { $gast }; bar()
<p6eval> rakudo 7c0820: OUTPUT«OH HAI»
(Code is unnecessarily convoluted because doing it in the obvious, simpler way triggered a bug I wasn't aware of until now.)
- What do you think about making "quasi" a macro that expands into an AST?
The question makes little sense to me; we're probably coming from different directions here.
Perhaps you are assuming more metacircularity than we currently have in Rakudo. The compiler provides macros in userland, but does not itself have them. Hence, quasis can not easily be implemented using macros.
- In the last code snippet, could a quasiquote refer to the $a which is local to the macro? E.g. via
quasi { $a }.
Yes, and that's definitely a feature. In that sense, the quasi is just a normal closure.
<masak> r: macro test { my $a = "OH HAI"; quasi { $a } }; say test
<p6eval> rakudo 170c90: OUTPUT«OH HAI»
Kindly,
// Carl