yloiseau/RFC-macros.md

## RFC-macros.md

      
    Raw
  

              RFC-macros.md
            
          
    RFC for macros in golo


Macros for Golo are becoming a reality 😄
See https://github.com/yloiseau/golo-lang/tree/wip/macro-quote
By the way, some choices need to be made, hence this RFC.
Syntax elements

Currently I have added several special syntax elements:


quote { golo code } is replaced in the Ir tree by CodeBuilder calls
that create a tree equivalent to the enclosed code. This allows functions
(macro) to easily return an Ir tree.


unquote(expr) or ~var that is the reverse of quote, and allows to
re-inject values in the tree built by quote


macro foo = |params| {...} to define a macro. Regular functions can do the work,
but it makes the compilation step easier. I'll try to get rid of this one since
this step is not dry yet.


&foo(args), &foo(args) { block } or &foo { block } to call the macro.
Hard to get rid of this one (see expansion step).


Definition step

A macros is just regular function that takes Ir nodes as arguments and returns
an Ir node (a new one or its argument, transformed or not).
The quote/unquote keywords allow to build easily simple Ir trees to
return. More complex trees can be build using the gololang.macros.CodeBuilder
module.
For instance, code like
function macroTest = {
  return quote {
    println(1 + 2)
  }
}
generates a Ir equivalent to:
function macroTest= {
  return functionInvocation()
          : name("println")
          : arg(
             binaryOperation(PLUS(), constant(1), constant(2))
          )
}
Since macro are just regular functions, hence static methods, they even can be
written in Java, provided the class is available to the compiler.
Currently, only statements can be quoted. It is not possible to write code like
return quote {
  function foo = |arg| { ... }
}
or
return quote {
  struct Foo = {bar, baz}
}
since in the grammar, the quote keyword is followed by a Block.
However, macros returning top-level statements (currently only functions, but
structs and augments are coming soon) can be written by using the
CodeBuilder facilities. For instance, let the macro in macros.golo
module TopLevelMacros

import gololang.macros.CodeBuilder
import gololang.macros.CodeBuilder$Operations

function myMacro = -> publicFunction()
  : name("foo")
  : param("x", "y")
  : block(
    returns(plus(refLookup("x"), refLookup("y")))
  )
called in test.golo like
module TopLevelTest

&TopLevelMacros.myMacro()

function main = |args| {
  println(foo(1, 2))
}
results in an Ir equivalent to:
module Test

function foo = |x, y| -> x + y

function main = |args| {
  println(foo(1, 2))
}
and golo compile macros.golo && golo golo --files test.golo prints 3 (this is actually working code 😄)
I don't plan to allow quote to contains such top-level statements (yet).
Compilation step

More work is needed here. Macro must be available at compile time
(see expansion step), thus they currently must be defined in a separate module
that must be compiled before the compilation of the module using it, and the
produced .class must be in the CLASSPATH (more precisely in
CLASSPATH_PREFIX) of the golo command when golo compile or golo golo
the module using it. That is why I introduced the macro keyword, to make a
macro only compilation step and inject the produced classes in the class loader
dynamically. Using this, macros will be compiled in a different class than normal
code before the compilation of the code itself.
For instance
module Foo

macro myMacro = |params| { ... }

function foo = {
  &myMacro(args)
}
will produce a Foo.Macros class containing only the myMacro method that
will be injected in the compiler class loader to then create the Foo class
containing only the foo method with the macro expanded. This is not currently
working yet.
However creating two modules macros.golo and test.golo containing
module Macros

function myMacro = |params| { ... }
and
module Test

import Macros

function foo = {
  &myMacro(args)
}

function main = |args| { ... }
respectively, and running it with
golo compile macros.golo && golo golo --files test.golo or
golo compile macros.golo test.golo && golo run --module Test
works. The order of file in the second command is relevant.
Maybe adding a command line option to explicitly add macro containing modules to the
compiler could also be useful.
See this other gist for more discussion on this aspect https://gist.github.com/yloiseau/8a8ce445cf9393239bec
Expansion step

When compiling “normal” code,  the Ir tree is walked by a visitor that:

look for macro invocation nodes (marked with the & prefix)
find the corresponding method and invoke it (thus the need for availability at compile time)
visit the result to expand macros recursively
replace the macro invocation node in the parent node by the result node

I'd like to add an other kind of macros, that take the expansion context as a
first argument, much like a method invocation takes the object itself as first
argument, e.g. the module itself for a top-level macro call. This would allow
the macro to modify directly the context, instead of just returning a node that
will replace the macro invocationi (think JS that change the DOM tree...)
the context parameter being automagically added to the call (just like
method call).
The question here is how to define this kind of macros:

just a regular macro with a special invocation mark (like method that are
just regular functions but called differently), for instance &:

macro foo = |context, arg1, arg2, block| { ... }

&:foo (a, b) { ... }

Pros: no special declaration, different use possible (normal call with an
explicit different context), possible to use existing functions.
Cons: unexpected result if called normally, nothing besides doc tells the
user that this macro must be called with special mark.

special declaration and normal invocation, e.g.

contextual macro foo = |context, arg1, arg2, block| { ... }

&foo (a, b) { ... }

Pros: compile time check of the presence of the fist parameter (like for
augment), can't be called without a context.
Cons: add another keyword, can't use regular function as macro.

everything implicit: the context parameter is not even specified in the
macro signature, and it is automagically added to the reference table when
calling the macro (similar to the this behavior in JS)

Pros: no special syntax.
Cons: explicit is better than implicit, the behavior may be difficult to
predict depending on the implementation (see JS this 😄).

everything explicit: the macro specify a parameter for the context, the
context is explicitly passed as argument in the macro call, e.g.

macro foo = |context, arg1, arg2, block| { ... }

&foo (__CONTEXT__, a, b) { ... }

Pros: everything is explicit.
Cons: more verbose, must provide a “special” variable that reference the
context, e.g. __CONTEXT__ here, which can be replaced dynamically by the Ir visitor,
that would be only a symbol (search for Ref{name=__CONTEXT__} and replace
it by the actual node).
I personally like the first solution, or a mix of 1 and 4. Any thoughts?
Some use cases

(all this is working code 😄)
Ever wanted a repeat until construct in Golo?
function repeatUntil = |cond, repeatBlock| -> quote {
  unquote(repeatBlock)
  while not ~cond {
    unquote(repeatBlock)
  }
}

&repeatUntil (a < 0) {
  println(a)
  a = a - 1
}
Ever wanted string interpolation in Golo?
function str = |s| {
  let matcher = Pattern.compile("\\$\\{([^}]+)\\}"): matcher(s: getValue())
  let pattern = constant(matcher: replaceAll("%s"))
  let func = functionInvocation(): name("String.format")
    : arg(pattern)
  matcher: reset()
  while (matcher: find()) {
    func: arg(refLookup(matcher: group(1)))
  }
  return func
}
function main = |args| {
  let a = 42
  let b = "world"
  println(&str("Hello ${b}, the answer is ${a}"))
}
(ping @danielpetisme)
To be done


pre-compilations step and class injection to allow “on the fly” macro
compilation when doing golo golo --files macros.golo test.golo
command line option to inject macro class in the compiler
top level macros to generate structs, augmentations and so on
special macro working directly on its context
tests, doc, code cleaning, predefined macros library, …