A "Hello, World!" of Lisp macros.

hello-lisp-macros.md

"Hello, Macros!"

A "Hello, World!" of Lisp macros

tl;dr

(defmacro pass-args (code &rest args)
  `(,@code ,@args))

> (macroexpand '(pass-args (format) t "Hello, World!"))
(FORMAT T "Hello, World!")
T

> (pass-args (format) t "Hello, World!")
Hello, World!
NIL

> (pass-args (+) 1 2 3)
6
> (pass-args (-) 1 2 3)
-4
> (pass-args (*) 1 2 3)
6
> (pass-args (/) 1 2 3)
1/6
> (pass-args (list) 1 2 3)
(1 2 3)

About

As a Lisp beginner, I think the most succinct program capable of portraying the beauty of Lisp is (+). I first saw this expression in Paul Graham's ANSI Common Lisp (page 8) [1], and despite its apparent simplicity I found it shockingly powerful. In this gist, I use the (+) expression to explore the notion of "code as data", and try to figure out what the "Hello, World!" of Lisp macros is, a challenge proposed by the book's author in this tweet.

Let's dive in

The (quote ...) operator

The key to understanding Lisp macros lies in the concept of protecting expressions from evaluation. Lisp programs are expressed as lists, and programs are executed by evaluating their list representations. Protecting lists from evaluation means doing something to avoid their interpretation as programs (i.e. an operator followed by its arguments), or simply to prevent their execution when they actually are programs.

The quote operator (abbreviated as ') protects its arguments from evaluation. Clearly this is useful for building lists of data without triggering the evaluation rule of function calls, but why would we want to prevent actual programs from being evaluated as such? In Lisp, the answer is straightforward: so that we can manipulate programs as data and transform them into other programs. This is where macros come into play. But first, let's go back to (+).

The (+) program

The (+ 1 2 3) expression is a list representation of a Lisp program that adds the numbers 1 to 3 together. It uses prefix notation to express that the arguments 1, 2 and 3 should be passed to the + function.

> (+ 1 2 3)
6

This notation makes the + operator appear only once, whereas infix notation would make it appear twice (1 + 2 + 3). The elegance of this approach is that it holds for an arbitrary number of arguments (e.g. (+ 1 2 3 4 5 ...)), but what I find most surprising is that it holds for zero arguments as well. We can thus write a program that adds zero numbers together:

> (+)
0

This is so beautiful, and even more so that it also holds for (*), (and) and (or). Admittedly, a program that adds zero numbers together is not useful at all, unless we can somehow modify it to take more arguments before it is evaluated. One way of doing this modification is by appending expressions to the list representation of a program. Let's try to use the built-in append function to concatenate the (+) program with the list (1 2 3):

> (append (+) (list 1 2 3))
           |
  (append  0  (list 1 2 3))
...TYPE-ERROR...
The value 0 is not of type LIST

Unfortunately, trying to evaluate this expression causes an error, because arguments passed to function calls are always evaluated. Thus the (+) program will evaluate to 0, but the append function only takes lists as arguments. What we really need to do is concatenate arguments to the (+) list.

The '(+) list

Let's try to protect the (+) program from evaluation before passing it to the append function. We can do so with the quote operator (here in abbreviated form):

> '(+)
(+)

The result of preventing the evaluation of the (+) program is the (+) list. Even though they look exactly the same, the former is code, while the latter is data. This is the distinctive notion of "code as data" in Lisp. We could now use the append function to concatenate the lists (+) and (1 2 3):

> (append '(+) '(1 2 3))
(+ 1 2 3)

Notice that both arguments are written with the ' operator, so as to prevent the evaluation of the (+) expression as a program (which would return 0), as well as the evaluation of the (1 2 3) expression as a function call (which would cause an error because 1 is not a function). However, the resulting expression (+ 1 2 3) is data, not code, so we still need to find a way to make Lisp treat it as code. Would defining a function solve this problem?

(defun pass-args (code &rest args)
  (append code args))

> (pass-args '(+) 1 2 3)
(+ 1 2 3)

Unfortunately not. This pass-args function evaluates the expression (append code args), where the code parameter must be a list representation of a program, and &rest args will build a list of zero or more arguments to be passed to code. But whether we call the append function directly or define a function that calls append internally, the final expression still needs to be evaluated as code. What else could we try then? Enter (eval ...):

> (eval (append '(+) '(1 2 3)))
6
> (eval (pass-args '(+) 1 2 3))
6

These expressions finally produce the desired result. However, calling eval is not the best way to cross the line between data and code [1] (page 161) mainly due to efficiency reasons (run-time vs compile-time). As we shall see next, macros are in fact the best way to manipulate code as data and evaluate data as code.

Hello, (defmacro ...)!

Macros are programs that write programs [2]. They do so by handling programs as data and evaluating their transformed representations as code. This is possible because, unlike functions, macros do not evaluate their arguments when they are passed. Thus, in order to modify the (+) program, we can redefine the pass-args function as a macro and call it with an unquoted expression of the (+) program:

(defmacro pass-args (code &rest args)
  (append code args))
  
> (pass-args (+) 1 2 3)
6

We finally get 6 instead of just (+ 1 2 3). Notice that, in contrast to the pass-args function call, there is no need to use the quote ' operator before the (+) program because macros operate on the unevaluated expressions for the arguments [3]. But once inside the body of a macro, we do need a way to specify that we want to treat some data as code again before returning the value of the transformed expression.

The backquote ` operator turns evaluation off like the regular quote ' operator, but we can use , and ,@ within a backquoted expression to turn evaluation back on [1] (page 163). The ,@ is especially useful because it inserts the elements of a list into a template, so given a term ,@args, if args is the list (1 2 3), then its elements 1 2 3 will be inserted without the enclosing parentheses. The same would apply to a term ,@code where code is (+), only the + operator would be inserted. This allows us to rewrite the pass-args macro without the append function:

(defmacro pass-args (code &rest args)
  `(,@code ,@args))

> (pass-args (+) 1 2 3)
6

Here too we get the expected result, but something different happened under the hood: a process known as macro expansion, which turns the template (,@code ,@args) into actual code by inserting the required argument expressions. We can visualize the result of this process by applying the macroexpand function to a quoted expression containing the macro call that we want to expand:

> (macroexpand '(pass-args (+) 1 2 3))
(+ 1 2 3)
T

> (macroexpand '(pass-args (+ 1) 2 3))
(+ 1 2 3)
T

This is extremely useful when designing macros, since it allows us to see what code will be generated when passing specific arguments to a macro call. Notice that both expansions generate the same expression (+ 1 2 3), even though the former passes the 1 2 3 arguments to the (+) program, while the latter passes the 2 3 arguments to the (+ 1) program. Therefore, both macro calls will ultimately evaluate to 6 as expected.

Hello, World!

Now let's use the pass-args macro and the format function to finally write a "Hello, World!" of Lisp macros:

> (macroexpand '(pass-args (format) t "Hello, World!"))
(FORMAT T "Hello, World!")
T

> (pass-args (format) t "Hello, World!")
Hello, World!
NIL

This macro call passes the arguments t and "Hello, World!" to the (format) program, which expands to (format t "Hello, World!"). Then it prints the string Hello, World! and finally returns NIL. Notice that the (format) program is not a valid Lisp expression because format is a two-argument function. Thus, unlike the (+) program that could be executed with no arguments, trying to evaluate (format) causes an error:

> (format)
...ERROR...
invalid number of arguments: 0

This means that the pass-args macro is also useful for modiying programs that would otherwise fail to evaluate. Trying to evaluate the (format t) expression would fail too, but if we pass it to the macro along with the "Hello, World" argument, it will expand to the same valid expression as the (format) program with the t and "Hello , World" arguments, and thus produce the same results:

> (macroexpand '(pass-args (format t) "Hello, World!"))
(FORMAT T "Hello, World!")
T

> (pass-args (format t) "Hello, World!")
Hello, World!
NIL

And that's it! A "Hello, World!" of Lisp macros.

Goodbye, reader!

Of course there is much more to macros than just appending arguments to a list representation of a program, but I just started learning Lisp and wanted to share the thinking process that helped me understand this feature of the language. Hopefully this will help other learners see the power that comes from the ability to pass code as arguments, transform it as data and finally treat it as code again. This is what makes Lisp the ultimate programmable programming language.

References

1. ANSI Common Lisp (chapters 1-3, 10).
2. Beating the Averages.
3. GNU Emacs Manual