Some thoughts on how to do native runtime closures in Scopes in a transparent, yet controllable way:
While we're typechecking a function, the function is eventually accessing a runtime variable outside its function scope. Currently, I'm detecting that already, but just to produce a compiler error. But it could be allowed, and a environment record could be built as the function is typed, of all the expressions outside of its boundary that need to be put into it.
The only way we're going to get these expressions into the function is through an environment parameter of tuple type. So we're not just recording unbound variables, we also rewrite their access; a new equivalent value is added to the environment tuple, and the environment tuple entry is extracted instead. We also record in a map which exterior values need to be mapped to which member in the environment.
When we're done typing the function, we have a complete record of what exterior values need to be captured, i.e. added to the environment tuple. Every caller needs to fill the context accordingly. As we're always typechecking the caller, we can then propagate captured values accordingly and build an environment type for the calling function. If the exterior value to the callee lives within the calling function, we can just bind it directly; otherwise we need to start building an environment for the calling function.
Passing around runtime closures has limitations though: the closure needs to be typed before it can be returned from a function (we can get around that - see next paragraph). When it is then referenced by value, it is automatically converted to a pair tuple of environment and function pointer, capturing the environment of where it was first referenced. The environment will have to be stored in heap memory in order to make runtime closures storage format compatible, and so a runtime closure is also a unique type that needs to be tracked & autodestroyed.
We can get around the type-before-use limitation by stupidly creating an environment of all locals in the lexical scope up to the point of capture, and pass around that environment as a type, along with its template function as a method. This way, the environment is good for all instantiations of the template. We can further optimize our environment size by crossreferencing our locals with the locals referenced by the template (and all subsequent templates) and so trivially exclude locals from the environment that are provably not going to be needed.
In order to give the programmer greater control over when a runtime closure is generated, so that they can rely on whether they are receiving a function pointer or a closure, I would propose to add, in addition to fn (which represents classic C functions) and inline (which is similar to C macros, but hygienic), a third function declarator named capture (which is classic C function + implicit environment).
fn make-capture (y)
# implicitly captures locals in a new environment
capture testf (x)
print x y
# type the function and generate a call to the
function with the previously created environment.
testf "hi" # prints "hi" y
# we can also export captures as closures.
if true
return testf
# for C functions which take function and environment separately,
we can split up the closure
let env f = (unpack testf)
# and pass the argument explicitly
f env 303 # prints 303 y
The fun part starts when we build captures that call captures:
fn curry-two-arguments (x)
# capture `x` in a new environment
and return a new closure
capture make-capture (y)
# capture imported `x` and `y` in a new environment
and return a new closure
capture testf ()
print x y
# returns `make-capture` as a closure that captures x
let cl = (curry-two-arguments x)
# returns `testf` as a closure that captures x and y
let cl = (cl y)
# prints x and y
(cl)