some lua code to create closures

jitclosure_lua.md

The Problem

LuaJIT does not compile creating closures (see http://wiki.luajit.org/NYI#bytecode). However, sometimes we do want to create closures at runtime and pass them as callback parameters.

Example:

	-- load time
	local function maybe_call_callback(f)
		if math.random() > 0.5 then
			f(42)
		end
	end

	register_some_callback_function_that_is_called_in_runtime(function(a)
		local b = 13
		maybe_call_callback(function(c) -- creates a function at runtime, not compiled
			print(a, b, c)
		end)
	end)

Bla bla

The function passed to maybe_call_callback catches a and b as it upvalues. But it's always a function with the same code for its body. Let's call all function instances that are created via the same function definition to be of the same function class.

Now, let us replace the function creation with a function call:

	-- ...

	-- constructor for our closure class
	local closure_class = create_closure_class(function(upvalues, c)
		local a, b = upvalues[1], upvalues[2]
		print(a, b, c)
		upvalues[2] = 14
	end)

	register_some_callback_function_that_is_called_in_runtime(function(a)
		local b = 13
		local upvalues = {a, b}
		maybe_call_callback(closure_class(upvalues))
		b = upvalues[2]
	end)

Whether everything in that function called at runtime can be compiled now solely depends on the constructor closure_class for instances of our closure class, which is returned by create_closure_class, a helper function to create our closure classes. Note that the code would look a little nicer if we didn't modify b.

A naive implementation for create_closure_class could look like this:

	local function create_closure_class(f)
		return function(upvalues) -- the closure class
			return function(...) -- the closure instance
				return f(upvalues, ...)
			end
		end
	end

This closures the values via the normal lua upvalue mechanism by creating new functions at runtime.

Instead creating new functions, we could create callable tables and store the upvalues in the table or in the table's metatable:

	local create_closure_class
	do
		local function mt_call(t, ...)
			return f(getmetatable(t).upvalues, ...)
		end

		local function create_closure_class(f)
			return function(upvalues)
				return setmetatable({}, {__call = mt_call, upvalues = upvalues})
			end
		end
	end

And this should work fine. But what if we want to keep the upvalues secret? Instead of calling the table, one could access the upvalues from the table. And if we set the __metatable field in the metatable, the mt_call function could not work anymore. For a solution to this problem, see below.

A general Solution

A general solution, that still keeps its upvalues secret, which I've found, looks like this:

	local function create_closure_class(f)
		local secret_key_table = {}

		local function mt_call(t, ...)
			return f(t[secret_key_table], ...)
		end

		return function(upvalues)
			return setmetatable({}, {
				__call = mt_call,
				__index = {[secret_key_table] = upvalues},
				__metatable = true,
			})
		end
	end

The important detail here is, that secret_key_table can not be accessed. If you would store the upvalues in the returned callable table itself, one could simply use next on it to find the secret_key_table. By assigning a value to the __metatable field, we assure that the __index table remains secret. And because it is the __index table, anyone who has access to the secret_key_table of the closure class can access the upvalues.

More

Now to something that you most certainly won't ever need: What if you also want to have the create_closure_class function compiled, and not only the callables returned by it?

The thing that keeps it from being compiled is function creation: It creates a local mt_call function and returns another function. But now we already have a tool for solving this: our create_closure_class helper. Let us implement a compiled version of create_closure_class with the help of closure classes created by create_closure_class:

	-- the compiled version of `create_closure_class`
	local create_closure_class_compiled
	do
		-- the closure class for the mt_call instances
		local mt_call_class = create_closure_class(function(uvs, t, ...)
			local f = uvs[1]
			local secret_key_table = uvs[2]

			return f(t[secret_key_table], ...)
		end)

		-- the closure class for the instances of the returned closure instances
		local ret_class = create_closure_class(function(uvs, upvalues)
			local secret_key_table = uvs[1]
			local mt_call = uvs[2]

			return setmetatable({}, {
				__call = mt_call,
				__index = {[secret_key_table] = upvalues},
				__metatable = true,
			})
		end)

		-- do the same as in create_closure_class, but use our closure classes
		create_closure_class_compiled = function(f)
			local secret_key_table = {}

			local mt_call = mt_call_class({f, secret_key_table})

			return ret_class(mt_call)
		end
	end

This is quite useless as you most certainly create new a function when creating a new closure class anyways, but it shows quite nicely how to use those closure classes.