Clang-based in-game "REPL" console

magnum-repl.md

Clang-based in-game "REPL" console

Goals

• Use as much existing functionality as possible for implementation
• No new APIs, no new language bindings
• Use Clang for state-of-the-art C++11 support and diagnostics
• Keep It Simple, Stupid

What in-game console is used for

• Querying game state
• Changing game state

What in-game console isn't used for

• Changing existing game code
• Creating new game code

Using any IDE will be much more convenient than doing this using in-game console. Live code editing (gameplay, AI...) is already possible by developing an plugin with the comfort of your IDE and then reloading it in-game.

REPL implementation issues

• Calling functions is easy as global state is managed by game itself and thus persistent between evaluations
• Maintaining local state between evaluations is hard
• Do we need to maintain any local state at all?

Use cases

• Modifying object positions and other parameters
• Changing rendering parameters
• Querying performance counters, cleaning up caches
• Enabling and configuring debug draw
• Instancing new objects and referencing them later

Not needed (beyond the scope of simple oneliner):

• Ability to define new classes or standalone functions

Required features

• Retrieving reference to one or more objects (by name, ID, position...)
• Changing object parameters, applying some (user defined) function to a set of them
• (Automatically) print values to output
• Create object instance and have it managed and "garbage collected" by the game

C++11 features to the rescue:

• Lambda functions remove the need to define local functions, allowing for one-line statements
• Range-based for loops make one-line cycles easier to write

API features to the rescue:

• Method chaining to allow changing many parameters in one run without the need to explicitly save and reuse reference to some instance

Engine features to the rescue:

• Using resource manager (either some global one or one specific to console) to save a named instane of one of the types allowed by the manager and the ability to retrieve any named object back.
• All commonly used functions already implemented (no such thing as "hey, I need to implement vector addition because this shit hasn't any"), reducing the need for writing new ones.
• Debug classes to print the values on output. Either explicitly (Debug() << 5 + 3;) or with some special syntax (e.g. omitting semicolon at the end would cause the result to be printed to output).

Why not to support local variables

• To have persistent local state, we need to either run whole history every time a new line is added (breaking everything on the way) or implement some persistent stack and then pass it back to the code. This out of the scope, it might as well be another huge project.
• The stack will grow up, but never down (no scope ending), allocating variables via new will create memory leaks if user forgets to call delete.
• For user convenience it would need some garbage collector, also way out of scope of current project.

Sample usage

> Debug() << scene["Bomb37"].transfomation();
DualComplex({1.0f, 0.0f}, {3.5f, 0.75f})
> scene["Bomb37"].translate(Vector2::xAxis(2.0f))
]   .rotate(35.0_degf);
> inventory.get<Launcher>("Launcher").launchBombs();

Actual implementation

Basically we need to compile, load and run the code without restarting and recompiling whole application.

1. Take a line from the console, create some boilerplate code around it and compile it with Clang.
2. On error print the Clang message and do nothing.
3. If compiled successfully, use Corrade's PluginManger, load the module with it and run it.
4. If the user wants to do the same thing again (Up Arrow and Enter), just run it again without recompiling, otherwise repeat from step 1.

Issues:

• Saving the file to disk and calling external Clang executable to compile it will be slow -- would it be possible to embed it somehow?
• Loading the module from disk will be slow -- would it be possible to portably call dlopen() from some ramdisk or better directly from memory?
• PluginManager does things which are not needed here (dependency checking, configuration file parsing) and which will slow down things another bit.
• What global state expose to the user -- it will be specific to given game, which needs to provide these to the plugin code.
• What about #includes -- explicitly including whole engine will slow down the compilation a lot (think about ~100k LOC included). Catching some prefix (e.g. import Math/Vector4) and adding that as #include might work, but it is inconvenient if needed too often. Clang modules will be a wonderful solution not only for this.
• Fool-proof -- how to disable new/delete operators to avoid leaks and crashes?
• Actual console implemetation -- what editor features to offer? Key shortcuts? Copy/paste? Line wrapping implementation? What about ligatures, caret positionig etc. in non-ASCII text? Use HarfBuzz or some layouing library for exploiting the properties? Or (at least for now) use something well tested (e.g. Qt's QTextEdit), maybe in separate window so it doesn't mess up the rendering and event loop.