Skip to content

Instantly share code, notes, and snippets.

@reduz
Last active April 22, 2024 21:15
Show Gist options
  • Star 23 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save reduz/9b9d1278848237fd9a9a8b6cc77c8270 to your computer and use it in GitHub Desktop.
Save reduz/9b9d1278848237fd9a9a8b6cc77c8270 to your computer and use it in GitHub Desktop.
From Larvita to Godot

From Larvita to Godot

Origins of Larvita

Our first engine with Ariel was called Larvotor. It was the basis of what later became Regnum Online. After this, we wanted to focus on Nintendo DS, so we created a new game engine for it.

Remember, this was a console that had 4mb of RAM and lacked an FPU (yes you could not use floats natively). For this the new engine was rewritten from the ground up and called Larvita (Little larvae in Spanish). There is not much I can find on its codebase but it was basically similar to Larvita 3 but much simpler and with all fixed point math classes.

Eventually, we got offered other projects to work with and we moved on. These projects required using other third party engines of the time (such as Vicious or Torque), so we put on hold Larvita.

For Atmosphir, ultimately we dropped Vicious engine and returned to our own game engine as we felt more comfortable with it. Larvita went through some rewrites to turn it back into a PC engine. First as Larvita 2, and some months later as Larvita 3.

Atmosphir was a fantastic project to work on. It became evident pretty soon that a custom engine would be required because the game would need allow users to edit levels with millions of blocks that users could place all around the scene. Additionally levels could be uploaded and shared online.

It was a precursor to Minecraft and Roblox and it initially was more popular than those. It could have become a massive success like those other two, had it not been for internal politics and management infighting at the company. As we were just contractors providing technology and had no say, we moved on to other projects and started using Godot instead. Atmosphir eventually moved to Unity, but this proved to be too much for the Unity of the time (2009) and the game lost users and died out.

Either case, Larvita 3 source code is now up. It is mostly of historical value, I would not recommend anyone making a game on it unless you want to have fun.

The design philosophy of Larvita 3

The design philosophy of Larvita 3 meant to apply a lot of the things (good and bad) we learned with Larvotor. If you look at the codebase, it seems similar to Godot, but it is also very different. It's like an alternate dimension version of it.

It had fundamental design choices that were different to Godot:

Lua-Based

Larvita3 was a lua based engine. In fact, it was a library that ran Lua code similar to other engines such as Love2D, although it was a 3D engine.

The main binary file was script_exec and you would execute your game like this:

$ script_exec game_dir

A game would consist of a Lua script that inherited the SceneMainLoop class (this is the precursor to Godot's SceneTree).

Funnily enough, you can still use this same exact workflow in Godot today, even though nobody does. You inherit SceneTree in GDScript and execute it this way:

$ godot -s mygame.gd

This functionality nowadays makes it easier to do CI tasks or unit tests, but It's not used for games.

Additionally, the Lua bindings were created with Ariel Manzur's Tolua++ and they worked by scanning the project together during the engine build (thanks to the magic of SCons). You added a function in C++ and automatically Lua would make everything available to script.

This was quite handy... when it worked (more on that later).

The editor in Larvita 3 was also made in Lua! Ariel did a great work with this, allowing to create and prototype editor code much more quickly than if it was in C++. Here you can see the editor, entirely made in Lua.

Running the editor meant simply doing:

script_exec editor/

How cool is that?

IAPI

The Object class in Godot is nowhere to be fund. We had something called IAPI (an API to provide Instrospection). It was kind of the same, but much more limited. Its what allowed to export and edit properties in the editor.

The only supported way to do this was via set, get and get_property_list. These still remain in Godot Object, but are more like fallbacks when you want to have a lot of customization of what is exposed.

Not everything nodes

Larvita 3 uses a different compositing approach that is more complex than Godot. Scenes are also nodes. Yes I know, but this means actual nodes. To explain it better, there was a type of node called SceneNode, which was a scene you could instantiate into the tree from disk that contained other scenes.

At first this made sense, but it ended up being all sorts of akwardness. It just made scenes more complex, and situations where you wanted to instantiate the same scene quickly was inefficient because it had to be loaded from disk.

Nodes also used a different memory model via the Ref<> class. This means that child nodes were reference counted in Larvita 3. This was dropped in Godot because it was very easy to create cyclic links.

No resources

Larvita 3 did not have a common resource system. Every type of resource was loaded manually with their own loader. No common class existed.

GUI was its own thing

Larvita3 uses the PIGUI framework, which is the same one I used for Chibitracker a long time ago. It is a portable GUI toolkit that is disconnected from the game engine.

This is exposed to Lua directly as code, you can't edit UI from the editor.

SDL

Larvita 3 used SDL, because it made it easy to go cross platform with our limited resources.

Road to Godot

Eventually, development wrapped up with the completion of Atmosphir and the engine was not used for anything else (other than a few prototypes for clients). Work on Godot had already commenced simultaneously.

So what changed in Godot and why?

Goodbye SDL

SDL is great and by all means I would recommend anyone starting a game engine to use it. But at some point, after an application becomes more complex and more integrated to the operating system, it often requires to do it in its own way.

Examples of this is how we integrate with OS filesystem paths, OS windowing, boostrappers, threading, IME, etc. At some point SDL always falls shorts when you need very specific small things.

The truth is that the OS layer is generally not that big in a game engine, specially if you make it for your exact needs. Writing those from scratch in Godot was not as hard and, today, Godot has several very skilled contributors in each platform that can ensure the interaction is as smooth as possible.

While SDL 2.0 fixed many of the shortcomings of SDL 1.2, it was not available when work in Godot begun. Additionally, Godot always needed more than what was available for SDL at the time. Ultimately, we still use some parts of SDL in Godot that make sense like the Joystick database. Glory to FOSS.

Everything is an Object

The first thing that became evident is that we wanted to grant this sort of "dynamic typing / introspective" design that was used for IAPI in Larvita3 to the whole, entire engine.

Inspired in Qt's QObject, the Object base class was created, serving a kind of similar role.

This way all object classes can be introspected, serialized, etc.

Signals also became more dynamic and introspectable, though a little too dynamic and string based. Godot 4 added Callables, filling the gap.

Good bye Lua

Lua is a great language, but It's a pain. The incremental garbage collector is pain and so is the overly dynamic nature of it. Collecting too much? stalls. Collecting too little? Memory growing out of control.

Classes had to be emulated using table fallbacks, the binder is hard to maintain, etc. Lua had to go.

Initially, we moved to Squirrel, which as a language it worked far better tha Lua:

  • Actual classes.
  • No GC
  • Friendlier syntax.
  • VM that was not a pain to create glue for.

But then, we needed a new binding system. Originally the idea was to redo Tolua++ in something else. We investigated Swig and other libraries, but ultimately we felt an approach where binding is done with code would be far simpler. This is where ClassDB comes from in Godot.

In Godot. binding a class is done like this:

ClassDB::register_class<ClassName>()

And a method:

ClassDB::bind_method(method)

And everything works using template magic. If something does not work it was much easier to fix.

The main drawback of this is that it limited the data types available to the binder, but ultimately this was a much bigger advantage then a drawback. The truth is that, 99% of times you can get by with the same reduced amount of datatypes to expose to any API, either in pure or composited way. The exceptions needed workarounds, but they were rare and are now fully supported in Godot 4.

Still, after a few years working with Squirrel, there was still significant pain:

  • No thread support (good luck running a script from multiple threads).
  • Glue code was still pain and inefficient. If you used native types such as int or float, you are great performance wise but using other types such as Vector3 required heavy pointer usage, hashtables for methods, etc. which quickly became a bottleneck.
  • The editor integration we wanted to have, geared for ease of use, was still difficult. Difficult to do code completion, difficult to make things like exporting properties in easy to use way, difficult to do preloads, etc. The pain points kept piling up.
  • And a few other things.

We investigated and experimented with Python and other languages, but finally decided to roll our own, GDScript. In retrospect this was a great idea, because it is one of the main pluses Godot has today (its very tight editor integration).

Editor in C++

The Godot editor was written in C++. To be honest, Lua and scripting languages are fantastic for prototyping or creating games where you write code and rarely touch it again after the game is published.

Maintaining a very large codebase of tools is a different matter and code refactoring/navigation can be a big pain if not using statically typed languages.

Larvita 3 had a prototype editor in C++ that never reached maturity as part of what was open sourced. This was the last piece of work done before moving work to Godot.

A unified resource system

Many things were obvious after working with resources for so many years:

  • A centralized loader and dependency system makes things easier.
  • In 99.99% of the cases, when you ask to load a resource from different places in the code, you want that same resource (code instance), not a copy.
  • Everything on disk must be a resource. Everything (more about this later).
  • Resources need to be reference counted, so when no longer in used they free themselves automatically. Unlike nodes, though, we found cases where cyclical resourcereferences are pretty much non existant. And even if they were, the editor could easily detect this at edit time and warn the user. This realization was a big win and in large part what makes Godot resource system so simple, intuitive and efficient compared to other game engines.

So, suddendly all became resources in Godot:

  • Packed Scenes (more on this later)
  • Scripts
  • Textures
  • Meshes
  • Anything

Additionally due to Godot design, resources can be built-in into other resources. This keeps your project on disk a lot less cluttered than in comparable game engines.

Ultimately, Resources is also another very strong point in Godot.

Everything nodes

Larvita 3 was halfway there, but in Godot we decided to go all in.

There were a couple of challenges in Larvita 3 that had to be solved:

  • Having a SceneNode as intermediate in scene compositing was a big hassle.
  • Loading scenes from disk without intermediary form was inefficient for reusing.

Godot solved this with the introduction of PackedScene:

  • PackedScene is a resource. It is saved and loaded to disk as such.
  • To instantiate into nodes, you use PackedScene.instantiate().
  • This also allows to keep your scene ready in a way where instantiation is very efficient.

This change, however, was much harder than expected. Today, one takes for granted that scenes in Godot can only have one single root node and this feels right and natural. But, back then, we felt this was very troublesome and that users would still prefer to have many root nodes when instantiating a scene (like in Larvita3).

Ultimately after we started experimenting with mandated single root nodes, this turned out to be the right choice all along. It made compositing super easy to any level. It's one of the key distinguishing features of Godot today too.

No more script_exec

Our experience told us that, in most cases, a main orchestrator script was not needed, and that everything could be done with scenes:

  • Scenes you load and unload.
  • Singleton scenes that are persistent and handle orchestration.

This is why, in Godot, you can very easily just run any scene you edit stand-alone and it will work, which is another key feature of the engine. The singleton scenes are always there to provide what is needed to the running scenes.

GUI as nodes

Additionally, the GUI library we used for Larvita 3 was also rewritten using nodes. This was taken even further in Godot 4 with the addition of the Window node.

Having the GUI as nodes allowed making UI interfaces as if using QtDesigner or Visual Basic, all in the same editor UI for everything else.

2D engine

Finally, a 2D engine was added to Godot some time before it was open sourced. This is because we had clients requesting to make 2D games with the engine.

The reality is that we thought going with a 2D over 3D engine like in Unity would be a big hassle. Becuause of this, a dedicated, separate, 2D engine exists in Godot that was made with all the love to ensure it feels as native as possible.

Closing Words

The transition from Larvita 3 to Godot was one of learning from our mistakes. By the time Godot was open sourced, we had published several dozen games with our own technology. This allowed us to iterate at little cost until we found something that hit the nail.

Even then, it took many more years for Godot to become what it is today, which is orders of magnitude more incredible than what it was when it went public. This was only be achieved thanks to the collective talent of thousands of contributors over a decade, helping to truly take Godot to the next level.

@seanballais
Copy link

Finally, a 2D engine was added to Godot some time before it was open sourced. This is because we had clients requesting to make 2D games with the engine.

Was the 2D engine added around the time when the Mr. Bean and Dog Mendoza games were being made?

@reduz
Copy link
Author

reduz commented Oct 10, 2023

@seanballais No, it was added for The Mystery Team.

@punto-
Copy link

punto- commented Oct 10, 2023

A few additions:

  • Lua and floats: the first version of larvita had a modified version of Lua that added an extra type for integers, and removed automatic conversions from ints to floats in arithmetic operations, intended to run on the DS (which was somewhat successful but never really tested in production). This didn't survive to larvita3, but we learned a lot about parsers and compilers, which later went into creating our own shader language and eventually gdscript. Lua is a great codebase to learn how programming languages work in general. Lua in larvita3 also had a modification in the garbage collector code that eliminated a big stall, but that was more of a hack that basically disabled a whole sorting step in the generational garbage collector. This taught us to avoid garbage collectors from then on :)
  • Animations: larvita3 had an early version of godot's animations, where instead of having only bone tracks with position/rotation keyframes, we also had tracks that were paths to nodes and properties, and the value was a Variant. This is obvious now, but at the time animations were only bone tracks. This was developed for Atmosphir, where we needed to play a footstep sound for the character animations, and I wanted the artists to have control of this. This was also a big influence in the decision of making everything nodes and improving the introspection, to improve the integration with the editor and the animation system
  • 2D wasn't obvious at first, after all it was the late 2000s/early 2010s, so 2D was "dead" (some game engines even had "3D" in their name; it took a while for people to figure out 3D sucked on the iPhone and 2D was revived). But Godot had a very good UI and it was integrated with the scene, so the whole canvas API was already very good in the VisualServer. While working for a client making a 2D PSP game (The Mystery Team), at first we told them to use TextureFrames as sprites and other workarounds, but eventually it becamse obvious that we needed some proper 2D nodes. This was also the first time the animation system was used extensively, we showed their artists how to animate a simple UI element, the next day they showed us these amazingly complex UI transitions. This company had a relatively big team (25 people, 5 programmers, the rest artists and misc roles), and we learned a lot about improving the tools and workflows with them
  • If you look at early Godot codebases, there was also an SDL port. This was added also for The Mystery Team, because one (only one :p ) artist in their team used Mac, and he wouldn't switch to Windows. So that day we agreed to add OSX support, I went back to the office and struggled with the OSX native APIs for a few hours, until I gave up and added SDL, and this was the way Godot run on OSX until somebody implemented it the proper way

@reduz
Copy link
Author

reduz commented Oct 10, 2023

@punto- Right. I completely forgot about this. The animation system in Godot was created for The Mystery Team. This was a company that had artists coming from Flash (this was Farmville times), so the way animation in Godot was implemented was inspired by it.

@punto-
Copy link

punto- commented Oct 11, 2023

@reduz no the animation system was already created, it was pretty much the same as in larvita3, with the difference that larvita3 didn't have the UI in the scene. When they started using it for TMS we might have made some changes to the editor, but the point is, that was when we got confirmation of the true potential of animations, but the idea came from larvita3 during Atmosphir

@reduz
Copy link
Author

reduz commented Oct 11, 2023

@punto- makes sense checking the code, then it is as you say, we made the UI for it for Mystery Team.

@punto-
Copy link

punto- commented Oct 13, 2023

@reduz no the UI was all made as well, we might have made some changes based on their feedback, but it was a complete feature by then. In fact the larvita3 animation editor also has UI to add keyframes for nodes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment