GameScript identities

extension.md

This is meant as extension on: https://wiki.openttd.org/en/Development/Design%20Drafts/Scripts/Capability-based%20API

In addition to having capabilities, GS would benefit from being able to instantiate controllers for certain objects.

For example, take a town. Currently either a GS controls all towns or it doesn't. By adding capabilities, as written in the wiki, it can either share all towns with other GSes or keep them all for him self. I would propose a more fine-graned method: instances.

Every GS has a function like: HasInstanceFor(capability). As example: HasInstanceFor(TownGrowth). The GS either returns 0 if it has nothing for this capability, or N with N > 0 for how many different forms it has for this instance. Next, OpenTTD adds up all the Ns, picks a random number in range, and selects that GS + index to handle that town now forths. For example: GetInstanceFor(TownGrowth, 2).

In Squirrel, the GS creates an instance, which has a few mandatory functions defined to handle the TownGrowth capability. This is all in addition to claiming capabilities as written in the wiki document.

Basically, this assigns "AI"s to certain objects of the game.

This could work for several things

• Towns
• Cargo
• Industry (single industry)
• Industry group (single type)
• Even vehicles (not sure this is wanted, but one could argue this could do funky stuff like ATCs)

There are things of course that are more global, like subsidies. This would benefit more from a shared capabilties as described in the wiki, although each subsidiary would be assigned to a single GS of course.

The benefit of this approach is that each of the above objects can have their own identity. For example, I can have 10 towns with 10 different growth algorithms spread over 4 different GSes. It would mostly replace any "exclusive" claim on a capability; but it is also not really shared.

Communication from C++ to such instances could be done in an event-based system as we currently do, but instead of delivering the event to the AI or GS, it is delivered to the Squirrel instance of the object. An example: someone hits "Fund Town". For this C++ creates an event which is delivered to the Squirrel instance of the object to handle. In other words, there will be many small event-loops.

This would make GSes more cooperative.

The downside is that this might be more difficult to present in UIs.

I was wondering how GS could control how towns are built.

• As _dp_ pointed out, towns are defined by roads: So when a GS builds the roads for the town, OTTD/NewGRF can put the houses next to it.
• Let's say there were GS with TownConstruction capability.
  • This capability is only about how towns grow / where they place road.
  • This is separate to the capability TownGrowth, which decides cargo/other requirements, i.e. whether towns grow.
• Now this GS must be called during map generation.
  • Town generation must be done before industry spawning and lots of other things. So this is a different call than the current GS init, which is at the end of map generation.
• Option 1:
  • OTTD decides for a town location, puts down the center road piece, and then calls a GS method to grow the town to a certain size.
  • On failure, OTTD reverts this action, by deleting all roads owned by this town. (this is what OTTD already does today)
• Option 2:
  • Everything is done inside the GS.
  • OTTD calls a single GS method.
  • The GS reads the settings from the player (number of towns, etc)
  • The GS decides town positions, and grows the towns.
• Option 3:
  • Both: one GS decides for town locations (TownLocation), another one for town construction (TownConstruction).
  • In total 3 capabilities:
    • TownLocation: decides placement of towns during map generation.
    • TownConstruction: build roads inside towns (not between towns) during map generation and during the game.
    • TownGrowth: defines growth requirements, and decides whether a town shall grow or not.

Why do I dump this blabla here?

• It looks like we need to carefully decide, what things must be covered by the same GS, and which could be combined from different GS.
• I would try to avoid putting any settings into OTTD, and use the GS settings instead.
• TownLocations and TownGrowth look rather global to me, sharing this across multiple GS is weird.
• TownConstruction could be different per town.
  • By default OTTD could pick one at random for each town.
  • The TownLocation GS could pick them.
  • I would avoid putting any meta-GS-settings to weight the choice into OTTD.

@frosch123:
Although towns grow because of road, there is also some logic around placement of houses and where they can grow. So I don't fully agree that TownConstruction should only manage roads; I think it should also manage where houses can be build. But those are all minor details we can work out later :D I think the current town_cmd.cpp gives a template for what is needed to make a town grow (in terms of construction), where other parts show when a town can grow (your TownGrowth).

Regarding your options 1/2/3, I have an idea that is "somewhere in between" those. But more on that in a bit :D

---

I have been wondering how to push this idea forward, so I have been experimenting a bit with some ideas. There are some technical limitations to what we can do, so here is a proposal with the aim to make towns more moddable; but I think it can be deployed to other parts too, it was just simpler to start with something small. This is more of a brain-dump to get my own thoughts in order, than anything else, but it might help start a conversation on what we have to balance.

Controllers

Parts of the game can be assigned to controllers. This can be done on many levels, depending on what we want to control. So for example, this ranges from a single Town how it should grow (TownConstruction), up till map generation. What exactly we assign a controller is something we have to evaluate.

Every GS can tell the game (via capabilities) what type of controllers it implements. One GS is always in charge of a single controller, but because the level of controller can vary, this can have different implications. For example, there is only a single controller in the whole game that deals with map generation. On the other end of the spectrum we also have one controller for each and every town in the game, to manage its construction.

There is also a dependency between controllers. For example, the map generator controller will want to create towns. This means he needs to create a controller for TownConstruction, and let it create a town. For example, I can imagine a TownConstruction that builds beach-towns, and will find a spot close to a beach to generate this.

There can be levels between those two, these are just examples. For example, I can imagine there is a MapRegion in between, like every tile above height N (snow), or "Industrial Regions", etc.

We should be able to draw a graph how one level of controllers communicates with the next. What we have to consider is how they communicate. Do we pre-plan this, or do we leave this to the GSes to figure out on their own (community-consensus, so to say).

On a technical level, how controllers can work is as follow:

A GS tells the game its capabilities. For example, there are three GSes that say: I support TownConstuction. When the map generator wants to create a town, it asks the game to give him a controller he can boss around, the game asks the three GSes: who is interested in giving me a controller for TownConstruction; if multiple answer, one is picked (see below), and returned to the map generator.

I think there should be some metadata given from the map generator to the GSes, so they can evaluate if this is a request they want to participate in. As mentioned earlier, one of the things I can imagine is: town above snowline. Some GSes might have TownConstructions only for those, and some might not implement it at all. This allows the game to find a good match. GSes should also return some weight as to indicate how much they prefer to be the controller for this town, because of the following:

As fallback, in OpenTTD hardcoded are a few GSes that do the base-game. We already do this for DummyAI, and basically we extend this a bit to make sure there is at least always a TownConstruction that is willing to pick up the job. The weight for this is of course the lowest possible: if there is any GS that wants to build the town, by all means, he should. But if they all say: no tnx, this part kicks in and takes over. Again, exactly as we do for AIs (only we should implement this a bit better :D). Weights can be replaced with a simple fallback system if that works better for us.

The main thing I have been wondering in all this, is how GSes initiate a controller. There are two paths I see:

1. we do a callback to the GS: give us an instance for this controller. He returns the instance, like the GetInstance() is currently implemented, and Squirrel after that calls functions on the instance returned. This means the GS can only store information about the controller; it cannot share information between controllers in Squirrel variables. This means we do have to supply some API to do this "global" storage; as a bonus that would also most likely mean that this can work between different GSes. If for example we use JSON as storage, I can imagine the community working out a schema that works for them, and multiple GSes can implement this and work together. You see this happening with other games too, for example OreDict in Minecraft.

2. we call a function on the main GS instance to tell him to initiate a Controller. How-ever he does it, he does it, we don't care. He returns us something unique, we will use in all next calls to indicate which controller we meant. This means the GS has total overview of all the controllers he is running. It can freely exchange information between them. I am a bit scared of this approach, as I think it means that GSes will not cooperate with each other, kinda defeating the idea of building this up this way. If we go this route, we might as well just say: there can only be 1 GS that implements any TownConstruction. Completely viable btw, but allows for less configuration, and much bigger GSes.

Besides this choice, the other problem to tackle are deadlines. Currently, GSes (and AIs) don't really have any deadlines; that is to say, if they take a really long time doing something, more gamedays will pass before they are done. Each GS gets, I believe, 1000 opcodes per game-tick to do, and it is scheduled out if it is not done yet. It can continue on the next tick. This is why we use events to notify GSes of things, and not callback.

As most of the things we want to add, would need to run on callbacks, this becomes a bit more tricky. As example: if we want to extend a town, the controller should build, say, a road. But the controller decides the check every tile of the map before saying: this tile will get a piece of road. This works fine for a 256x256 map, but will be very slow on a 4kx4k map. Scheduling the callback out will be very difficult, as it might be that the callback is not even done by the time the next callback has to be called.

Currently we have a similar problem, with Save(). It is the only callback into GS instances we have (we have more callbacks into GS Info, but not into the instance itself). It does have a deadline: if you are not done within, I believe, 1000 opcodes, you are aborted, an error is throw: you did not make the deadline, and the save is not done (not even retried). This works fine for Save(), but for a generic solution it might be a bit painful.

We can also approach this the other way around, where we use an event-based system, and give the GSes as much time as they need .. they just become very slow (in terms of game-ticks) to do actions. And it is up to the GS author to handle this properly. For this to work properly, ideally we should have a single Squirrel VM per controller; but having 20k+ VMs in a 4kx4k map for Towns alone might be a bit overdoing it. But scheduling Squirrel VM out in an eventloop of one controller trying to schedule in the next is also not going to work. So an event-based system is basically not an option.

This means we need to decide how we want to do the callback system, as in: what happens if a GS doesn't meet the deadline we set (per controller-type, I guess). So for example, a TownConstruction controller only gets 1000 opcodes every 10 game-days. And if it doesn't make the deadline, the callback is killed and rolled back. Warning to the user, and continue, hoping next time it does make the deadline?

NewGRFs of course have a similar issue: they don't have deadlines either. But they are limited in what they can do in general, and if you make an action very slow, people will complain soon enough. So that might also work here: just let the GS take how-ever long it wants, and people will complain sooner or later anyway (hopefully to the GS author).

---

In summary, what it boils down to:

• Allow callbacks into GS, making it similar to NewGRFs (but in another language)
• Instantiate a Squirrel instance per controller
• Instead of running C++ code, call the callback of the Squirrel instance for that controller
• Have a hard-coded fallback in case there is no GS willing to instantiate a Squirrel instance for a given controller

The memory-overhead this way is minimal: every Town for example gets an additional pointer to a Squirrel instance, and the Squirrel VM increases in memory by the size of the class that was instantiated. But it is not zero, so maps with 20k towns will notice the increase in memory. To estimate it: 1k for Squirrel instance and 8 bytes for Town entry makes ~1k per Town extra for the TownConstruction controller. This would mean 20MB of additional RAM in an insane case.

The code needed to implement this is also minimal: we already do this with GetInstance() and Save(); we do need to manage everything a bit, and make sure debugging information is as clear as we can to the user. Otherwise there is not really much to this.

Most of the work is figuring out how controllers should look, and how they should talk to each other. I have some thoughts on that too, but I kinda first want to validate the above approach, see if that really works. It is a bit backwards: starting with what technically is possible before defining what you want it to do, I am aware, but knowing the technical solution also allows to shape how controllers should look.

Anyway, lot of words .. and I think most had already been said in one form or the other; this was mostly to order my thoughts, but I would love input and opinions on the matter :)

PS: in regards to capabilities, it means we don't have "exclusive" and "shared", but more like: "limit: 1" and "no limit", of how many GSes can be loaded for a certain capability. It is somewhat the same, but slightly different :)

To make a Proof-of-Value, I suggest we take this approach:

• Implement the above summary

• Create TownConstruction as capability, which takes over:

  • DoCreateTown in town_cmd.cpp
  • Callback from CmdExpandTown
  • Callback from TownTickHandler

  (in other words, all roads to GrowTown). We can work on having enough API functions to do what needs to be done.

• Make a C++ based MapGenerator controller, which create a TownConstruction controller to create and grow a town. This controller is completely fake, and possibly even a single function-call in the current map generators, and only serves as proof the system works.

This sounds simple enough to do in a weekend, hence the scoping like this. This would allow other people to see if this is sufficient for them to build their own town growth algorithms, without going overboard.

The next step would be to find a way for them to communicate with each other, and work towards TownGrowth controller.