As Tikabot moves away from an XML-file-driven client app and towards a server/service, it will need a way to co-ordinate its configuration. When it becomes a service, it will need to not only store configuration somewhere, but also be able to update it on the fly.
The nature of the bot demands the configuration be cached by each node using said configuration, so it must use a system of update messages rather than a central database that gets repeatedly queried. I have decided to use these update messages as the core of the configuration system as a whole, as I believe it to be a highly adaptable architecture.
The whole configuration system shall be built on a sequence of "patch" commands. When configuration is updated, the changes are submitted to an updates channel. Nodes that use this configuration for processing will update their internal cache, while storage nodes will update the persisted configuration all based on a single update message. Pulling the full configuration from storage will use a similar model.
A configuration is identified by a hash. When a configuration is updated, so is the hash. Updates against unknown hashes are ignored.
Redis' Pub/Sub system will be employed for this architecture.
Nodes interested in updates will subscribe to updates:<hash>.
When a configuration is updated,
a message is posted to the hash's channel.
When an update is received,
a node should unsubscribe from that hash's channel,
and subscribe to the channel for the new hash.
Only the first update should be given any heed.
Nodes wishing to receive all updates (e.g. storage nodes)
will subscribe on updates:*.
All configurations begin with a "root configuration".
The encoding of this configuration is deterministic
based on the connection info used
(e.g. the root for the Twitch channel #tikatoo
will always be exactly the same).
The hash of the root configuration is the starting point
for all subsequent hashes for this configuration.
This is called the "root hash".
When a configuration is modified, the new hash is the hash of the patch (which encodes the old hash within). Thus, the hash tracks the update chain, not the full configuration.
Configuration is stored as key-value pairs. The key is the latest hash for a configuration, and the value is the patch that takes the root configuration up to the latest configuration. An additional mapping is created of root hashes to latest hashes.
The storage engine listens for updates to all hashes. When it hears an update, it:
- Loads the configuration;
- Applies the patch;
- Stores the resultant configuration;
- Updates the root hash mapping;
- Deletes the old hash.
In order to load a configuration from storage (e.g. when a node is starting up), an update request is submitted. The update request consists of a root hash. A patch taking the root configuration to the current configuration is then sent on the update channel for the root hash.
A node wishing to fetch the full configuration
must first subscribe to the channel for the root hash.
It then posts that root hash to update_requests.
The patch that is then sent on the root hash channel
can be used as if it were any other update patch.
Internally, a node will usually establish a full connection based on the root configuration before sending the update request. This allows the exact same code that handles all configuration updates to be re-used in its entirety for loading an existing configuration.