I want cheshire to be my own mega-application. Incorporating all the great ideas and things I imagine. To be distributed platform, wonderland container, personal helper, and enormous number of other things.
Cheshire itself is currently written in CoffeeScript. It's supposed to be run by node's coffee command, so there is no any build system or js equivalent of coffee code.
As I wanted Cheshire to do lots of different things, that sometimes by no means correlate to each other, it was decided to use some kind of modular system. After a bit of googling, I found no ready-made modular system, except for npm and node-plugin-manager. Npm seems not really the thing I wanted - there are lots of unneeded features, and the whole system seems not to be very configurable, though the base ideas (module is folder, there is some module descriptor, etc) are good. The second (plugin manager) has some strange pub/sub architecture, and is not really functional in my future environment. So, I've decided to implement my own module system. Planned list of features: - every module is a class (can be instantiated multiple times) (TODO: is that really needed?) - restart any module on files change (should be configurable) - some module configuration system, both static (controlled by file), and dynamic (configurable by module's code in runtime) - module enumeration and introspection - on-demand module loading - possibility to fetch modules from cheshire fs (see below) - dynamic dependency management (as any module can be unloaded/reloaded at any time, we should track this) - testing support, so that any module can be tested on-demand (ex, before loading or updating) So, basically cheshire main app will only load the initial list of modules, and let them run.
Starts kadoh-based dht node. Provides some custom features, and currently is used as main routing and data store facility for cheshire. List of things, modified in kadoh: - added encryption and signing - implemented multi-value store
Starts kadoh-based mainline dht node. It's absolutely compatible with any torrent client application, and can provide bootstrap source for our main dht instance (stores information about available Cheshire nodes and their udp addresses). May be also used in future to inject our data in torrent's dht.
Defines the root cheshire filesystem (see below), which will be seen by user. Later, other modules may introduce some inner filesystems, and extend anything they want. By default, this module exports this fs using 9P protocol, and default Inferno's tcp port (6666).
Can export various styx-server, running on local machine. Can export (create tunnels) for network-based servers, or start our own. Reads configuration from local config file. Later these shares can be stored in DHT by it's module, or imported on remote Cheshire nodes by styx-client module.
Config file example:
{
"/some/path": {
"uid": "52daa2bcbf7a2eb8c6e38a1b6a9e0e429ec81c16",
"server": "export",
"root": "~"
}
}Encryption is not currently implemented, though there are some ideas. Possibly will be expanded below.
Current list of internal server types: - export: exports local real fs entry - proxy: proxies connection to some locally-running styx server - dht, cheshire, modules, etc: some internal control functionality, that can be exported, to allow to remotely-control this Cheshire instance - *.js: can be specified to directly execute provided javascript (or some convertable, like coffee- or live- script, or others) file, and assume it provides an instance of styx server
When some other module finds exported styx server, it tries to 'import' it, establishing connection, encryption, etc, and incorporates it into styxroot.
Currently, we have dht as our main server-info provider. We query dht on every walk in our styx-root, and fetch info from the dht, by hashing the path and possibly appending some static-key to it. So, we have key-value pairs stored in dht, where key = sha1(path + salt), value = server definition, which includes the id of the node that stores the server and some access details.
We need to be able to reload any module in runtime. So, the module system should be able to detect module's source file changes, stop the currently running code and start the new one. But the main problem here lays in dependencies between modules.
For example, styc-client module depends on styx-root and dht. When we restart styx-client, it will just clean up it's callbacks and pending tasks, and restart gracefully, acquiring new styx-root and dht handles. Though, when we restart dht, we need to somehow notify each module, that uses it dynamically, that the module is about to stop and that we need to stop using it. That can introduce lots of complexity in module implementation, because of the need to track every used module, and due to possible module incompatibility.
On of the possible solutions is to...
to be continued
to be continued
to be continued