p2p key-value store over datakit
Data Model
Key -> Path
Value -> Json
The value is possibly JSON automatically generated for OCaml type definitions using ppx_deriving_yojson library. Merge function must be explicitly specified:
val merge : coancestor:Yojson.Safe.json
-> theirs:Yojson.Safe.json
-> mine:Yojson.Safe.json
-> Yojson.Safe.json
Merge must always be defined, and any error state explicitly encoded in the resultant type to be handled by the application.
Network model
The current design is that each peer will run a datakit 9p server, which exposes a 9p fs interface. Each peer also mounts its own datakit 9p volume and also the desired set of peer’s 9p volumes. This allows the programmer to describe the network explicitly.
Peer info stored in a separate branch. Questions: Can peer info also be managed the same way as usual kv info? Would the peers watch for updates to the peer info table and reorg the n/w dynamically?
Execution model
The peers can update their local snapshot of kv store, explicitly fetch changes from peers (which fetches updates in the mounted 9p volume and merges using the user-defined merge function), or watch a subset of peers & keys for updates (and automatic merges).
Status
- Modelling the network by hand. WIP dockerfile: https://github.com/philipdexter/datakit-ssh
- Filed a bunch of issues:
- size: moby/datakit#183
- fetch: moby/datakit#180
- watching: moby/datakit#178