Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?

p2p key-value store over datakit

Data Model

Key -> Path
Value -> Json

The value is possibly JSON automatically generated for OCaml type definitions using ppx_deriving_yojson library. Merge function must be explicitly specified:

val merge : coancestor:Yojson.Safe.json 
         -> theirs:Yojson.Safe.json 
         -> mine:Yojson.Safe.json 
         -> Yojson.Safe.json

Merge must always be defined, and any error state explicitly encoded in the resultant type to be handled by the application.

Network model

The current design is that each peer will run a datakit 9p server, which exposes a 9p fs interface. Each peer also mounts its own datakit 9p volume and also the ​desired set​ of peer’s 9p volumes. This allows the programmer to describe the network explicitly.

Peer info stored in a separate branch. Questions: Can peer info also be managed the same way as usual kv info? Would the peers watch for updates to the peer info table and reorg the n/w dynamically?

Execution model

The peers can update their local snapshot of kv store, explicitly fetch changes from peers (which fetches updates in the mounted 9p volume and merges using the user-defined merge function), or watch a subset of peers & keys for updates (and automatic merges).

Status

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment