Skip to content

Instantly share code, notes, and snippets.

@kayceesrk

kayceesrk/datakit-kv-store.md

Created July 18, 2016 21:10
Show Gist options
  • Save kayceesrk/d3edb2da0aa9a3d40e9e3f838b67bd1a to your computer and use it in GitHub Desktop.
Save kayceesrk/d3edb2da0aa9a3d40e9e3f838b67bd1a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
datakit-kv-store.md

p2p key-value store over datakit

Data Model

Key -> Path
Value -> Json

The value is possibly JSON automatically generated for OCaml type definitions using ppx_deriving_yojson library. Merge function must be explicitly specified:

val merge : coancestor:Yojson.Safe.json 
         -> theirs:Yojson.Safe.json 
         -> mine:Yojson.Safe.json 
         -> Yojson.Safe.json

Merge must always be defined, and any error state explicitly encoded in the resultant type to be handled by the application.

Network model

The current design is that each peer will run a datakit 9p server, which exposes a 9p fs interface. Each peer also mounts its own datakit 9p volume and also the ​desired set​ of peer’s 9p volumes. This allows the programmer to describe the network explicitly.

Peer info stored in a separate branch. Questions: Can peer info also be managed the same way as usual kv info? Would the peers watch for updates to the peer info table and reorg the n/w dynamically?

Execution model

The peers can update their local snapshot of kv store, explicitly fetch changes from peers (which fetches updates in the mounted 9p volume and merges using the user-defined merge function), or watch a subset of peers & keys for updates (and automatic merges).

Status

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment