Skip to content

Instantly share code, notes, and snippets.

@meyerdan

meyerdan/next-steps.md

Last active June 7, 2017 06:26
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save meyerdan/3ff829f31faff96a9b92a4313398bd7f to your computer and use it in GitHub Desktop.
Save meyerdan/3ff829f31faff96a9b92a4313398bd7f to your computer and use it in GitHub Desktop.
Download ZIP
My Zeebe Questions
Raw
gistfile1.md

Everything related to Stream Processing is currently more or less well structured in the sense that it is clear which components exist, what their responsibilities are, how they interact and what the main mechanisms are. Yes, there are problmens but they are not "structural". However, everything besides Stream Processing is chaos. We need to bring some order and structure here -

Broker

  • Which kinds of interactions exist besides Stream Processing? Which facilities do we have to address them?
  • Which components are inside the broker and what are their main responsibilites? How can they interact?
  • Hat state is there and how is it managed?
  • Where are certain "aspects" of the client api protocol managed? For instance, if a request for which a broker is currently not leader is received it needs to be rejected. Currently this works differently for commands vs. control messages. This should probably be unified.

Client <> Broker Cluster

  • How does the client discover the cluster?
  • How does the client know to which node to send a request?
  • How are partition-aware subscriptions managed?
  • Lifecycle of a "request" from the client perspective pobably needs a more expressive state machine since it can be multiple requests to different brokers.

Cluster

  • Which decisions / interactions require a central entity? So far, we have
    • Deployments of workflows
    • Topic Management.
Raw
next-steps.md
  • I think that the "control messages" are problematic. Some aspects that are currently done using "control messages" should probably be stream processing. Also the way they are implemented is not optimal.
  • For all "clustering" aspects above gossip & raft we need to clarify which components exist, what their reponsibilities are and how they interact
  • Define state machine for a request from the point of view of the client. Should be possible to use it both in client and broker.
  • I needs some level of better intuition about cluster-aware subscriptions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment