blog-distributed-systems.md

Applied distributed system communication stuff

Scenario

Frontend API (FA):

1. User adds wallet
2. User KYC verifies
3. User changes wallet
4. User changes wallet
5. User changes wallet
6. User tries to change wallet but hits change limit enforced by FA

Backend Endpoint (BE):

Needs to add any wallet to allowlist if it's been KYC-verified.

What events should the FA publish to the bus for BE consumption?

Options

We need to think about which event the business cares about.

Do our systems care about the KYC verified event and need the wallet to go with it? or do they care about one of the wallet updates? or both?

If a system cares about both, they should probably be persisting ones that have the state they need to aggregate with a later event instead of sending state of multiple entities in a single event (edited)

Comm breakdowns to consider

• communication types: pub/sub (always async), request/reply (usually blocking)
• transport types: json, xml
• protocol types: amqp, https

The ones we conflate the most are pub/sub and request/reply:

• pub/sub - doesn’t know client or what it needs (within reason); instead, raises awareness (past-tense) to any clients who opt-in to consumption
• request/reply - knows who (e.g. api key) and where (e.g. api.example.com) client is and what it needs (e.g. http verbs - POST method sends entire entity schema with all it’s values)

The effects of conflating usually impact publish endpoint creation (the pub of pub/sub). For example, if we craft the pub event coming from FA to be more like a "command" or a "pull" from BE (resembles if BE was polling FA via HTTP or something), then we could end up with something like this:

{
  "kyc_verified_at": [timestamp],
  "wallet_address": [string],
  "created_at": [timestamp],
}

This looks fine but it couples the wallet address update trigger to the KYC table (requires a join before raising event). What do we do if we refactor the KYC state to a separate database? An alternative could be something like this:

// message 1 - wallet_updated
{
  "user_id": [string],
  "wallet_address": [string],
  "created_at": [timestamp],
}
// message 2 - kyc_verified
{
  "user_id": [string],
  "kyc_verified_at": [timestamp],
}
// message 3 - wallet_updated
{
  "user_id": [string],
  "wallet_address": [string],
  "created_at": [timestamp],
}
// message 4 - wallet_updated
{
  "user_id": [string],
  "wallet_address": [string],
  "created_at": [timestamp],
}

This does require the client (BE) to persist the KYC verified state of aggregate (user_id). However, if the BE cares about doing that aggregation, then it will need to take responsibility for that. The reality is that it does care about aggregating users so it should do it!

This forces the single responsibility principal onto pub endpoints and sub endpoints which empowers sub endpoints to create their own "dimension" of an entity's lifecycle (e.g. user_id in this case). We spend too much effort avoiding persistence that we turn our loose-coupling systems (bus) into reverse request/reply systems (again, imagine if BE was polling FA directly).

Analogy: circulatory system

TODO