Instantly share code, notes, and snippets.

Embed
What would you like to do?
MelbJS - Nov 11th 2015

Intro

Event Sourcing

James Hunter

  • Twitter: @cccc00
  • GitHub: orodio

LOL WAT

Instead of mutating state directly, we store the events that caused the state to mutate too.

  • Like Git but for your data.
  • Complete log of all the state changes.
  • Better debugability and the ability to recreate states.
  • Can take things like Flux to the next level.
  • Can be unbelievably performant.
  • Can be unbelievably concurrent.

Preview

We are going to make a bank so we can highlight:

  • some extreme basic concepts
  • a super common pitfalls

Our non-event sourced feature

Given account foo has a balance of 0
When 50 is deposited into account foo
And 30 is deposited into account foo
And 60 is withdrawn from account foo
Then the balance of account foo needs to be 20

But...

We lost all that intermediate state.

We can't go back in time to see past values, or see how the things that happened impacted our state.

And Then...

NEW FEATURE ALERT

PO has entered the conversation

PO: Is it possible to know the average deposit amount DEV: We only know the current balance :( DEV: If you make a story we can start tracking this for the future PO: Okay just thought I would check...

PO has left the conversation

In Event Sourced Applications

We can still have our beloved current state

But we also have something that could looks similar to the following

import { SET_BALANCE, DEPOSIT, WITHDRAW } from "./event_types"

events = [
  [ DEPOSIT,  "foo", 50 ],
  [ DEPOSIT,  "foo", 30 ],
  [ WITHDRAW, "foo", 60 ],
]

We have a sequences of all the events that correspond to a mutation of state in our system

For the crime of mutating state I Sequences you to death!

Thats great and all but how does that help me?

Let us

  • Learn a little about sequences
  • Then come back to building our bank

Destructuring

// example 1
var [ a ] = [ 1, 2, 3 ]
// a === 1


// example 2
var [ a, b, ...rest ] = [ 1, 2, 3, 4, 5 ]
// a === 1
// b === 2
// rest === [ 3, 4, 5 ]


// example 3
var { a, b, c } = { a: 1, b: 2, c: 3 }
// a === 1
// b === 2
// c === 3


// example 4
function head ([a]) {
  return a
}
// head([1, 2, 3]) === 1


// example 5
function tail ([a, ...rest]) {
  return rest
}
// tail([1, 2, 3]) === [ 2, 3 ]

And my axe

Lucky for us there, sequences are so common there are heaps of tools we can use to help us out

  • filter
  • map
  • reduce

Filter

Takes a predicate function, and a sequence. The function takes the next value and returns true or false, if true the value is kept in the resulting sequence otherwise it is removed

Filter In Action

import { gt } from "./tools"

[ 1, 2, 3, 4, 5 ].filter( gt(2) ) // [ 3, 4, 5 ]
```                         ^
                            |
                     [ the same as ]
                            |
                        d => d > 2





### Map

> Takes a transform function, and a sequence.
  It returns a new array where the transform function has
  been applied to each element in the sequence.





### Map In Action

```js
import { double } from "./tools"

[ 1, 2, 3, 4 ].map( double ) // [ 2, 4, 6, 8 ]
```                   ^
                      |
               [ the same as ]
                      |
                  d => d * 2





### Reduce

> Takes a sequence, a function, and an accumulator.
  The function takes the current accumulator and the next value
  of the sequence, and then returns the next accumulator.
  Repeat until nothing left in the sequence, returns the accumulator.





### Reduce In Action

```js
import { add } from "./tools"

[ 1, 2, 3 ].reduce(add, 0)
   [ 2, 3 ].reduce(add, 1) // add(0, 1) => 1
      [ 3 ].reduce(add, 3) // add(1, 2) => 3
         [].reduce(add, 6) // add(3, 3) => 6
```                 ^
                    |
             [ the same as ]
                    |
             (a, b) => a + b





### Reduction all the way down

# Map and filter can both be implemented as a reduce function

```js
import { double, gt } from "./tools"
var seq = [ 1, 2, 3, 4 ]




// map
seq.reduce((acc, data) => {
  return [ ...acc, double(data) ]
}, []) // [ 2, 4, 6, 8 ]




// filter
seq.reduce((acc, data) => {
  return gt(2)(data)
    ? [ ...acc, data ]
    : acc
})

Efficiency

For each map|filter|reduce in the chain we add an additional number of iterations equal to the number of items in the sequence each map|filter|reduce iterates through

import { double, gt } from "./tools"

var seq = [ 1, 2, 3, 4 ]

seq.filter(gt(2)) // [ 3, 4 ]  // 4 steps
   .map(double)   // [ 6, 8 ]  // 2 steps
                               // 6 steps

Since we can write a reduce function that does a map or a reduce we should be able to do one that can do both a map and a reduce

function doTheThing (acc, data) {
  return gt(2)(data)
    ? [ ...acc, double(data) ]
    : acc
}

seq.reduce(doTheThing) // [ 6, 8 ] // 4 steps

Back on Course

So we have a history of events. And we need to find the average withdraw amount?

import { DEPOSIT } from "./event_types"
import { eventIsType, add } from "./tools"
import { hisotry } from "./history"

function calcAverageDeposit (history) {
  var amounts = history
        .filter( eventIsType(DEPOSIT) )
        .map(([_type, _subject, amount]) => amount)

  return amounts.reduce(add) / amounts.length
}


calcAverageDeposit( history ) // (50 + 30) / 2 === 40

OMG

We were able to get that information, simply because we had the history.

Another day another feature

PO has entered the conversation

PO: Hey so the average deposit amount is super useful PO: But it turns out we actually wanted was the average deposit amount for a given account uuid DEV: Yeah thats no problem DEV: Get me a story and I can add that in easily

PO has left the conversation

left 4 dev

import { eventIsSubject } from "./tools"

calcAverageDeposit( history.filter( eventIsSubject("foo") ) ) // 40
calcAverageDeposit( hisotry.filter( eventIsSubject("bar") ) ) // 10

calcAverageDeposit( history ) // 18.57

Replay

Given the same history, and a pure reduce function, we should be able to replay over our history and rebuild the state of our application

Well sh^t

PO entered the conversation

PO: We have a pretty serious bug. DEV: ...? PO: Our users can withdraw more money then they have. PO: This needs to be fixed right now. DEV: On it.

DEV leaves the conversation

s/bad/good

we find something like this

function calcBalanceForSubject (history, subject) {
  history.filter(eventIsSubject(subject))
         .reduce((balance, [type, _subject, amount]) => {
           switch (type) {
             case DEPOSIT:
               return balance + amount

             case WITHDRAW:
               return balance - amount

             default:
               return balance
           }
         }, 0)
}

and sure enough we can reproduce the bug with a history like this:

var history = [
  [DEPOSIT,  "foo", 50],
  [WITHDRAW, "foo", 100],
  [WITHDRAW, "foo", 50],
]

calcBalanceForSubject(history, "foo") // -100

We Got This

// ...
  switch (type) {
    case DEPOSIT:
      return balance + amount

    case WITHDRAW:
      return balance - amount  // old
                               // old
                               // old

    default:
      return balance
  }
/// ...

Do not pass go, do not collect $200

we just done goofed

  • The history never lies
  • Our interpretation/projection of this history however is not always so honest

What happened

  1. foo deposited 50
  2. foo withdrew 100
  3. foo withdrew 50
[
  [DEPOSIT,  "foo", 50],
  [WITHDRAW, "foo", 100],
  [WITHDRAW, "foo", 50],
]

What should have happened

  1. foo deposited 50
  2. foo attempted to withdraw 100 but failed (not enough moneys)
  3. foo withdrew 50
[
  [DEPOSIT,  "foo", 50],
  [FAILED_WITHDRAW, "foo", 100, "not enought moneys"],
  [WITHDRAW, "foo", 50],
]

Final Notes

  • Event sourced systems are super powerful
  • Events need to be facts
  • Be careful with your projections (false positives, side effects)
  • This isn't the solution for everything
  • CQRS and Actors pair well with event sourcing.
  • There are ways of doing this where you don't have to replay through every event in the history:
    • Snapshotting
    • Caching Accumulators
    • Actors
    • others

LOL WAT

Redux is event sourced

CQRS Actors, CSP

Preview

Not a real bank By reading this you agree that I am not responsible for any outcome of you actually making a bank and basing the underlying architecture on this presentation

Our non-event sourced feature

foo.deposit(50)
foo.deposit(30)
foo.withdraw(60)
foo.balance() === 20

Methods are called directly on where the data is, and mutate it in place.

But...

Totally zen

And Then...

NEW FEATURE ALERT

PO = Product Owner

they probably wants to improve the UX or base features off of data or some crazy thing like that, silly project owner.

to cry

In Event Sourced Applications

foo.balance() === 20
export var SET_BALANCE = "SET_BALANCE"
export var DEPOSIT     = "DEPOSIT"
export var WITHDRAW    = "WITHDRAW"

I am trying to keep this simple Event usually have a little more info :P

For the crime of mutating state I Sequences you to death!

This stuff is almost all dealing with sequences anyways

Destructuring

I lied, these are definitely not sequences sequences are next I promise

Filter

filter :: [a] -> (a -> Bool) -> [a]

A predicate is function that returns True or False

Filter In Action

export function gt (value) {
  return function partially_applied_gt (data) {
    return data > value
  }
}

Map

map :: [a] -> (a -> b) -> [b]

Map In Action

export function double (data) {
  return data * 2
}

Reduce

reduce :: [a] -> (b -> a -> b) -> b -> b

Reduce In Action

export function add (a, b) {
  return a + b
}

Efficiency

NOTE: we arent actually going to do the stuff in this slide in the rest of the presentation but it is important to know that it can be done this way.

    cause 'FFICIENCY

It only needs to go through the sequence once

Back on Course

export var eventIsType = expected => ([type]) => expected === type
import { SET_BALANCE, DEPOSIT, WITHDRAW } from "./event_types"

export var history = [
  [DEPOSIT,  "foo", 50],
  [DEPOSIT,  "foo", 30],
  [WITHDRAW, "foo", 60],
]

OMG

And the power of sequences

Another day another feature

didnt cry this time +10 points

left 4 dev

export var eventIsSubject = expected => ([_type, subject]) => {
  return expected === subject
}
export var history = [
  [DEPOSIT,  "foo", 50],
  [DEPOSIT,  "foo", 30],
  [DEPOSIT,  "bar", 10],
  [DEPOSIT,  "bar", 10],
  [WITHDRAW, "foo", 60],
  [DEPOSIT,  "bar", 10],
  [DEPOSIT,  "bar", 10],
  [DEPOSIT,  "bar", 10],
  [WITHDRAW, "bar", 10],
]

Replay

beware of side effects (notifications)

Events vs Commands

Well sh^t

pff, over react much, we are a bank we are made of money

We Got This

// ...
switch (type) {
  case DEPOSIT:
    return balance + amount

  case WITHDRAW:
    return balance >= amount  // new
      ? balance - amount      // new
      : balance               // new

  default:
    return balance
}
// ...

Do not pass go, do not collect $200

We broke the first rule of event club :(

foo withdrew that money foo would be totally pissed if they had to give it back now especially since its completely our fault

More Resources

James Hunter

Martin Fowler

MSDN

Other

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment