Intro
Event Sourcing
James Hunter
- Twitter: @cccc00
- GitHub: orodio
LOL WAT
Instead of mutating state directly, we store the events that caused the state to mutate too.
- Like Git but for your data.
- Complete log of all the state changes.
- Better debugability and the ability to recreate states.
- Can take things like Flux to the next level.
- Can be unbelievably performant.
- Can be unbelievably concurrent.
Preview
We are going to make a bank so we can highlight:
- some extreme basic concepts
- a super common pitfalls
Our non-event sourced feature
Given account foo has a balance of 0
When 50 is deposited into account foo
And 30 is deposited into account foo
And 60 is withdrawn from account foo
Then the balance of account foo needs to be 20But...
We lost all that intermediate state.
We can't go back in time to see past values, or see how the things that happened impacted our state.
And Then...
NEW FEATURE ALERT
PO has entered the conversation
PO: Is it possible to know the average deposit amount DEV: We only know the current balance :( DEV: If you make a story we can start tracking this for the future PO: Okay just thought I would check...
PO has left the conversation
In Event Sourced Applications
We can still have our beloved current state
But we also have something that could looks similar to the following
import { SET_BALANCE, DEPOSIT, WITHDRAW } from "./event_types"
events = [
[ DEPOSIT, "foo", 50 ],
[ DEPOSIT, "foo", 30 ],
[ WITHDRAW, "foo", 60 ],
]We have a sequences of all the events that correspond to a mutation of state in our system
For the crime of mutating state I Sequences you to death!
Thats great and all but how does that help me?
Let us
- Learn a little about sequences
- Then come back to building our bank
Destructuring
// example 1
var [ a ] = [ 1, 2, 3 ]
// a === 1
// example 2
var [ a, b, ...rest ] = [ 1, 2, 3, 4, 5 ]
// a === 1
// b === 2
// rest === [ 3, 4, 5 ]
// example 3
var { a, b, c } = { a: 1, b: 2, c: 3 }
// a === 1
// b === 2
// c === 3
// example 4
function head ([a]) {
return a
}
// head([1, 2, 3]) === 1
// example 5
function tail ([a, ...rest]) {
return rest
}
// tail([1, 2, 3]) === [ 2, 3 ]And my axe
Lucky for us there, sequences are so common there are heaps of tools we can use to help us out
- filter
- map
- reduce
Filter
Takes a predicate function, and a sequence. The function takes the next value and returns true or false, if true the value is kept in the resulting sequence otherwise it is removed
Filter In Action
import { gt } from "./tools"
[ 1, 2, 3, 4, 5 ].filter( gt(2) ) // [ 3, 4, 5 ]
``` ^
|
[ the same as ]
|
d => d > 2
### Map
> Takes a transform function, and a sequence.
It returns a new array where the transform function has
been applied to each element in the sequence.
### Map In Action
```js
import { double } from "./tools"
[ 1, 2, 3, 4 ].map( double ) // [ 2, 4, 6, 8 ]
``` ^
|
[ the same as ]
|
d => d * 2
### Reduce
> Takes a sequence, a function, and an accumulator.
The function takes the current accumulator and the next value
of the sequence, and then returns the next accumulator.
Repeat until nothing left in the sequence, returns the accumulator.
### Reduce In Action
```js
import { add } from "./tools"
[ 1, 2, 3 ].reduce(add, 0)
[ 2, 3 ].reduce(add, 1) // add(0, 1) => 1
[ 3 ].reduce(add, 3) // add(1, 2) => 3
[].reduce(add, 6) // add(3, 3) => 6
``` ^
|
[ the same as ]
|
(a, b) => a + b
### Reduction all the way down
# Map and filter can both be implemented as a reduce function
```js
import { double, gt } from "./tools"
var seq = [ 1, 2, 3, 4 ]
// map
seq.reduce((acc, data) => {
return [ ...acc, double(data) ]
}, []) // [ 2, 4, 6, 8 ]
// filter
seq.reduce((acc, data) => {
return gt(2)(data)
? [ ...acc, data ]
: acc
})Efficiency
For each map|filter|reduce in the chain we add an additional number of iterations equal to the number of items in the sequence each map|filter|reduce iterates through
import { double, gt } from "./tools"
var seq = [ 1, 2, 3, 4 ]
seq.filter(gt(2)) // [ 3, 4 ] // 4 steps
.map(double) // [ 6, 8 ] // 2 steps
// 6 stepsSince we can write a reduce function that does a map or a reduce we should be able to do one that can do both a map and a reduce
function doTheThing (acc, data) {
return gt(2)(data)
? [ ...acc, double(data) ]
: acc
}
seq.reduce(doTheThing) // [ 6, 8 ] // 4 stepsBack on Course
So we have a history of events. And we need to find the average withdraw amount?
import { DEPOSIT } from "./event_types"
import { eventIsType, add } from "./tools"
import { hisotry } from "./history"
function calcAverageDeposit (history) {
var amounts = history
.filter( eventIsType(DEPOSIT) )
.map(([_type, _subject, amount]) => amount)
return amounts.reduce(add) / amounts.length
}
calcAverageDeposit( history ) // (50 + 30) / 2 === 40OMG
We were able to get that information, simply because we had the history.
Another day another feature
PO has entered the conversation
PO: Hey so the average deposit amount is super useful PO: But it turns out we actually wanted was the average deposit amount for a given account uuid DEV: Yeah thats no problem DEV: Get me a story and I can add that in easily
PO has left the conversation
left 4 dev
import { eventIsSubject } from "./tools"
calcAverageDeposit( history.filter( eventIsSubject("foo") ) ) // 40
calcAverageDeposit( hisotry.filter( eventIsSubject("bar") ) ) // 10
calcAverageDeposit( history ) // 18.57Replay
Given the same history, and a pure reduce function, we should be able to replay over our history and rebuild the state of our application
Well sh^t
PO entered the conversation
PO: We have a pretty serious bug. DEV: ...? PO: Our users can withdraw more money then they have. PO: This needs to be fixed right now. DEV: On it.
DEV leaves the conversation
s/bad/good
we find something like this
function calcBalanceForSubject (history, subject) {
history.filter(eventIsSubject(subject))
.reduce((balance, [type, _subject, amount]) => {
switch (type) {
case DEPOSIT:
return balance + amount
case WITHDRAW:
return balance - amount
default:
return balance
}
}, 0)
}and sure enough we can reproduce the bug with a history like this:
var history = [
[DEPOSIT, "foo", 50],
[WITHDRAW, "foo", 100],
[WITHDRAW, "foo", 50],
]
calcBalanceForSubject(history, "foo") // -100We Got This
// ...
switch (type) {
case DEPOSIT:
return balance + amount
case WITHDRAW:
return balance - amount // old
// old
// old
default:
return balance
}
/// ...Do not pass go, do not collect $200
we just done goofed
- The history never lies
- Our interpretation/projection of this history however is not always so honest
What happened
- foo deposited 50
- foo withdrew 100
- foo withdrew 50
[
[DEPOSIT, "foo", 50],
[WITHDRAW, "foo", 100],
[WITHDRAW, "foo", 50],
]What should have happened
- foo deposited 50
- foo attempted to withdraw 100 but failed (not enough moneys)
- foo withdrew 50
[
[DEPOSIT, "foo", 50],
[FAILED_WITHDRAW, "foo", 100, "not enought moneys"],
[WITHDRAW, "foo", 50],
]Final Notes
- Event sourced systems are super powerful
- Events need to be facts
- Be careful with your projections (false positives, side effects)
- This isn't the solution for everything
- CQRS and Actors pair well with event sourcing.
- There are ways of doing this where you don't have to replay
through every event in the history:
- Snapshotting
- Caching Accumulators
- Actors
- others