nateabele/reinventing-on-unprinciple.md

## reinventing-on-unprinciple.md

      
    Raw
  

              reinventing-on-unprinciple.md
            
          
    (Re)-inventing on (un)-principle

A super popular activity in the JS ecosystem is to reinvent things.
Sometimes for fun, sometimes for style pickiness, but usually out of ignorance that the problem domain has already been explored and broken down into fundamental Named Things™—not based on the whims of the moment, or the pendulum-swing reaction to the design pattern you got burned by in your last big work project, or looking at The Popular Thing™ and thinking of all the things you hate about it.
Some people call this reasoning from first principles vs. reasoning by analogy (hint: people call it that because That's What It's Called—this is going to come back as a theme later).
One example of this that has a long and distinguished history is programmers poisoned influenced by OO style finding lots of different ways to reinvent function composition.
For some background on what function composition is and why it's cool ← read through this deck.
On that note, today we're going to take a look at a well-meaning attempt to solve state management in browser-based applications, which focuses on something called state adapters. What's a state adapter? I was wondering the same thing. Apparently...

State adapters are objects that contain reusable logic for changing and selecting from state. Each state adapter is dedicated to a single state type/interface, which enables portability and reusability. Everywhere you need to manage state with a certain shape, you can use its state adapter.

Okay, so my two questions are:

What is state? What type of state are we talking about here?
What's in scope for managing it?

Let's look at an early example of state management from the article:
return ({
  ...state,
  checked: !state.checked,
});
Oh, okay.
So state is just values for rendering a UI, and managing just means changing. Let's see if we can get any new insights on what's really going on here by taking a second look at that definition and doing a map(replace(...)) on some of the words.

State adapters are objects that contain reusable logic for changing and selecting values. Each state adapter is dedicated to a single type of value, which enables portability and reusability. Everywhere you need to change values with a certain shape, you can use its state adapter.

Huh. Sounds a lot like functions. Let's map() over the definition again and see if it fits:

Functions are objects that contain reusable logic for changing and selecting values. Each function is dedicated to a single type of value, which enables portability and reusability. Everywhere you need to change values with a certain shape, you can use its function.

Hey, that seems like it works! So far so good.
Moving further down, here's an example of a before-and-after:
Before
on(TodoActions.createSuccess, (state, { todo }) =>
  adapter.addOne(todo, { ...state, loading: false })
),
After
on(TodoActions.createSuccess, (state, { todo }) =>
  adapter.addOneResolve(todo, state)
),
The point seems to be that before is bad, because it has more boilerplate, and after is good, because it has exchanged that boilerplate for a newly-minted Named Thing. Finally, if the shape of our values is always the same, we can roll it up to:
on(TodoActions.createSuccess, adapter.addOneResolve),
Cool. So the next question is, where did all the boilerplate go, and what does it look like now?
{
  set: (state: Option, payload: Option) => payload,
  update: (state: Option, payload: Partial<Option>) => ({ ...state, ...payload }),
  reset: (s: Option, p: void, initialState: Option) => initialState,
  setValue: (state: Option, value: string) => ({ ...state, value }),
  resetValue: (state: Option, p: void, initialState: Option) => ({
    ...state,
    value: initialState.value,
  }),
  setChecked: (state: Option, checked: boolean) => ({ ...state, checked }),
  resetChecked: (state: Option, p: void, initialState: Option) => ({
    ...state,
    checked: initialState.checked,
  }),
  setCheckedTrue: (state: Option) => ({ ...state, checked: true }),
  setCheckedFalse: (state: Option) => ({ ...state, checked: false }),
  toggleChecked: (state: Option) => ({ ...state, checked: !state.checked }),
  selectors: {
    value: (state: Option) => state.value,
    checked: (state: Option) => state.checked,
  }
}
Oh... huh.
Maybe that's better than subtly reimplementing different versions of the same function all over the place.
Wait, we've just subtly reimplemented different versions of the same function in the same place now. I mean, I guess that's better?
(Granted, in this case API is generated, so you don't have to manually type everything—for reasons we'll explore further down, this is not necessarily an improvement).
Another part of the OO poisoning influence is looking at a bunch of instances of the same thing, but seeing a bunch of different things.
One of the big insights of the Clojure community is that there really aren't that many data types. You pretty much have scalars, lists (or arrays, or vectors), and dictonaries (or maps, or records). Yes, records really are just dicts with strict typing. Sometimes OO-🧠 people say 'classes' when they're trying to think about records or dictionaries.
The real question is: What's the point? What's the big idea? What are we actually doing here?

See how booleanAdapter.setTrue became optionAdapter.setCheckedTrue?

Ah, okay, so we're taking small pieces of functionality, building them up, and assigning names to them. As it turns out, this practice has a long and well-established tradition in functional languages in the form of using function composition to build a domain-specific language, or DSL. In fact, standard practice among LISP programmers when starting any new application is to think about the entites of the domain of that application, design a DSL around those entities, then use the DSL to build the application.
So, how is that different from what's going on here?
Great question. Glad you asked.
We saw above how state adapters are really functions, but as is often the case, when people give special names to things, it's because they're trying to create a special case of that thing. However, the thing that makes functions so special is that they compose. Special-casing them often requires you to give up composition, or at least give it up in the general case, even if you get to keep it in specific cases.
The thing is, we've already seen that all values roll up to a few basic types, and that function composition is used to build expressive DSLs for applications. So it follows then that, instead of needing some other special thing for state management, we should be able to have a single, universal library of primitive operations for doing basically any kind of computation.
Turns out, that library exists. It's called Ramda. (Yes, there are others that exhibit the same properties—I like this one and it serves the illustration well). I encourage you to read through the descriptions of the functions and what they do before continuing.
As you can see, we have at our disposal a general-purpose library for composing functions that can transform any type of data. Provided that (and taking a couple liberties with the API), here's what a first pass of the above might look like:
import { always, evolve, identity, mergeLeft, mergeRight, not, nthArg, objOf, pick, prop, useWith } from 'ramda';

export default {
  set: always,
  update: mergeRight,
  reset: nthArg(3),
  setValue: useWith(mergeRight, [identity, objOf('value')]),
  resetValue: useWith(mergeRight, [identity, pick(['value'])]),
  setChecked: useWith(mergeRight, [identity, objOf('checked')]),
  resetChecked: useWith(mergeRight, [identity, pick(['checked'])]),
  setCheckedTrue: mergeLeft({ checked: true }),
  setCheckedFalse: mergeLeft({ checked: false }),
  toggleChecked: evolve({ checked: not }),
  selectors: {
    value: prop('value'),
    checked: prop('checked'),
  }
}
This has quite a number of advantages over the previous example.


You wanna talk about reducing boilerplate? There are zero explicit function definitions above. Every single function is defined in terms of other functions that come from the library.


Again, the library functions & combinators are generic, and can operate over any object or data type. You can even use them with your own custom classes just by implementing a few interfaces.


Partially applying functions lets you build up function calls incrementally, carrying around values in the newly-minted function definition itself, rather than having to pass every value in on every call, for example:


const reset = always(initialState); // <-- initialState now lives *inside* of reset

/* ... Elsewhere ... */

on(Actions.doReset, reset),
// ^ Here, *any* parameter passed in will cause initialState to be returned.
// We don't need to keep carrying it around to pass it in different places
Perhaps most importantly, the code does exactly what it says. Having explicit names and consistent naming conventions is great, but what if, given a well-designed DSL, your code is so easy to read that hiding things behind more names actually makes it worse? What if it's already so simple that implementing your reducers in-line is actually the simplest thing?
The cognitive overhead of having extra named things, and the value of locality (i.e. having things in the same place) are two topics that have already been discussed ad nauseum, so I don't feel the need to rehash them here. Hopefully the point is self-evident: having a small number of composable names that allow you to write self-describing code without needless abstraction is strictly better than the alternative.
Let's take a second look at the earlier examples, within this new paradigm.
Before
on(TodoActions.createSuccess, (state, { todo }) => ({
  ...state,
  todos: [...state.todos, todo],
  loading: false,
})),
After
on(TodoActions.createSuccess, adapter.addOneResolve),
Pretty terse, but doesn't tell you much about what we're adding one of, or what resolve means. You have to go look somewhere else. What if you could just say:
on(TodoActions.createSuccess, evolveTwo({
  todos: useWith(concat, [identity, prop('todo')],
  loading: false
})),
It says exactly what it does.
All we need is one extra helper function:
export const evolveTwo = keys => ((object, msg) => (
  Object.keys(keys).reduce((newObject, key) => mergeRight(newObject, {
    [key]: is(Function, keys[key]) ? keys[key](object[key], msg) : keys[key]
  }), object)
));
And this helper function is general. Notice I didn't call it evolveState. You can use it to build up an object any place that takes a 2-arity (i.e. a two-parameter) function. It becomes a general part of your vocabulary when working with your application.
Actually, useWith(concat, [identity, prop('todo')] is too noisy.
const addToList = field => useWith(concat, [identity, prop(field)];

/* ... */

on(TodoActions.createSuccess, evolveTwo({ todos: addToList('todo'), loading: false })),
Much better. Again, it's a general-purpose function that you can drop in anywhere you want to add an object property in one parameter and add it to a list in another.
Let's take a look at a more complex example further down:
Before
return optionEntityAdapter.updateMany(
  state.ids.map((id: string) => ({
    id,
    changes: { selected: true },
  })),
  state,
);
After
return optionEntityAdapter.setManySelectedTrue(state, state.ids);
Here, we're rolling up the ability to update a value in a list. Further down it is elaborated on thusly:
startsWithA: option => option.value.startsWith('A'),
/* ... */
entityAdapter.setStartsWithASelectedTrue
One thing that that immediately jumps out at me is, you need a custom adapter function for every predicate × update scenario. For even moderately complex apps, that's a combinatorial explosion waiting to happen.
What if instead we could just say:
evolve({ todos: map(mergeLeft({ selected: true })) })
That covers the base case, but what if we want to only update some items in a list, based on a condition? Easy.
import { map, identity, ifElse, whereEq } from 'ramda';

/* ... */

on(TodoActions.markSelected, evolve({
  todos: map(ifElse(
    whereEq({ selected: true }),
    mergeLeft({ completed: true }),
    identity
  ))
}))
Meh, still too noisy. Let's make a new combinator:
import { map, identity, ifElse, is } from 'ramda';

export const mapWhere = (pred, fn) => map(ifElse(is(Function, pred) ? pred : whereEq(pred), fn, identity));
Now we can selectively map array values based on any predicate function, or even short-hand it if we're matching a spec:
evolve({ todos: mapWhere({ selected: true }, mergeLeft({ completed: true })) })
Not too shabby. But now what if we want to filter based on a value passed in through the action?
import { ... contains, flip, whereAny } from 'ramda';

const inList = flip(contains);

/* ... */

on(TodoActions.markSelected, (state, { ids }) => evolve({
  todos: mapWhere(whereAny({ id: inList(ids) }), mergeLeft({ completed: true }))
}), state)
Looks a little awkward. We need ids out of the second parameter, but evolve() operates on state, so we have to take it in and then pass it along. It would be nice if we could just return the evolve() expression, but that would cause the reducer to replace the state value with a function, which is clearly not what we want. Once again, we can get out of this jam with another high-order function:
export const unroll = (fn) => (...args) => {
  const result = fn(...args);
  return is(Function, result) ? unroll(result)(...args) : result;
}
This allows reducer functions to return reducer functions, which enables some interesting recursive, nested, and nested recursive patterns. The simplest form of those patterns is the solution to our problems:
on(TodoActions.markSelected, unroll((_, { ids }) => evolve({
  todos: mapWhere(whereAny({ id: inList(ids) }), mergeLeft({ completed: true }))
})))
Awesome! Now you're thinking with portals functions. Ideally, you'd want to roll this directly into your action dispatcher so everything worked that way automatically without you having to call anything extra.
Now let's suppose our team has gotten tired of typing { loading: false } all over the place, and we want to make a done function part of our function vocabulary. Well, we'll rarely just be marking something as done loading. We'll usually want to do that along with some other transformation. We need our reducers to do two separate things.
We have a choice: we can make done() take a reducer and return a reducer, handling the loading field update internally, or we can realize that we just ad-hoc reinvented left-to-right function composition, and instead make it explicit:
import { ... mergeLeft, pipe } from 'ramda';

const done = mergeLeft({ loading: false });

on(TodoActions.createSuccess, pipe(
  evolveTwo({ todos: addToList('todo') }),
  done
)),
Let's take a more complicated example. Suppose we have a list of users, each with a list of shipping addresses, and a back-office admin needs to fix a typo. What would that look like in the state adapter paradigm? setStreetForAddressForUser(userIndex, addressIndex, street, state)? Are we forced to break things up and nest extra components just to avoid dealing with too many things at once?
Or, what if we can simplify our approach to state management so we don't have to? Once again, function composition to the rescue. Part of Ramda's API is something called lenses. They work like this:
import { lensProp, view, set } from 'ramda';

const user = { name: 'nate', ... /* other stuff */ ... };
const nameLens = lensProp('name');
console.log(view(nameLens, user)) // --> 'nate'

const updateName = set(nameLens, 'Nate');
console.log(updateName(user)) // --> { name: 'Nate', ... /* other stuff */ ... };
Okay, cool, but what is a lens?

In functional programming, a lens is a structure that combines program fragments (functions) and wraps their return values in a type with additional computation. In addition to defining a wrapping lens type, lenses define two operators: one to wrap a value in the lens type, and another to compose together functions that output values of the lens type (these are known as lens-ish functions)

Just kidding, that's the definition of 'monad'.
A lens represents a path into a value. It could be the key of an object, the index of a list, or, for deeply-nested data structures, an array that's a few of each. (If you were expecting dense, obtuse Haskell-ese... sorry? I'll try harder next time. Maybe.)
Once you've made a lens, you can apply it to a data structure using the 3 lens operations:

view: get the value the lens is pointing at
set: the name says it all, but since lenses are functional, and functional things tend to be stateless, it returns a copy of the data structure passed in
over: behaves similar to set, but instead of an updated value, it takes a function that transforms the target value and returns a new copy of the structure, sort of like mashing view and set together with a mapping function in between

(Fun fact: you can actually derive set from over using one other Ramda function we've looked at in previous examples—for extra credit, see if you can figure out which one—the answer's at the bottom).
Anyway, as you can see in the example above, simple lenses are kinda boring on their own. Where they get interesting is when you start sticking them together (ahem, composing them). Rather than contrive examples of constructing and composing individual lenses for the sake of it, let's just cut to the chase. Here's our data:
const state = {
  users: [
    { id: 123, name: 'Nate', addresses: [{ id: 456, street: '801 My Street', city: '...' }] },
    /* ... */
  ]
}
We want to be able to update deeply-nested fields within this data by specifying a path into it. Suppose our updateAddressField action has a type like:
type UpdateAddressField = {
  user: number;
  address: number;
  field: 'street' | 'city';
  value: string;
}
Our dispatcher code ends up something like:
import { lensPath, nthArg, pipe, set } from 'ramda';

// A li'l extra short-hand to get the action message data
const msg = fn => unroll(pipe(nthArg(1), fn));

on(Actions.updateAddressField, msg(({ user, address, field, value }) => set(
  lensPath(['users', user, 'addresses', address, field]),
  value
)))
That's it. That's the whole thing. 9 times out of 10, this is all most web apps need: get a path to a value, and update it with a new one. In fact, if you make your action message just a scalar value, you can... wait for it... adapt a function signature so that it matches the dispatcher signature, like so:
import { is, lensPath, lensProp, set } from 'ramda';

const setAt = path => (state, value) => set(
  is(String, path) ? lensProp(path) : lensPath(path),
  value,
  state
);

/* ... */
on(Actions.setField, setAt('field')),
on(Actions.setOtherField, setAt(['other', 'field'])),
on(Actions.setThirdField, setAt(['deeply', 'nested', 'field'])),

/**
 *  - Aside -
 *
 * We have to wrap the return value of `setAt` in another function, because the dispatcher
 * signature is the reverse order of what `set` will accept. If only there was a function that
 * would 'flip' the parameter order of a 2-arity function. Sure enough:
 *
 * const setAt = path => flip(set(is(String, path) ? lensProp(path) : lensPath(path)));
 *
 * Much better.
 */
In fact, why not just scrap the whole action-per-field pattern and stick the path in the message? (If you're writing plain JS, the answer is 'no reason at all', but if you're doing TS, you can lose out on some type-safety guarantees without extra ceremony and some funky acrobatics, but I digress.)
Let's step it up a level one more time.
Suppose we need to change a city name a bunch of times (cities change names sometimes, it happens). Again, because we're working with functions, we can compose them. We can compose map with lenses, evolve with map, or any of our other custom combinators. In this case, I think using evolve with our mapWhere function works best:
//             ↓ { from: 'Bombay', to: 'Mumbai', field: 'city' } ↓
on(Actions.updateAll, msg(({ from, to, field }) => evolve({
  users: map(evolve({
    addresses: mapWhere({ [field]: from }, mergeLeft({ [field]: to }))
  }))
}))
This isn't too bad, but I still pine for the succintness of array lens paths. None of Ramda's built-in lens functions can handle mapping over an array, but Ramda also ships with functions to build custom lenses, and you can do pretty much anything with custom lenses. I could imagine something like this:
import { ..., lensPath, over } from 'ramda';

const Index = Symbol(/* ... */);

const customLens = path => {
  /**
   * Imagine splitting up the `path` array when you encounter an `Index` value,
   * inserting some `map()` logic, and joining it back together with standard
   * `lensPath()` lenses for the static parts.
   */
};

on(Actions.updateAll, msg(({ from, to, field }) => over(
  customLens(['users', Index, 'addresses']),
  mapWhere({ [field]: from }, mergeLeft({ [field]: to }))
)))
We're actually starting to bump up some of the ideas in LINQ, which I wouldn't consider good inspiration for structure or syntax, but it's absolutely good inspiration conceptually.

So functions are pretty powerful, and you can do some cute tricks with them, but who cares? What's the point of all this?
The point is, most of the logic of browser apps is just shoveling data around and shuffling it from one form to another. A lot of that logic is going to be generic: munging object, mapping arrays, etc. As you build it up, however, you'll find that the domain begins to needs its own specific idioms for doing that shoveling effectively without losing your marbles.
The real power in having a universal, granular, composable library of functions is that not only does it address the base case well, but it allows you to choose how to grow your application. Not the library author, not the framework author, you. You choose the names, you choose where to draw the abstraction boundaries. You grow the application by growing the language of the application.
Oh, I almost forgot about one of the best parts: where applicable, all Ramda functions are transducers—meaning they don't just operate over arrays & objects; they can operate over all your favorite value conveyor-belts: streams, observables, even promises—with no modifications.

The thing is, I already did all this 7 years ago, and packaged it into an Open Source project. I even gave a whole talk on it. I had big plans for it, but it never really took off. I could never get the types quite right (I'm not confident that TypeScript's inference will ever be powerful enough for really hardcore FP), and the routing layer turned out to be kind of a mess.
Also, the JS ecosystem has a nearly bottomless tolerance for mediocrity, and nobody ever got fired for choosing Facebook. Mostly though, I was just a lazy evangelist.
Should you use it? Meh, probably not. If you're really that into this style of programming, just try Elm.
If you refuse to try Elm, you're looking for a respite from Redux, and you're as pedantic as I am, then maybe consider it.

Conclusion

So having seen all this, we're forced to ask the question: do we need libraries for state management at all? If state is just static values to render a UI then... maybe not. Maybe we just need a good function library.
However, browsers have a lot going on that could be considered 'state' in need of management: routing & URLs, animations & page transitions, network connections & long-running requests, async values with complex dependency trees. These things go beyond updating values in response to a single, isolated event. Most of them are or are involved in multi-step transactions with a variety of failure modes that require careful coordination and thoughtful sad-path handling.
Don't get me wrong: modeling application state as a single, immutable value, and only changing it in response to messages is a clear, unambiguous win. Elm showed us that, and it's still true, even though Redux bastardized it. It's a good model, it's just not perfect. As applications grow, messages proliferate, and it's easy to get lost in the weeds.
So, if your idea of a state management library is something that provides high-level abstractions that allow you to coordinate everything that's happening on a page in a simple, comprehensible, composable way, then I would very much like to review that library and tell everyone how awesome it is, and maybe even help you hack on it.
(Incidentally, I also helped write that library as well—but we're coming up on 10 years and it's getting a little long in the tooth, and lot's happened since then).

* If you guessed always for the lens pop quiz, give yourself a pat on the back.