db-crud.md

This is a doodle about a re-frame idea that's been rattling around in my head for a few days. It hasn't quite clicked, but I don't want to let it go yet.

I propose a new standard effect.

Currently, re-frame has the :db effect which replaces what's in app-db holus-bolus.

Imagine a new effect named, say: :db-crud.

You would use :db-crud instead of :db.

The value of :db-crud would be a vector of proposed changes to app-db. Update this. Delete that. Add another.

Imagine:

   {:db-crud   [{:add  [:some :path]  :val  2}
                {:del  [:a :path]}
                {:upd  [:a :b 2 :c]   :val  "hello"}]

The format used is not very important right now. The sketch above is just for flavour.

Question: why do it this way?
Answer: more information

When you use :db to make a complete app-db change, you pass on one piece of information - state has changed. But you don't say what, how or why.

A :db-crud effect says more. It says what path(s) changed, and how. It could even say (somehow) "why" the path changed, maybe. More information.

So, how could this additional information be useful?

Assume we could look at the nominated paths looking for matches (maybe using something like meander)

On finding a match we could, what? ...

• rerender?
• dispatch an event ?
• compute further app-db crud?

A simpler, less pure variation of alex-dixon's precept?

This information could be used like a database WAL. Or a Kafka-like, event-sourcing bus-ish thingo.

There's a good, useful idea lurking in here, I think.

But ... the key is working out what useful thing can be done with the "extra information".

Every bit of flexibility is potentially useful, yes, but it always comes with a cost. I'm still trying to figure out the tangible benifit.

Can't the what and how of the state change be calculated with something like clojure.data/diff ?

good idea, worth to play with

I like the idea. However shouldn't we allow passing a function in addition to value for :upd operation?

One possible defect of :db-crud (which would be shared with the original :db efffect) is that updates express the final desired state, and not the actual programmer intent.

It's the same reason why clojure.core/swap! is preferrable to clojure.core/reset! the former is safer to use in concurrent programming, and it expresses the intent of updating, rather than expressing a final result (which provenance might not be so immediately clear in comparison, and which value would be less useful in an event log).

So, the following might could be better:

   {:db-crud/v2  [{:assoc-in  [:some :path] 2}
                  {:dissoc-in [:a :path]}
                  {:update-in [:b] inc}]}

Note that my :assoc-in and :dissoc-in are essentially equivalent to :add and :del. But they help make more evident whether :update-in would have been better, by using an homogeneous and familiar api (given the clojure.core/*-in functions)

Is {:update-in [:b] inc} a glorified version of (clojure.core/update-in [:b] inc)? Yes, but 'glorified function calls' are common/necessary in reactive frameworks (e.g. Redux actions, re-frame.core/dispatch).

I like the idea of describing the change (e.g. like assoc-in) instead of passing and mutating db directly. One thought, I find it difficult to write helpers to which pass and contribute to generating the fx context. For example, if two helpers use :dispatch then one clobbers the other. I guess the fix is some kind of consolidation step in between each helpers. Specific issue related to :db-crud is how to apply :db-crud to db so that a helper gets a unified view of db. Not sure I've communicated that well.

I would call it :db/transaction or :db/tx. Probably changes can be merged and batched. To simplify usage some helper functions like rf/assoc-in rf/udpate-in can be provided.

A few musings on the idea.

The implementation would not be trivial. E.g. [{:add [:a], :val {:b 1}, {:upd [:a :b], :val 2}, {:del [:a]}] is essentially a no-op, but only if the key :a was never present in the DB in the first place. It means that in order to undertake meaningful actions based on the changes, you have not only to consolidate the changes vector, but to also take the current DB values into account. I would not want {:del [:non-existing-key]} to trigger something important.

Updates will become more verbose since you won't be able to reuse a path. You will have to compute a value in advance and the :upd it.
It also makes it impossible to tell what e.g. a merge has changed exactly. It may be nothing, or it may be the whole map. The info is just not there.

The data schema cannot possibly take into account all possible transformations of the DB. E.g. a multi-step process that needs to check the accumulated DB value in order to compute the next step. It's still possible to implement something like this if you make it possible to consult with the consolidated set of changes. But it will be so less clear and so much more cumbersome.

Every single library that already uses :db will have to adapt its code to use :db-crud instead. Otherwise, they will face issue reports from users saying that the library hides some information that should be there.

All in all, I agree with the first comment - we can use clojure.data/diff to robustly compute the set of actual changes without having to change a single event handler. The solution to the problem of missing information is already there, because the information is already there. We have both DBs, one before the event handler has done its job and one after.
I'd go even further and say that re-frame doesn't really need to use diff by its own - it's the user that has to decide what to use, given both DBs.