(the 'grand unified' part is halfway jesting, but this is a nice generalization of the main requirements of programs.)
A program (software program), has at most two responsibilities: (1) State-management, and (2) Effects-propagation.
These terms, state-management and effects-propagation should be interpreted in the context and lingo of functional-programming. The FP theorists like to say that functions should ideally be pure, which is an attribute that contributes to them being predictable. So called side-effects (doing IO for example, or drawing something on the screen) make a function impure. All this doing stuff, we call effects propagation. Then there is state-management, which is calculating what the program thinks about itself and the world. Traditionally state-management is considered most tractable and scalable when it is handled in a purely functional way. The situation and architectural challenges in this regard are substantially more complex in a distributed system, but the principles are the same. The Flux/Redux system in React (iirc inspired by the state-management system architecture in Elm, a purely functional language), while not purely functional, can be made to approach such by adding some additional constraints such as an immutable (with the Immutable library) state variable. At any rate, the entire state-management system can be described as a single closure, a parametrically polymorphic function (the parameter is the action type). This function is structurally identical to an API, or every API is likewise a parametrically polymorphic function -- but where there are multiple parameters. It turns out that the old saying: "Everything is an API" has rich import in this sense, when I'm designing and building a rich and complex stateful application with multiple effects surfaces, everything indeed is an API. They can be composed to the moon, nested etc.
'In the wild', that is, in production, generally speaking, when we want to build a big horizontally-scalable distributed system, our server processes will be stateless, which is to say, they won't have any memory of what they are doing, they won't maintain a model of the world or anything. This is because it's pointless to maintain state in a local machine process when the state also needs to be shared across a big distributed system. So, in the world, Node.js processes, Golang processes, etc will be stateless. According to the 'grand-unified-theory' of software described above, these programs would have only one responsibility, that is effects-propagation. Being stateless, they, individually, don't have state-management responsibilities, except in the most volatile and transient sense. On the other hand, as components of a distributed system, they share some responsibility with the program of the distributed system as a whole in terms of state-management. This state-management responsibility will fall to some combination of message-queues like RabbitMQ, in-memory caches like Redis, and the stalwart databases like PostgreSQL, Cassandra, or whatever.
So, the question may arise, when would we ever need a stateful e.g. Node.js process? My answer would be only in special cases. Some kind of standalone worker that doesn't need to scale horizontally and that needs no failover protection. Actually, the only real application I can forsee for a stateful server process in the context of web-services would be as a robot, a bot, as part of a botnet. The bot might open websocket connections and maintain some kind of internal state, while reporting to and taking direction from an orchestration process. Even in this case, the best architecture might well adopt the general pattern of distributed systems, relying on a shared state-management system built of message queues, in-memory caches and so on. However, the bot, like a physical drone, might need to maintain its own state to the extent that IO could be compromised. A self-replicating bot (for example in a cyber-attack scenario) that has no certainty regarding IO status, would need to maintain autonomous state. These are very special cases, not encountered in general in the development of web-applications built with standard web-services. Single-process state-management is therefore only encountered as a requirement on the client-side, in browser applications, and then only for volatile state regarding the user-interaction; these programs will, like their server-side correspondents, generally rely on distributed state-management systems as the source of truth.
In the past year I've gotten into a flight simulation project with Rust and Vulkan. Insanely complicated is Vulkan in particular. In all that complexity and pain, what good does the above rather glib simplification serve me? Painting stuff on the screen is just one of those effects, but it's very very difficult to do right. Truly, software has immense challenges. If it didn't, it wouldn't pay so well. The conceptualization above only helps with scaling and managing at a high level responsibilities. It's really not so original, it's just an extension of what Flux/Redux gave us, which was a way to manage the immense emergent complexity (cascading side-effects!! oh my!!) associated with state-management in a growing application. What I've done is simply to show that all that's left after state-management, is so called side-effects, which are an even simpler case; in most web-services applications side effects are just IO, maybe web-sockets, so this is quite a gift. Clearly when the side effects are Vulkan graphics, there are a lot of buttons to push, and so it's hard...
The gift is knowing that any web-services applications, any stateful program, can at the root level be implemented with just two functions. This gives us a universal architecture for any stateful program. If we're doing a stateless program, it's even simpler, just one function, the effects function. That could be a RESTful API, or whatever.
# NOTE not an enforced purely functional implementation
c = console.log.bind console
state =
stuff: 32,
more_stuff: 44,
yet_more: 56
API = {}
API.reducer_1 = (payload, state) ->
# replace state and return new state
state
API.reducer_2 = (payload, state) ->
# replace state and return new state
state
reducer = (type, payload, state) ->
if Object.keys(API).includes(type)
API[type] payload
else
c "No-op in reducer with type:", type
It turns out that effects propagation is an even easier case to implement than state-management:
c = console.log.bind console
API = {}
API.do_something = (payload) ->
# doing something
api = (type, payload) ->
if Object.keys(API).includes(type)
API[type] payload
else
c "No-op in API with type:", type
I had a very elegant pattern to wrap these together I called Helix, but apparently I only implemented this at my last job, never on the webs. So I've opened a repo at:
https://github.com/kulicuu/Helix
I'll hopefully get this written tomorrow. (that would be Sept 30, 2020)