A blog post about custom runtimes

Blog.md

Design your own Runtime

By Jakub Hampl

The Elm Architecture is one of the great innovations Elm brought to the software industry. But one of the questions that often comes up is how to scale it up for bigger applications. In this article I want to show various ideas that allow you to enhance the Elm architecture to make developing larger applications easier.

But first, we should discuss the Runtime. Elm, as a programming language is completely pure (with some mostly insignificant exceptions). To make it useful, we pair it with the Elm runtime, that performs effects on our behalf and calls our code with the results.

We set this relationship up when we define main, by calling (typically) a function from elm/browser. These functions that elm/browser exposes are our hooks into the runtime, telling it what functions to call when, and from which functions to accept instructions as to which effects to perform.

However, these runtimes aren't all magic. In fact they form a hierarchy where each successive version can be implemented on top of the previous runtime. The most basic one we have access to is Platform.worker:

worker :
    { init : flags -> ( model, Cmd msg )
    , update : msg -> model -> ( model, Cmd msg )
    , subscriptions : model -> Sub msg
    }
    -> Program flags model msg

The first thing you'll notice is that there is no view function! But this function gives us the basics we need:

• it tells the runtime how to initialise the main loop
• it tells the runtime on how to update the loop
• it tells the runtime where to execute effects from and how to receive input from outside

Now imagine that Elm gave us the following function renderHTML : Html msg -> Cmd msg. We can now make our next runtime in the hierarchy, Browser.element:

renderAfterUpdate : (model -> Html msg) -> (model, Cmd msg) -> (model, Cmd msg)
renderAfterUpdate view (model, cmds) =
    ( model, Cmd.batch [ renderHtml (view model), cmds ] )


element :
    { init : flags -> ( model, Cmd msg )
    , view : model -> Html msg
    , update : msg -> model -> ( model, Cmd msg )
    , subscriptions : model -> Sub msg
    }
    -> Program flags model msg
element config =
    Platform.worker
        { init = config.init >> renderAfterUpdate config.view
        , update = \msg model ->
            config.update msg model
                |> renderAfterUpdate config.view
        , subscriptions = config.subscriptions
        }

(To see how you would actually build this using ports if you didn't have the above mentioned magical renderHTML function, I have written an article just about that.)

In a similar manner you can than build all the other functions in the Browser module, slowly layering on more functionality, that declaratively makes the applications more powerful. In the rest of this article, we are going to explore some ideas how you can also layer on more functionality and give your application superpowers.

Effect Types

I mentioned previously that your pure Elm program instructs the runtime which (side) effects you would like it to run. This is achieved through the Cmd type. However, the Cmd type has a few features that can make it challenging to use in some situations:

1. It is opaque, so there is no way to see inside it.
2. It can contain functions, so it can't even be reliably =='d.

Effect Types circumvent these problems, and involve creating a custom Effect type, that enumerates all the possible effects that your application can perform. Then you change the signature of your update functions to update : Msg -> Model -> (Model, Effect Msg). Finally, you make a wrapper that actually evaluates these effects by turning them into Cmds.

This has the following benefits:

• Testing your application either as a whole or even just testing individual update functions becomes straightforward, as all you need to do is provide a fake implementation of the function that runs effects. You can check out avh4/elm-program-test for some testing goodness based on this idea.

• You can treat things as effects that aren't. For example notifications are typically much easier to model as an effect that is managed by the runtime, even if the implementation is just some model field that get's modified and a regular Elm view. Likewise there can be top-level state (we'll talk about this more a bit later), that components might want to modify as an effect (for instance switching a colour theme).

• You can describe effects at the right level of abstraction. For instance sometimes request/response pairs of ports are unavoidable, but you can't implement them as a Cmd Msg (meaning you can't make it a Cmd that gives you a Msg back, you have to manually manage the corresponding subscription). However, nothing is stopping you implementing an Effect that stores a "request" in the model, ads the corresponding subscriptions, handles matching request/response semantics by id, timeouts and error handling, and packages all this as a single Msg back to the caller.

  In a similar manner, you can also do things like if an effect consists of making several HTTP requests in parallel, the machinery for that can also be abstracted away.

You can make your effects composable by implementing batch and map, or skip that if you don't want it. There is a lot of flexibility in the approach.

Declarative Effects

As mentioned above, Elm treats some effects specially by not using Cmds, most notably HTML rendering. Instead a declarative abstraction is offered, which runs a function of the model, then compares the current result with the previous result, and runs effects based on the diff.

In fact you could think even of subscriptions as implemented like that:

worker :
    { init : flags -> ( model, Cmd msg )
    , update : msg -> model -> ( model, Cmd msg )
    , subscriptions : model -> Sub msg
    }
    -> Program flags ({model : model, subs: List (Sub msg)}) msg
worker config =
    primitiveProgram
        {   init = config.init >> Tuple.mapFirst (\m -> {model = m, subs = []})
            update = \msg model ->
                let
                    newSubs =
                        config.subscriptions model.model

                    (added, removed) =
                        diffSubs model.subs newSubs

                    (newModel, cmds) =
                        config.update msg model.model
                in

                ( { model = newModel, subs = newSubs }
                , Cmd.batch
                    [ cmds
                    , List.map addEventListener added |> Cmd.batch
                    , List.map removeEventListener removed |> Cmd.batch
                    ]
                )


        }

The code above features a bunch of imaginary primitives and indeed all of this is actually implemented in Kernel code, but I think it illustrates how one might think of building declarative effects in their own app.

Some use-cases that I've seen is for example a function that renders web audio, but perhaps this could also be used for declaratively changing the URL for routing, or non-HTML based rendering (a PDF renderer?).

Automatic effects

Evan Czaplicki's TodoMVC implementation also sort of includes a custom runtime, and this does a simple thing: after anything happens, it serialises the model and saves it to localStorage. Similarly, your runtime can automatically do things, and your code doesn't have to worry about it.

Context

Another useful addition you can do (often in combination) with effects is providing your functions read only values from the runtime. For instance, it can be powerful to know what the window dimensions are in the view, or what the name of the current user is in the update, without having to write model logic to deal with these things is great.

view : Context -> Model -> Html Msg

update : Context -> Msg -> Model -> (Model, Effect Msg)

-- etc for init, subscriptions

You can of course provide Effects to allow the app to modify these values.

(Master) View Types

The standard Elm runtime mandates that you return Html from your views. However, Html is a format designed for writing scientific articles in the early nineties, and as such might not be the ideal thing to express highly dynamic applications in. Furthermore, to make HTML actually work practically, it has to be coupled to CSS, a language designed to separate the formatting of documents from the act of writing them. Thankfully, as elm-ui has demonstrated, we are not limited to the past.

You can design your own view language. This is particularly nice if you are already using a design system with predefined components and layout patterns. You can then translate these into a system of Elm modules that implement the system directly.

module View exposing (View, Attribute, Background, TextColor, Padding
    , render
    )

type View msg =
    Box { background : Background, padding : Padding}
    | Text {textColor: TextColor, text : String}
    -- ...
    | EscapeHatch (Html msg)

type Background =
    Level1 | Level2

type TextColor =
    Primary | Secondary | Disabled | Active


render : View msg -> Html msg

and so on.

Doing this has some interesting benefits. First of all, you can ensure that your teams produces highly cohesive interfaces, since the system either nudges or enforces this (depending if you provide an escape hatch).

Secondly, testing becomes more high level, since your views are now a custom type that you can write matching functions for trivially, and their semantics are more obvious (i.e. a tabbed interface will be a Tab constructor, rather than a see of divs with some arbitrary classes applies to them).

Taking this further

Once your views are defined in your own types, you can start building a lot of interesting capabilities on top of these. One experiment that I wrote allowed you to include local state for these view types.

TODO: embed the ellie

The trick to doing that is to actually call the users view function in update, then traverse the resulting tree and match it up to the component state. Then you run each components update function on its own state and save the resulting tree in the model. When it comes time to calling view, you simply transform this resulting tree into Html. Most of the code is simply bookkeeping to match the view tree to the state tree. Is this is a good idea? I think a lot more testing would be needed to find out.

Another useful thing that is easy to implement is extending the context idea into the view:

withDeviceDimensions : ({ width : Int, height : Int } -> View msg) -> View msg

--

view =
    row [ ]
        [ withDeviceDimensions (\{width, height} ->
            svgDrawing [
                text_ [ x (width / 2) ]
                    [ text "Hello from the middle of the screen!" ]
            ]
        )]

This can be particularly helpful for localisation, as it frees you from needing to pass the locale all over the place.

I would like to also see if these tricks could be nicely combined with animation, so we can have neat enter and leaving animations without giving up the Virtual DOM style of programming.

URL Coders

I have to admit that one of my least favourite core Elm apis is Browser.application. Even after years of using it, I still find the combination of onUrlRequest and onUrlChange and how they interact with pushUrl confusing. Every time I need to use a Url.Parser, I stare at its type signatures with puzzlement. Perhaps frontend routing is confusing simply by its nature.

However, that doesn't mean we can't try fixing it. I've written a very rough sketch of an alternative API, that would be suitable for some kinds of apps. I foresee it most useful for apps that truly have a central model, like calculators, dashboards, maps, etc. On the other hand content based apps might want to stick with the builtin (or just with server side routing).

The main idea is to treat URLs as a encoded version of the model. And for encoding and decoding we have very nice tools in Elm: encoders and decoders. Leonardo Taglialegne has extended this idea into that of Codecs, which combine encoders and decoders into a single structure, that (mostly) guarantees that they are mutually compatible. So the idea is to build a codec for URLs, but of course we typically don't want to store the whole model in the URL, hence we would call it a lossy encoding.

That means that the encoders would loose data from the model, and the decoders would need ways to recover a model (possibly not the same one). This recovery can either simply be providing a default value, or it can be a side-effect like reading something from the environment (i.e. flags or Browser.getViewport) or recovering the whole value by fetching the data from a server.

This would allow to get rid of init, onUrlRequest, onUrlChange, pushUrl, replaceUrl and Url.Parser at the cost of providing a single URLCodec, that unifies all those things.

Admittedly this is the least fleshed out idea out of all of the above, but the point is to encourage you to find better APIs that fit the app you want to write.

Make it your own

This wasn't meant as a list of things you need to do, but more a menu of delicious dishes that you can combine into making your application code clean and expressing the ideas that it represents rather than a lot of mechanical details mixed in with the business logic. Feel free to discuss this article on the Elm slack.