Skip to content

Instantly share code, notes, and snippets.

@csherratt csherratt/ui.md
Last active Jun 29, 2019

Embed
What would you like to do?
Flora's Rust UI ramblings

Rust UI Difficulties

I wanted to give a little bit of a discussion on all my thinking about why UI's are a tricky to get right in Rust. I want to try and differentiate this discussion because there are a number of decent UI frameworks that have been bound to Rust. This is great! I do not want to discourage any of their work, they are wonderful members of our community.

What this is about is how it would be possible to write a good UI framework for rust that actually leveraged Rust's features set to build something unique to Rust.

I want to mention that I am not a UI expert, I am not even a Rust expert. So I will overlook solutions, and feel free to point out the errors in my logic.

Why is this hard?

There are two things that are in conflict in building a UI library in Rust. The first is ownership, and the second is event routing, and threading. These are related problems, and have effects on a UI framework as a whole.

Ownership

The biggest problem with A UI framework is that everything is UI normally reduces to a graph. This is problem because graphs are hard to represent in Rust. Well at least hard if you want any form of mutation, which you are going to want.

You can solve this in a few ways, you can throw every object into the graph in a Arc<Mutex<>>, or an Rc<RefCell<>>. To me this is a valid, but maybe suboptimal solution. It can get a bit heavy handed as I doubt there is a nice way to hide this mechanism completely from the user. Maybe if they didn't write views or widgets, but I don't think they are getting very far if they are not writing their own types to add the graph.

You do end up with the normal problems with using reference counted types, having to know what is keeping all the references alive for example. Who 'owns' what? You can also run into problems with event routing, i'll get to that in the next section. The bigger problems is leaking a bunch of widgets because they all have circular references to each other. And this could be hidden because of event routing.

You could get around using reference counting if you use something like an ECS. Everything is owned by a central store of various data-types. This means all references are just id's to lookup the data that is stored behind them. This actually has some very neat advantages in Rust, as an ECS allows expanding of values assigned to an type beyond what the designers of the type envisioned.

ECS have a few problems that make them less attractive to me, the first is a question of learning. It's something new to learn about, and new concepts make frameworks hard to learn. The second is that you still need some way to drop dead references out of the datastore. The other ugly bit is that you always have to refer to the central store to get values from the store, you gotta carry this thing with you everywhere, gross.

Events

Events are a little tricky in Rust. An onClick event for example can exist in at least three different ways.

  1. As a enumeration. This lets you pattern match on the events! Which is super idiomatic of Rust! There are two tricky parts to just passing around enums

    The first is the expressiveness problem. Basically, if I have a ButtonEvent that contains every possible event a button can produce, I quickly run into a problem if people want to expand on this functionality. They would have to wrap the entire ButtonEvent in their own enumeration, and move all the values from the ButtonEvent into their new fancy type. Yuk.

    The second problem is the question of how to pass enums around. You could pass it to a closure via a callback, or have central channels that you can read events from. The central channel has the issue of expressiveness again, imagine a enumeration with every possible event in your entire engine in it. While this would be cool, it's obvious that this would not be practical. Passing it via a closure is probably the same solution as #2 for events, but with more steps.

  2. Events via Any type casting is yet another way to handle this. Basically objects are passed Box<Any> that they have to lookup a handler for. If they have one they can be act on the event. This is nice because any type can handle any event, and can also forward events to other objects. The downsides are not terrible.

    The first downside is that using Any types to escape the type system tends to feel unidiomatic in Rust. This is a minor nit, but I always feel if the common path doesn't play to the strengths of the language you are doing something wrong.

    Secondly, there is a fixed overhead for every event that is being passed around. I honestly don't feel the overhead would be that large, but every the idea that every event in the UI is causing an allocation also doesn't feel right.

  3. Closures to the rescue! This is probably the solution that is most practical but it does have some issues. The biggest issue for me is the ownership problem. Every closure has to own a reference of every thing it touches. That means lots of cloning of references before you create a closure, and once you are in you end up with an issue where cyclic references can screw up things.

    Why is this cycle a problem? It is a problem because if a chain of closures hits the same object twice, you are going to deadlock (or panic for a RefCell).

    let mut view = View();
    let mut button = Button();
    
    view.addChild(button.clone());
    
    let onClickCopyOfParent = view.clone();
    button.onClicked = Some(Box::new(|| 
         onClickCopyOfParent.setBackgroundColor()
    ));
    
    // This sends a click event to it's children of the view, clicking the button.
    view.mouseDown(0, 0);
    
    // the stack would look like this
    //   view.setBackgroundColor()
    //   onClicked()
    //   button.mouseDown()
    //   view.mouseDown()
    //   main()
    //
    // the danger is that view had enter itself twice, in each case it would 
    // likely need some form of mutable access to it's own content. For user's
    // sake let's assume our framework here does this automagically. This mean
    // the second access to the contents of `view`would qualify as unsafe.

    There are a few ways this could be prevented. The event propagation could always be done from the 'main', meaning anytime you send an event to another type, it is always sent after you have returned from handing the event yourself.

    This is a little counter intuitive, and requires extra house keeping, but I can't see why it can't work.

Multi-threading

The last bit I wanna talk about is how multi-threading interacts with the UI. It's much easier to give up on the goals of having an easy to thread UI framework but I think this is not playing to Rust's strengths. If you can figure out a nice way to multi-thread the UI framework that doesn't just eat up overhead in context switching, that would be an ideal solution.

The problem here of course is that most platforms are not thread blind. If the UI framework in anyway depends on host UI toolkits or needs to read from the event queues we are in a situation were the UI framework has to be careful which thread work is being done on.

Am I doing this wrong?

All of these UI frameworks I have described come down to trying to emulating how object-ordinated frameworks that already exist to do UI work. I imagine this is the wrong solution in Rust, but it's also the most familiar solution to people working in existing frameworks.

@madmalik

This comment has been minimized.

Copy link

commented Mar 5, 2018

Thanks for writing this up!

I want to mention that I am not a UI expert, I am not even a Rust expert. So I will overlook solutions, and feel free to point out the errors in my logic.

First, i'd like to subscribe to your disclaimer. All that applies 100% to me ;)

I hope it's ok that i just think out loud for a moment.

All of these UI frameworks I have described come down to trying to emulating how object-ordinated frameworks that already exist to do UI work. I imagine this is the wrong solution in Rust, but it's also the most familiar solution to people working in existing frameworks.

I agree with you that simulating OOP frameworks might be a mistake.
The way classic frameworks work seems to be one the few things that maps really well to classic object oriented programming (in the java/C++, not necessarily in the smalltalk sense), which is not that surprising because the development of both was heavily intertwined. But classic OOP with deep inheritance trees and all that jazz does not mash well with rust and building GUIs that way kinda worked, but was not so great in the first place.

Of course, there are a lot of valuable lessons hidden in there, but so are in modern web development and all that functional reactive stuff of late. Maybe we should take a step back think about "how to GUI in 2018" before concrete translation to rust.

I don't want to ignore your excellent writeup about Ownership and Events (and i've some thought about that, and I'd love a thorough discussion of that), but i'm not really "there" yet.

The problems i'm seeing in general are:

  • state is not actually "solved". The classic GUI frameworks have a lot of tiny silos of state everywhere. For example, text fields own their text, which leads to problems with "syncing" all that state to the general logic, emergent behaviour because of state changes that are not anticipated etc. Sometimes this leads to programs where the general logics operates directly on the UI-component-state, which couples programs so tight to their UI that nothing can be ever properly tested or safely refactored.
    The functional reactive pattern of having a model where state lives and the rest is just a unidirectional data flow/transformation is not without problem either. Its more controlled, but without solutions to encapsulate this state further i don't see it scale very well. (but i've to add, i've got a lot less experience with that than "classic GUI programming", so please correct me here if i'm wrong)
  • UI seeps through abstraction boundaries like a warm knife through butter. All but the most simple example applications will do something different than everyone else, and the mental model the GUI framework is based on will be deformed beyond recognition to do that. I think it is not without reason that many of the more complex apps like image or audio manipulation software roll their own completely (i also think that complex software that solves hard problems is the more likely niche for rust than small simple tools with simple UI needs)
  • Native but cross platform is appealing on paper, but a dead end. I don't want to go into that to much, because it's most likely also a dead end for the discussion since this is very opinion based... i just say, i wouldn't want to create that. Chasing platform changes, driving round pegs through square holes the whole time (our GUI model to several platform GUI models, also the point before this one...) and deal with a ton of additional complexity, just to have a bad product in the end that HAS TO BE worse than their pure native counterparts. I'd be miserable. This is hard enough on its own.

"Doing GUI the rust way" is a lot about ownership and rusty APIs, but it could also be about project structure. Developing carefully together as a group, making proposals and discussing them thoroughly, iterating in prototypes before committing to a solution, be pragmatic, but take the time to do it right. I'd like to be part of something like that...

@Kleptine

This comment has been minimized.

Copy link

commented Nov 23, 2018

Can you elaborate more on the ways something like Elm/React couldn't fit the Rust model?

A react-like system solves the issues with circular ownership. The state is driven by a data object that is a DAG, and the components themselves are fairly self contained.

Elm's event system makes me think of Rust event loops and async programming. It seems like that might also go a long way to solving the circular reference issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.