Clearwater and the Virtual DOM

gistfile1.md

I've been experimenting with a lot of things in Clearwater since I first introduced it at B'more on Rails. The reason I showed it to people was because I was hoping to get people talking about front-end development in Ruby with Opal (I've also given a presentation about Opal at B'more on Rails). One of the things that Clearwater does on every link click, at the moment, is rerender the entire app. It does it with a single call, so it's not rendering chunks of HTML here and there, so it's not the worst performance in the world, but I did notice it was clobbering input fields, and micromanaging event handlers (for individually rendered views and attribute-bound blocks) has been a nightmare.

Experiment #1: Components with templates

Components are an interesting way of thinking about web development. Having all of your functionality specified as smaller and smaller subdivisions of content makes perfect sense on the web.

One of my experiments has been with adding components to Clearwater. These are Ember-style components rather than React-style, so they still have templates. They're basically a controller and a view in one, and unlike controllers and views, which are instantiated once, the components were designed to be disposable, so you could do something like this in your Slim templates:

ul#todos
  - todos.each do |todo|
    = TodoComponent.new(todo: todo)

But these components have a weird caveat: anything that has events associated with it (basically, anything interactive) needs to let the renderer know about them, so everything from the top of the render tree all the way down has to pass along a reference to the renderer. You had to do this by passing the view's renderer into the component:

    = TodoComponent.new(todo: todo, renderer: renderer)

I've got a few ideas on how to make it better, but for the moment it's just a bit awkward (especially if you're using a bunch of different components in a single view) and it doesn't fix the problem of clobbering the DOM on every link click.

Also, having a lot of small components, each with their own template, means that there's a cost to adding them to your project. Not only do you have to create two files (implementation and template) for each of those components, but each template has to be required into your app, so there's a significant amount of headspace involved.

Experiment #2: Compiling Slim to React

Virtual DOM implementations have become a pretty hot topic lately. First, React shook everything up with their virtual DOM implementation and the idea of just rerendering the entire app any time there's a change. Now, Ember's doing something similar with their new Glimmer engine. I decided to try to use React as the view layer for Clearwater by compiling Slim to React components instead of HTML.

The slim gem depends on a template-compilation gem called temple, which allows you to set up a pipeline where each step in the pipeline takes care of a specific portion of the compilation. What I did was take a Slim::Engine, remove the parts of that pipeline that generate HTML and instead use my own filters to generate React code.

This kinda worked. There were a few challenges and I never quite overcame them. I also realized that React provides a lot more than I need. All I wanted was the virtual DOM. I eventually abandoned this experiment.

Experiment #3: Virtual DOM, no templates

My latest experiment has been using a virtual DOM library, wrapping it with Opal. Initial experiments have been great and I'm currently working on including it into Clearwater.

I realized that this approach would be difficult because I would have the same trouble with converting Slim to virtual-DOM nodes as I did with React, so I decided I'd try it without Slim at all, and just include a DSL for elements:

class TodoList
  include Clearwater::Component

  def initialize(todos)
    @todos = todos
  end

  def render
    div({id: 'todo-list'}, [
      h1(nil, 'Todos'),
      ul({class_name: 'todo-list'},
        @todos.map { |t| Todo.new(t) }
      ),
    ])
  end
end

It seems to work really well in my own projects so far, so I've opened a PR on the repo's new home (notice the shiny-new clearwater-rb organization) to encourage conversation about it.

I haven't merged it in yet because it changes how Clearwater renders in a way I haven't found to be backwards-compatible. I'm sure it could be done, but it would require a bunch of work I didn't feel like doing at the time. I wanted to get some feedback from others before I merge it in. Also, I'm not quite done with it.

The virtual-DOM library has a few extra features that I'd like to support, as well, like selective rendering (similar to React's shouldUpdateComponent() function). Clearwater's virtual-DOM support will get there, and once it does, I expect it to be amazing. :-)

Your DSL looks a lot like paggio, which is how Opal browser and Lissio does it. It might be nice to go that way. @meh has put a bit of thought in to paggio wrt haml like class div.class1.class2 and id's div.id! that are nice touches. Standardizing might open up to more collaboration

Yeah, the idea was that the Component mixin just adds syntax sugar. A non-routed component (such as one instantiated within another component) can just be a PORO that returns a VirtualDOM::Node when responding to render (components being used as routing targets have a few extra requirements right now, but that may be relaxed in the future). This is a valid component:

o = Object.new
def o.render
  VirtualDOM.node('div', { class_name: 'foo' }, ['bar'])
end

The mixin just adds some syntax sugar, like how React's JSX does this conversion:

<Foo bar="baz">Quux</Foo>

… becomes …

React.createElement(Foo, { bar: "baz" }, "Quux");

That is pretty much identical to how the div method transforms into virtual DOM nodes. For the adventurous, this syntax is also supported:

VirtualDOM.node('div.foo', nil, ['bar'])

Notice the tag name is div.foo. It is just as if you explicitly said class_name: 'foo'. It might even be more performant since it doesn't have to convert the hash to a POJO. The Component mixin allows you to do this using the tag method: tag('div.foo', nil, 'bar') (in fact, all of the shorthand methods are implemented in terms of tag).

I really like some of the things Lissio does and I actually wondered about doing div.foo.bar! because of Lissio. I'm not entirely married to the current DSL, but I really like it. I also wonder about being too permissive with it because, since it could be called thousands of times per render, it needs to be as fast as possible. I'm probably going to inline the tag and VirtualDOM.node methods specifically for this reason.

It would probably be a good idea to get @meh in on this discussion. I don't think mentions are working because I seem to have made this a private gist.

IMO there's a certain beauty to paggio (and all the predecessor type of ruby DSL as markup libs, markaby, builder, etc.)

• it's basically slim w/end statements, ppl like slim and haml, for a number or reasons. I prefer end, cuz not having to think about indentation (esp. when really messing around w/the markup, moving things up, down, nest, unest) and just letting the editor figure out because you've correctly identified it w/the delimiter. I have flashbacks to trying to refactor coffeescript, messing up an ident and f**king everything up - would've never happened w/the correct delimiters, IMO. YMMV
• it's all ruby, in the same way you put ruby in slim/haml, you can do it but infact stay in the same language. One of things I want to blog about Opal, esp. when using Lissio, is when I do it, basically just program everything in Ruby, so my browser dev is just ruby. There's a certain elegance to the below w/o having to 'escape ruby', IMO

div.foo
  @elements.each do |element|
    div element.content
  end
end

That being said, I'm sensitive to the idea that performance concerns may lead you away from this. I'd be curious to how it all measures out -- ultimately if you're doing some of the crazy things that I've seen ppl show React is capable of, seems like maybe you should be doing straight canvas drawing or WebGL (coz they usually are graphics like things). Perhaps you support, paggio or paggio like for things that don't need to render like a video game and then have the 'faster interface' for when you really want crazy DOM manipulations at a high rate.

I completely think you should get @meh in on this, I'm interested in 2 things for lissio, 1) virtual DOM 2) server side rendering of lissio components. I'd be curious as to his take on anything we've discussed. Also think this doesn't need to be a private gist, I think the more brain power the better, and it'd be nice to get notified on changes.