Our challenge was to build an Admin framework that would remain as close to core Rails as possible, while not sacrificing what we felt were the real gains that JS MVC gave us.
In discussing this prototype, we came up with a core set of assumptions that directed our decisions throughout the process:
- Server knows best. Since the server is the true source of the data, it is both timely and accessible in the easiest manner.
- The Admin is more of a document store than a rich client app
- The server can render DOM's fast.
- There should only be one render path
- Rails default toolchain is the simplest
- Minimal JS will result in simpler JS
In a traditional Rails app, documents are rendered server side and sent back to the browser for display. Any Rails app of moderate complexity will also likely use AJAX to perform operations that can be seen as independent of the core flow of the page.
Turbolinks is a simple layer of Javascript that lets Rails apps feel more like single page applications as it fetches the content of the next page via AJAX, then replaces the body and other areas of the page (javascript and CSS includes) with Javascript. This gives the user the sense of not having really left the page as the body replace operation does not refresh the entire page. One important difference is that although it may seem like a stateful load, each page should be treated as a stateless navigation.
It became obvious early on that offloading some of the simple toggling and node value setting to a binding system would be key for a simple solution. Scattered snippets of jQuery are too reminiscent of poorly structured JS that we've all experienced.
For many, re-implementing a binding system from the start again screams NIH. After some thought, we decided to purposefully build our own minimalistic binding system for a few reasons:
- Avoid biasing ourselves to any existing implementation as many come with frameworks that bring other things with them
- Discover what we really need out of a binding system
- Write down what we think the simplest solution to each problem we face is, only look for abstractions once the patterns present themselves
- Even as we approach very similar concepts to what is out there already, we come to those decisions on our own terms and have a much better understanding of why we do or don't want what we have
It's important to note that we are still open to taking one of the mature binding systems off the shelf and tailoring it to our approach. This would allow us to avoid having to re-implement all the finicky details of working with different types of DOM nodes. The only restriction we currently have is that any existing binding system we choose to make use of must be modifiable to work with the "DOM defines the data" approach we've committed to.
That said, our existing system is less than 150 lines of JS. Combined with the white magic of Turbomodules described below, we're still less than 200 lines of framework JS.
Turbomodules attempts to introduce as little Javascript as possible in to the standard Rails application flow. It also attempts to be as explicit and declarative as possible with discoverability being of utmost importance.
A Page represents a single Rails action like show or index. Each page can be seen as its own application. While many pages will make use of common modules like publishing and bulk actions, each page should be treated as a stateless app with limited scope.
/admin/pages/index.html.erb
<!-- simple helper for setting title and sidebar nav -->
<%= layout_data section: 'pages' %>
<script>
Page(function(scope) {
scope.module('bulkOperations', BulkOperations, { path: "<%= set_admin_pages_path %>", idParam: 'page_ids'});
scope.module('deletePagesModal', Modal, {id: 'bulk_delete_pages_modal'});
});
</script>
<!-- content -->
<table></table>Here, the Page definition is inline with the action HTML and attempts to be very clear about what behaviour this page has. There are no additional files to track down, just the action.html.erb file and any module JS implemenations.
Scopes can be imagined as a simple POJO with properties and functions on it. These properties and functions can be bound to in the view using our binding system and are used at run time to get the value for a node or perform an operation on an event.
Each Module is mixed in to the root page scope and can be accessed via the property key you give it in the scope.module call. In the examples above, the root scope will have two properties on it, bulkOperations and deletePagesModal. Each of these properties references an instance of their respective Modules which contain their own descendant scope (one that is effectively namespaced on the root scope).
In other words:
// Page definition
Page(function(scope) {
scope.module('bulkOperations', BulkOperations, { path: "<%= set_admin_pages_path %>", idParam: 'page_ids'});
scope.module('deletePagesModal', Modal, {id: 'bulk_delete_pages_modal'});
}
// BulkOperations module, descendant scope
function BulkOperations(scope, options) {
scope.set('multipageSelected', false)
scope.perform = function(operation, dataFunction) {
// ...
}
}Would look something like this:
{ // root scope
bulkOperations: { // module scope
multiPageSelected: false
perform: function() { }
},
deletePagesModal: Modal() {}
}Each Page takes a scope (described below) and augments it with a set of modules that contain the client side javascript needed for the page.
In the case of the pages/index, we only have two Modules: BulkOperations and a Modal. BulkOperations contain all the generic logic for performing various bulk operations on a list of resources in Shopify and Modals are a simple wrapper around a node that allow you to .show() and .hide() the instance of the modal at your discretion.
Navigations are simply <a> tags that lead to another area of the admin. Turbolinks prevents the default behaviour of navigations by an XHR GET of the DOM behind the scenes and replacing the current body + layout, js and css tag diffs. We do nothing special here other than make sure we clean our state between these somewhat stateful navigations.
You will use a navigation when you need to go from one page to another, or in our way of thinking, one app to another.
These are about as Web 1.0 as you can get. Full page (hard) refreshes, and a server side redirect or render. No state is kept around.
While you may stay within a page / app on a form submission, this implies the server has taken some action that merrits refreshing the entire page. Since the server will simply render the page in the correct state on response, we don't need to do any magic here.
Any modern web application requires an element of asynchronous behaviour that allows the user to remain in context on a page while an operation happens. Traditionally, this would be anything that we wrap in an AJAX call. In Batman, this was brought to the extreme with all but the initial page render using XHR + JSON to fetch data and bind to it client side.
In our approach, when Navigation & Form submissions are not suitable, we use our own layer of asynchronous Javascript, but rely on the server to do all the heavy lifting.
This is the conventional rails way of doing it. You perform an operation (form submission, etc), the server responds with a JS format and you get something like this:
# create.js.erb
$('#comments').append("<%= render @comment %>");This, in the most basic example seems pretty straight forward and simple to understand. Our concern with this approach, is that over time it often starts looking more like this:
# create.js.erb
<% if flash[:error] %>
Flash.error("<%= flash[:error] %>");
<% else %>
<% if high_priority %>
$('#high_priority_comments').append("<%= render @comment %>");
<% else %>
$('#comments').append("<%= render @comment %>");
<% end %>
$('#comment_count').html("<%= @total_comment_count %>");
<% end %>We feel this approach couples the view implementation details to the JS too much and results in a lot of independently innocent jQuery snippets that compound to a messy, less desirable situation that the freedom bindings + client side rendering have given us.
You could approach the problem in a similar way by returning JSON from the server and updating your DOM manually with jQuery. You may be able to use a templating library to make this easier, but we feel like this would become even more burdensome than the SJR approach as a large amount of view details that are handled by the rails partials would innevitably creep in.
This approach is often the one most people think of first. Simply structure our views into nicely separated sections, broken down with Rails' existing partial system.
As opposed to having an SJR response do the $().append, you could move that to a controller sitting in JS land that would grab the HTML template returned by the server. Perhaps a better example would be as follows:
<table id="comments">
<% comments.each do |comment| %>
<tr>
<td><input type="checkbox" name="comment_ids" value="<%= comment.id %>"></input></td>
<td><%= comment.body %></td>
</tr>
<% end %>
</table>
<a onclick="doBulkAction('magic')">Magic!</a>// something that resembles a controller
function doBulkAction() {
var selection = $('#comments').find("input[type=checkbox].checked") // forgive my bad selector
var operation = $.post(/* stuff */);
operation.done(function(response){
$('#comments').html(response);
});
}# comments controller
def show
# a bunch of stuff to set up show view ivars
end
def bulk_operation
do_it
render partial: 'comments', @blog.comments
endThis approach is fairly straightforward to understand, seems to have a fairly repeatable pattern of $('#').html() replacement and may even help force devs to break their code up even better.
We had a few concerns with this approach initially:
- The number of partial end points will likely skyrocket
- If the partials start to cross-cut concerns, common set up may start to get messy between the actions. In this example, both
showandbulk_operationwould need to load@popular_stuffif all of a sudden thecommentspartial wanted to use it. - If a given action needs to update many areas of the screen, you either need to refresh larger areas or have multiple partial round-trips server side to refresh your screen. This would all have to be managed client side and we felt it may be rougher albeit a more rare scenario.
Just in case we've lost track, we're talking about soft-refreshing a page. This is the case where you need to update a portion of the page with some new data (think bulk actions) while leaving another form on the page (search box) untouched. We also would like to have the items (if they're still around) checked off when we get back from our action.
This approach returns to some of our early goals:
- Server knows best on how to render stuff
- We want one "draw loop", one canonical path to follow to see how a page is rendered
- We don't want to manage JS snippets that replace sections of the page
To follow the more complicated bulk actions example above, here's what our current situation looks like:
<table id="comments" refresh="comments">
<% comments.each do |comment| %>
<tr>
<td><input type="checkbox" name="comment_ids" value="<%= comment.id %>" bind="comments.<%= comment.id %>.selected"></input></td>
<td><%= comment.body %></td>
</tr>
<% end %>
</table>
<a click="bulkActions.doBulkAction('magic')">Add a comment</a>// something that resembles a controller
function doBulkAction() {
var resourceIds = // grab array of ids
var operation = $.post(/* do the action */)
// on success, refresh
operation.done(function(response){
Shopify.refresh({ onlyKeys: ['comments'] })
});
}What is a Page.refresh?
Page.refresh can easily been imagined as a redraw on a back buffer. In fact, this is all that Turbolinks really does before it replaces the entire body contents with the new page. In this case, we're being a little more selective.
We want the full redraw of the page to happen server side as it removes any need for us to care about how the DOM is structured and what parts of it will change. Unfortunately we can't just replace the entire body in this case because we may have a form field that we've started to fill out before performing our bulk operation and we don't want to lose that data.
Enter the refresh key. refresh is a way of hooking a DOM node or a set of DOM nodes, to be optionally refreshed when we perform a back buffer redraw by round tripping a simple /show action to the server. In the example above, any DOM node with the refresh="comments" attribute will be grabbed from the back buffer and replaced in the live screen.
This would also allow you to have a heading like this anywhere else in your DOM (not just the comments partial) and update it by adding the refresh key hook.
<h2 id="total_comments" refresh="comments">Your total comment count is: <%= @total_comment_count %></h2>This is really exciting and powerful as you can make use of the full extent of your Ruby model without having to worry about any partial setup, JS snippets or multiple XHRs to fetch all the data you need.
Imagine bulk actions where a set of items can be selected and an operation applied to them. After our refresh, we'd like to carry over our selected state so that each check box that was previously selected remains so. In this case however, we had to refresh our comments table because a bulk operation was performed.
Turbomodules provides you with a lastScope option on the passed in scope that allows you to re-initialize your module with any transient state you may need to reapply. The only rules here are that the state must be in the scope and that you won't get a lastScope if you do anything other than a Page.refresh as any other operation is considered stateless.
if (scope.previousScope) {
var selectedState = scope.previousScope.get('selected')
for (i in selectedState) {
if (selectedState[i]) {
scope.set('selected.' + i, true);
}
}
}We feel this does a nice job of decoupling the need to know what DOM elements may be updated, or to make sure you get enough data to redraw the screen as you'll always have the whole data to choose from.
Testing has admittedly taken a back seat so far in the prototyping under the following assumptions:
- A large portion of tests we currently imagine when thinking admin will go away since there is no API layer, no model layer, no routing layer
- Our modules are very straightforward and could easily be tested in any JS unit testing framework.
- We want to keep our JS to a minimum with this approach, therefore there will likely be orders of magnitude less JS testing than we currently imagine.
- More testing will be done in rails (functional tests include ERB rendering)
- We have not solved the E2E problem yet and are not trying to solve it with this specific prototype.
To review the problems and challenges outlined above.
Heavy duplication of state in the Shopify object model between JS and RB
Since the only state that is ever displayed is ERB templated in, there is only one representation of the model.
Extraction of core business logic in to the client that is missing server side
There is no longer a client side per say and therefore any mature modeling will be forced server side.
Large toolchain
This cuts down the toolchain to Rails and makes use of the asset pipeline.
Effectively two stacks, equally maintained and highly coupled in development process
No longer applicable as the only JS that is required is that to prepare the data for XHR/form submissions
Isolation, though artificial, between the admin team and core Ruby/Rails developers
This should make the admin more accessible to a larger audience by reducing the amount of JS and stack complexity though cultural barriers may still need to be worked through.
High cost of onboarding new developers in to Batman framework
Batman would no longer be a custom framework and we believe the concepts and size of this framework are simple enough to grasp quickly
Hesitation to spread Batman across Shopify
Again, we believe the scope of this framework is simple enough and opt-in enough to be used as needed.
Simplify admin technology stack
This brings it back to Rails with lightweight, structured JS.
Eliminate costly state duplication
Only possible state duplication would be formatting logic for client-side bindings
Speed up merchant experience on all platforms (mobile included)
Browsers are very efficient at rendering DOMs. Early tests indicate this is much more performant than current admin. One restriction is on the assumption that the server is able to render faster than client.
Speed up development onboarding
By drastically simplifying the admin, we hope that the developer onboarding will be simpler.
Make the right thing easy and the wrong hard
Time will tell.