rkh/two-oh.md Secret

## two-oh.md

      
    Raw
  

              two-oh.md
            
          
    Sinatra 2.0

I have been contemplating a major Sinatra rewrite/refactoring for a few years now. In fact, I have started work on this a couple of times, but have thrown away the resulting code every single time.
I think Sinatra is great and there's nothing fundamentally wrong with the current code base, but I also think that a major replumbing could bring some neat advantages and make sure Sinatra stays on par with the developments in Ruby land.
This document is to outline some of my thoughts on Sinatra 2.0, everything is obviously up for debate, including the overall question if there should be a Sinatra 2.0.
Overview

General ideas:

The main API stays the same, most people shouldn't notice that they upgraded to Sinatra 2.0. I do not expect 100% backwards compatibility, but only for features people don't usually rely on.
Performance for common use cases should stay the same or be improved.
A cleaner code base. I expect the code base to stay small. Certain parts for the code are currently quite bloated (compile/compile!, distpatch/invoke, set, our template logic come to mind).
It will stay simple, it will stay easy.
Leverage Ruby 2.0 features, like keyword arguments.
Add a new extension and hooks API. Sinatra currently supports hooks, but only very inconsistently and with not enough meta data. This has lead to some ugly patching, both in Sinatra and in external tools like New Relic.

Things I would like to separate out into gems:

The logic for turning string patterns into regular expressions. This has already happened and the library is already used by other projects and in production systems (University of Copenhagen, Travis CI). The sinatra gem would depend on this gem.
Development mode logic. I think all the Sinatra logic for development mode should move to a separate gem. The main Sinatra gem would then try to load it in development mode and display a warning if this wasn't possible. Doing this would allow the development mode to have additional dependencies, like better_errors, which then don't bloat the production setup, and should make reading through and understanding the main code base easier. The sinatra gem would not depend on this gem.

Big unknowns:

What's happening with Rack? This might involve talking to rails-core (especially @tenderlove).

Internals

Composition

Sinatra has a fairly large number of private and public methods in Sinatra::Base. All its features are implemented in the same class. This leads to clashes, especially when someone tries to name a setting the same as one of these methods (like routes). This also means the method cache for the class gets busted quite frequently on MRI. I propose using composition (ie, a router  object, a settings object, a template renderer object) and only have the public API and the application's logic on the Sinatra::Base subclass.
No more Monkey Patches

A few monkey patches have been sneaking in: Sinatra is for instance patching Rack::CommonLogger and extending certain String objects (see render). I think it shouldn't.
Route arguments

Since Ruby 1.9, it is possible to reflect on the parameters of a block or method. I think we should use this to pick the arguments passed to a block rather than just passing captures.
get '/:category/:page?' do |page: 1|
  # page is "1" for "/blog/"
  # page is "2" for "/blog/2"
end
This could also include query parameters:
get '/search' do |q:|
  # q is "example" for "/search?q=example"
  # status is set to 400 if q is missing
end
Extension and Hook API

I'm not 100% sure what this should look like. I few things that come to mind:

It should be possible to track metrics in production (via things like New Relic, Librato, Skylight, etc) without having to patch Sinatra::Base.
There should be a sane reflection API (list all routes, etc). People hack around with Sinatra's internal data structure in 1.x, which makes it hard to change it.

A good way to have a simple yet powerful extension API would probably be to turn large parts of Sinatra itself into extensions internally (that will then be always used), examples that come to mind would be templates and logging.
Patterns

This is more or less the only part that's already done.

All the Sinatra 1.x syntax will be supported (captures, splats, optional parts).
Performance for route matching will be equal or better compared to Sinatra 1.x.
It gives more useful error messages on broken patterns (rather than generating broken regular expressions).
Allows to configure capture constraints.
It introduces a few more syntax elements.
It implements proper "semi-greedy" matching.

New Syntax Elements


New Syntax
Description
Example Pattern
Matched String


\…
escaped char
/\:name
/:name


(…)
group
/:page(.:ext)?
/example.html


*…
named splat (like rails)
/*rest
/foo/bar


…|…
union ("or")
foo|bar
foo


Capture Constraints

Note: I'm not really happy with the API for configuring it, so if you have other suggestions, let me know.
You can limit the possible matches for a capture per route:
# remove known extensions, set Content-Type accordningly
before '*:ext', capture: Rack::Mime::MIME_TYPES.keys do
  content_type params[:ext]                 # set Content-Type
  request.path_info = params[:splat].first  # drop the extension
end
Or for all routes of the app:
set :pattern, capture: {
  ext: %w[png jpg html txt], # have :ext caputre png, jpg, html or txt in any pattern
  id: /\d+/                  # have :id capture only digits in any pattern
}

get '/:id' do
  # will match '/42', but not '/foo'
end

get '/:slug(.:ext)?' do
  # slug will be 'foo' for '/foo.png'
  # slug will be 'foo.bar' for '/foo.bar'
  # slug will be 'foo.bar' for '/foo.bar.html'
  params[:slug]
end
This feature is already in use at Travis CI.
Semi-greedy matching

In Sinatra up to 1.3.x, captures are greedy and splats are none greedy. In 1.4 some hacks were added to implement semi-greedy behavior for certain special cases, though it's really more of a hack and doesn't work for a lot of cases.
Imagine the pattern :name.?:ext? (or :name(.:ext)? with the new syntax) and the input string foo.bar.html.
With greedy matching, this would be parsed as {"name" => "foo.bar.html"}, with non-greedy matching this would be parsed as {"name" => "foo", "ext" => "bar.html"} and with semi-greedy matching, this will be matched to {"name" => "foo.bar", "ext" => "html"}.
Moreover, this behavior is configurable:
get '/:slug(.:ext)?', pattern: { greedy: false } do
  # slug will be 'foo' for '/foo.png'
  # slug will be 'foo' for '/foo.bar'
  # slug will be 'foo' for '/foo.bar.html'
  params[:slug]
end
Development Tools

Sinatra should try loading the dev tools extension when started in development mode, and print a warning about it not being loaded if that fails (possibly with a link to docs).
Development tools should include (potentially using other dependencies):

Nice error pages, with stack traces, potentially a Ruby prompt (see better_errors).
A reloader, preferably an out of process reloader, like rerun/shotgun.
Ways to easily trace whats going on during a request (dev mode logger).

Things not clear to me yet

Logging

This seems to be something users tend to struggle with, maybe Sinatra should offer a better logging solution, not relying on Rack?
Exception handling

Again, something we get issues about from time to time. The logic is quite complex right now, yet it's still very easy to accidentally leak file handlers when capturing exceptions.
Rack

Rack 1.x is dying. It's also a terrible tool for anyone but people writing Rack apps directly. There has been some experimentation by Aaron Patterson for a successor, which might be used by Rails.
Rack's main strength is its support: It is the defacto standard in Ruby land.
It might be possible to keep all the Rack specific logic in one place, so it can be replaced with another implementation. However, this might also overcomplicate things.
Potential Features

Below are some features that I have considered, but I'm not 100% sure yet if they should be part of Sinatra (they could also become their own extension). I imagine this features to be quite simple to implement.
Decorator Style

class MyApp < Sinatra::Base
  get '/', provides: :html
  def index
    slim :html
  end
end
This one is so easy to implement, that I implemented it once by accident.
Right now, we collect conditions in an array sitting there and waiting for a route to be defined:
get '/', agent: /Firefox/ do
end

# equivilent to
agent /Firefox/
get '/' do
end
We also generate a method from the given block (the see generate_method and compile). I experimented with treating the get part as a condition, and using a method_added hook, mostly to have a common logic for all the different types of routing conditions we have.
This came with the decoration style definition shown above.
Link generation

The pattern logic for Sinatra 2.0 actually parses the pattern string, instead of just gsubing it in hopes it becomes a valid regexp. This also allows expanding a pattern string with a given params hash to a full URL. Link generation would be the main use case for this. I'm not sure how to best expose this as a feature in Sinatra, or if we even want to, but it might allow for a few neat features:
get '/hello/:name' do |name|
  # will redirect /hello/Frank to /hello/Sinatra
  redirect to(name: 'Sinatra') if name == "Frank"
  "Hello #{name}!"
end
On the other hand, I guess, link generation makes more sense with named routes.
Renderers

Right now, Sinatra has a hardcoded set of supported return values (ie, a String will become the response body, an Integer the HTTP status). It is possible to add support for additional response types by overriding the body method (ie, serialize a Hash as JSON). However, this breaks in a lot of edge cases and is not officially supported.
I could imagine having custom renderers, which could use the same logic we'd use for handling exceptions, routes, etc.
Defining the default handlers could look something like this:
render(String)  { |value| response.body   = value }
render(Integer) { |value| response.status = value }
This could either be a separate extension, or used internally by Sinatra.
New Syntax	Description	Example Pattern	Matched String
`\…`	escaped char	`/\:name`	`/:name`
`(…)`	group	`/:page(.:ext)?`	`/example.html`
`*…`	named splat (like rails)	`/*rest`	`/foo/bar`
`…\|…`	union ("or")	`foo\|bar`	`foo`