raggi/rack2.md Secret

## rack2.md

      
    Raw
  

              rack2.md
            
          
    Rack 2.0, or Rack for the Future

Hi all,
As most of you know, I've bene essentially maintaining Rack for the last year or
so. During this time, and for a long time before I've been paying close
attention to what is working well, and what could do with a change.
Preamble

After a lot of discussion between myself and many community members involved in
the framework and web server space (essentially racks two most knowledgable
customer groups), I have come to agree with many of their desires. Some of the
changes proposed here may raise some controversy. They break away from racks
presently "gloriously simple" design in some minor ways. It should be noted that
I/we are in no way intending to make rack complex. Most of the proposal can be
implemented in clean, idiomatic OO. When you consider the complexity argument,
please consider our goals, as racks present "simpler" design simply cannot cover
them.
If rack does not move forward in this space, if we don't make some of these
changes, we are likely to alienate some significant customers over time. Most
notably, I cannot see Rails hanging on to the rack ecosystem unless we start
providing (at the design restriction level) for their future goals. Given that
even with the present restrictions, the abstraction layer has significant value,
and the project has a "Brand" to date, it is valuable to try and deliver this as
part of the Rack library. To do so will also avoid a fracture in the community,
and hopefully provide a clear ugprade path for all involved parties.
Retrospective

The Good:


Micro applications for debugging & testing are trivial to write
The SPEC has unified almost all servers and frameworks to a single API
The SPEC has been easy to understand for almost all consumers
The repository and code base are largely very accessible
The core code base has remained relatively small
The community has gained traction, and is in a mature phase
Key pattern and anti-pattern knowledge has been established
The software is being used to power some very major internet sites
A lot of thread safety concerns are well addressed by the dup._call pattern

The Areas for Improvement:


The relationship between Rack::ContentLength and Rack::Chunked is still hard
The enforcement of stack-only scheduling (using the return value)
A lack of parent classes, abstracting away pervasive idioms
Inability to efficiently do realtime input streaming
Difficulty to perform output streaming with some otherwise useful patterns
Multipart parser could still do with becoming much more clear
Encodings are quite a big mess still, if nothing else, documentation required
Constant re-creation of decorators without caching is a performance concern
The call-stack based middleware pattern is problematic for optimization
The call-stack based middleware pattern is problematic for configuration
The call-stack based middleware pattern can be hard to understand
rackup is a half-way house, almost ready for production deployment, but not
Our sites, documentation, and examples could do with updates / enhancement

Open Questions:


Should we go for a big-rewrite, or an iterative approach?
Do ContentLength and Chunked really want to be middleware?
Can we solve many of the problems by providing a "Context" object?
Should we just absorb the patterns from other major frameworks? (req, res)
If we create parent classes for middleware, can we still keep ease of use?
Can we provide a backward compatible API, or more importantly, should we?
Do we want to split SPEC slightly, to provde app and server facing lints?
Should we publicly define a class namespace pattern for middleware?
Should we add a simple contributor agreement? (implied, not "signed")
Should we just expose IOs, or abstract away IO? The latter is better & harder
Should we go for a UCS model, specifically targeting utf8? (much easier)
How can we better support server configuration, maintaining abstraction?
How can we provide better service to our customers? (more maintainers?)
Should rackup provide a true production ready service?
Should we move to 1.9.x only (would make encodings MUCH easier)

Recommendations:

Approach:

I have had some heated debates on this topic. There is interest from certain
customer community members in the idea of a rewrite. I think this origianlly
stems from a general lack of interest from the rack core team to address some of
their larger concerns. As noted above, having a rewrite, or a new project
spawning to do this, we will see fracture in the community. As rails is one of
the major interests here, this means such a project would get a certain
traction, and a fracture would certainly occur.
I would rather go for an iterative approach, but as major changes are desired /
being considered, this will require a clear high level roadmap. We will need to
select version numbers in a good, consistent way, and be performing very regular
releases to get to our goal. I would recommend, as best as is possible (whilst
avoiding too much waterfall), that we try to provide a final SPEC target. This
should make the transition easier on the community. An iterative approach is a
proven methodology for software development. We need to unfreeze changes to
SPEC and the object protocols, without causing too regular breakage.
A backward compatible shim should be quite easy, meaning that we can provide a
very simple transition path. I am unclear at this time if it is possible to
provide a sound attempt at a limited forward compatible API. Initially, I think
the core protocol is possible, but the semantics are not - thus the requirement
for change.
Specific goals:


Implement non-stack dispatching. (No more @app)
Conditional dispatch would rely on a Response object.
Make Applications conform to a SPEC, but also provide a "recommended"
superclass.
Provide a superclass for middleware. this will abstract away common patterns.
Some of these patterns may actually be eradicated by non-stack dispatching,
if sufficient, this superclass may be unnecessary.
Provide full request and response objects at each stage of dispatch.
Make request, response, and header objects decorable, contextually, with
minimal overhead (memoized inside the original objects).
Abstract IOs and provide a full spec for their handling

Incoming IO should provide a promise, with both success and error paths
Outgoing IO should provide a future, with both success and error paths


IOs may also need to be decorable.
Sync, Async, and mixed-mode scheduling should be natively supported
The core should remain "thread independent" - we don't want to maintain
locks, this steps on scheduling requirements, instead, remain idempotent (use
context copies, immutable structures (n.b. not freeze), etc). We may need
some mutexes on Request and Response for threaded servers. Could be
decorated.
Expect Encoding.default_internal from applications, binary from servers.


TODO


Boot time configurations

Provide for boot time introspection of app data for the server, and server data for the app (example, advanced input streaming stuff, etc)


Use a single context object instead of request, response. Allows for complex proxy style operations, particularly for hand-me-downs

Also unifies the idea of middleware, instead of middleware having to wrangle two different objects.
Makes certain classes of arounds easier / simpler


Provide for OOB operations in context.


Context also allows for dissemination of request-time information, like HTTP version and certain ensuing semantics as supported by the server.


Consider adding explicit support for chunked encoding "trailers", in the hope that browsers catch on that it's a good idea (other than Opera and Curl).


HeaderHash should preserve the first case, not the last case (performance)