These are some notes about a more efficient asset compiler than what is currently being used in Rails 3.2.x

asset-compiler-proxy.md

Improving Asset Compilation in Development

In order for this to work the following changes will need to be made in preparation:

• Change js to use require.js to specify dependencies
• Change html to use async loading of the js

Compilation

• Source files are monitored for changes using guard or something similar.
• When a source file is changed it is compiled to js and cached on the file system with the same mtime as the source.

Proxy Cache

• The Proxy intercepts all js requests.
• If the cached js file has the same mtime as the source, serve the file.
• If the cached js file has a different mtime (or does not exist), compile it and cache the file on the file system with the same mtime as the source, then serve the file.
• Conditional GET should be used when possible to save transfer time.

Most of the time the watcher would recompile the source before the request is made. The only reason the proxy exists is to catch cases where there is a race condition between the watcher and request.

Advantages

• Async loading would allow the js to be downloaded in parallel.
• Cache hits would be fast since it would be on stat per request.
  • A persistent cache could be made to save this work, provided the "watcher" had a way of signaling the proxy cache to invalidate changed URLs.
• Minimal recompilation would happen. Instead of having to walk the entire source tree to look for changes, we simply allow the changes to be made as the source changes and serve (mostly) static requests.
• Compiling a single changed file should be extremely fast, especially since we're working on "leaf nodes".
• The sprockets approach recompiles the application.js each time and checks everything for syntax errors in the rails app before the HTML is sent to the browser in the first place. This completely bypasses this delay.
• Does not have to be framework specific, can even be written in another language besides ruby and shared between multiple frameworks.

Problems

• When there is a syntax error, it won't get reported by the rails app since it is no longer doing up-front syntax checking on each request. However, the background watcher can report this immediately when the file is saved.
• This requires extra processes to run alongside the app. The watcher could be separate from the proxy app, or it could just be running in another thread depending on the design. At the very least it does mean one more moving part.

Interesting notes. I think if this were a Rack app instead of a separate process, you might get a lot more adoption in the community, since the setup will be considerably easier. Also, asset compilation solutions in (for example) the Python/Django community are pretty significantly different so this solution may not be as applicable outside of the Ruby community (but making a Rack app would still work well with things like Sinatra / Padrino).

@DavidAllison yeah, I was thinking the first version could be made in Rack. Rack should be plenty fast serving static assets, especially considering most responses will be 304's. The bottleneck will be asset compilation and that's pretty much going to be the same speed in any language I use considering I'll be using standard compilers.