Notes on streaming JS & long-term code compilation caching in Chrome

notes.md

Re: http://blog.chromium.org/2015/03/new-javascript-techniques-for-rapid.html

V8 Optimisations to enable fast page startup

As mentioned in our Chromium blog post, Chrome 41 introduces support for streaming parsing of JavaScript files using the async or defer attributes. This is where the V8 parser will parse any incoming JavaScript piece-by-piece so the compiler can immediately begin compiling the AST when script loading has completed. This lets us do something useful while waiting for the page to load. Compare:

This means parsing can be removed from the critical path when loading up the page. In these cases such scripts are parsed on a separate thread as soon as the download begins, allowing parsing to complete very soon after the download has completed (milliseconds), leading to pages (potentially) loading much faster.

Why only async and deferred scripts?

A common question we get on this new feature is why it's only applied to scripts that are asynchronous or deferred - why don't all scripts benefit from this optimisation?

The reason for this is V8 believe streaming primarily benefits scripts that are particularly large. We expect scripts using async or defer to be big (such as scripts loading from CDNs, ad networks etc). Unfortunately, it's challenging to know what the size of a script is ahead of time - the HTTP content-length header isn't sent for chunked downloads.

The main reason V8 doesn't always stream is there's overhead in thread synchronization and enabling the data required available to a background thread. This is why we need some heuristics to begin streaming when it's possible to determine if a script is large enough to benefit from streaming.

Any script you've already downloaded chunks of that comprises a larger than N response is part of the current heuristics used.

Code caching and the critical path

Chrome 42 also introduces code caching. In older versions of the browser, we would load JS, compile it and then load JS from the cache and re-compile it. Code caching changes that. It stores any of the machine code generated by V8 in the browser's cache, so on subsequent loads of the same script or page the browser is able to immediately begin executing code.

This means both parsing and compilation are removed from the critical path when loading for the second time.

Challenges ahead

Some of the interesting challenges still to be solved are figuring out a reasonable policy for when it makes senses to stream. Because we can't know the exact size of a script ahead of time (again, back to the limitations of HTTP Content-Length headers) there are difficulties in deciding streaming on script size. V8 also need to look at what script types make the most sense here.

With thanks to Marja, Daniel and the rest of the V8 team for their work on these improvements

I'd like to see the data that suggests that async/defer scripts are bigger than blocking scripts. It doesn't feel likely to me.

Also, the chunked-encoding thing doesn't match up to my experience either. Had a quick click around the web and I've yet to find a chunked script.

Great work team!
Do you mind, providing any benchmark data comparing the current and previous behavior for both streaming and caching?

@jakearchibald that's a completely valid ask. I'll see if there's data we can externalise that led to this assertation.

Wouldn't the size of the stale version of the same file in the cache (if applicable) be a nice heuristic as well?

at least it should be easy add this optimisation to the other scripts when Content-Length is available and looks big enough ;-)

Wow I've always wonder, why browsers parse the same file all the time while browsing same domain.

So, if I understand it correctly:
I will set mainjs-v123.js cache to one year. User will come to page A, js file get downloaded, parsed, compiled, cached, executed. Then user move from page A to page B with same js file. Browser will take complied js from cache and just execute it.

Is this also applicable to polymer included through HTML imports? Or maybe I should ask to inline js. Polymer in 0.5 was really slow. It wasn't just possible to achieve 200ms page load. 0.8 is a lot faster, but I'm not sure 200ms is achievable (with reasonable large page).

Registered custom elements could be cached too between page transition.

W.r.t to compiled code caching, what if the actual resource ( js file ) in the server is modified? How will this be detected?

So on the previous version, chrome only cached the source code and now they are caching the compiled code also, then what is the need of caching the source code again? Or did they eliminated the source code caching?

How does code caching interact with the new Cache API? It would seem to me that the new Cache API should break code caching, because the Cache API doesn't know anything about when the code is compiled.