| title | tags | |||
|---|---|---|---|---|
On hydration: an outsider's perspective |
|
NOTE: Final draft is available here.
There is a new exciting movement in the frontend JS framework space that seeks to go beyond the traditional Single Page App architecture. This movement is led by a host of new participants (Astro, Qwik) as well as a historical outsider (Marko) and the current leader in the space (React Server Component). Here, one of the key area of research and innovation is what is called hydration, broadly understood as making server-rendered static HTML page interactive by running client-side Javascript code.
I have been enthusiastically following this development over the last several months--but even more so since I became active on Twitter a couple months ago. Although many people find Twitter to be an unbearable mess, it has been a wonderful means for me to directly engage with JS framework authors and others who actively explore and participate in this area.
Before going any further, I must confess that I have gained (partly from my past training as a student of social studies) a habit of relying on observing what people say, rather than getting my hands dirty, to grok certain matters. I'm not so proud of this habit and often wish I were someone who contributes to the space in a concrete fashion (via code & implementations), not a mere spectator commenting on others' work. Alas, old habits die hard, and I am yet to reach the level of technical competence to make such contributions.
With that out of the way, I'd like to make a quick detour from the main topic of hydration and briefly share an observation I've made over the (admittedly short) course of time as a web developer. And that observation is:
Many prominent contentions in the field of web development 1) center around language and 2) are social, not technical, in their nature.
To illustrate this point, let me begin with an example: Web Components vs JavaScript frameworks. Here's a fictional presentation of how the conversation often goes:
Foo: You don't need JS frameworks. Use WC, use the platform!
Bar: But JS is part of the platform! What's wrong with using JS framework?
Foo: WC can be used anywhere unlike component X for JS framework Y, which can't be used without that framework. That's not using the platform.
Bar: Sure, component X needs JS framework Y, but it all boils down to leveraging JS to render HTML and handle states/events. So using the platform. Then, why would you not use JS framework that does A, B and C better?
Foo: But you can't take that component X and use it elsewhere without JS framework Y, can you? Your code is tied to that framework. How's that using the platform? Plus, WC has great support for A, B and C. You can use WC framework Z for that.
Bar: So if you need WC framework to build your app, how's that any different than just using JS framework? Also, JS framework Y can compile to WC, too. It's using the platform just as much!
[So on and so forth...]
While simplistic, I believe this fictional conversation correctly depicts that the key point of contention hinges on the meaning of "the platform" or "using the platform". From this, one can reasonably conclude that 1) the contention originates from the ambiguous use of these terms and 2) inventing a better, more precise definition for these terms will resolve the contention once and for all.
Here, I'd agree with the diagnosis, but not the prescription. This is because we cannot completely fix the meaning of a word. Any word, not just "the platform".
This is partly due to the inherent limitation of our natural language. I like to think of it in terms of resolution as in our everyday language being generally low in resolution. A lot of words we think as obvious in their meaning break down when we look just a bit more closely--truth, life, matter, intelligence... just to name a few--because they cannot afford to be precise in their meaning to such a degree. In contrast, a high-resolution symbolic system of mathematics can achieve a much greater degree of precision.1
Furthrmore, the meaning of a word is ultimately determined by its usage, which varies not only over time but also across groups & contexts at a fixed point of time. This is not to suggest the state of absolute anarchy with respect to the meaning of any word. A word's usage, the key determinant of the meaning, is a complex social process that tends to move slowly. If that's not the case, any communication would be hopeless.
Of course, I don't claim to be uncovering some great secret here. We all know this from our day-to-day experience, and I'm just stating the obvious.
Now, back to hydration. Before looking at the current contention over hydration, I believe it's useful to first survey how the term is defined and used in the world of web development.
Here's the first sentence of the Wikipedia article on hydration:
In web development, hydration or rehydration is a technique in which client-side JavaScript converts a static HTML web page, delivered either through static hosting or server-side rendering, into a dynamic web page by attaching event handlers to the HTML elements. [emphasis original]
And here's another definition from a web.dev article, "Rendering on the web":
Rehydration: "booting up" JavaScript views on the client such that they reuse the server-rendered HTML’s DOM tree and data. [emphasis original]
Why quote Wikipedia? Because they are one of the few online sources of information that are generally deemed credible, even authoritative in some cases. In case of web.dev, it may be considered an even better source of information since it specifically focuses on web technology and is created by a major industry player, Google. In other words, these resources are where many people would go to learn what "hydration" means and use the term accordingly.2
But what if the authors of these articles and definitions do not really work on hydration techniques?3 So let's hear from major JS frameworks and meta-frameworks themselves, starting with, of course, React.
Here's React documentation for its hydrate() API:
Same as
render(), but is used to hydrate a container whose HTML contents were rendered byReactDOMServer. React will attempt to attach event listeners to the existing markup. [hyperlinks removed]
And from Next.js documentation:
Each generated HTML is associated with minimal JavaScript code necessary for that page. When a page is loaded by the browser, its JavaScript code runs and makes the page fully interactive. (This process is called hydration.)
Also from Gatsby documentation:
Hydration is the process of using client-side JavaScript to add application state and interactivity to server-rendered HTML.
Meanwhile, Svelte goes a step further. Here's how Svelte documentation speaks of its hydrate option:
The
hydrateoption instructs Svelte to upgrade existing DOM (usually from server-side rendering) rather than creating new elements. [emphasis mine]
To be fair, a more detailed explanation can be found in SvelteKit documentation:
When fetching data during SSR, by default SvelteKit will store this data and transmit it to the client along with the server-rendered HTML. [...] Svelte will then check that the DOM is in the expected state and attach event listeners in a process called hydration.
It is noteworthy that not everyone introduces "hydration" in this broad way. In some cases, the same term is used to denote something more specific. For instance, here is how Vue.js documentation explains it:
To make the client-side app interactive, Vue needs to perform the hydration step. During hydration, it creates the same Vue application that was run on the server, matches each component to the DOM nodes it should control, and attaches DOM event listeners. [emphasis mine]
And from Docusaurus documentation:
In CSR-only apps, all DOM elements are generated on client side with React, and the HTML file only ever contains one root element for React to mount DOM to; in SSR, React is already facing a fully built HTML page, and it only needs to correlate the DOM elements with the virtual DOM in its model. This step is called "hydration". [emphasis mine]
From this collection of definitions and explanations, I belive it is fair to conclude that, in the frontend JS framework space, the term "hydration" is generally used and understood in a rather broad sense of making server-rendered static HTML page dynamic and interactive via some application JS code. How that works is mostly overlooked as an implementation detail although some hints can be found in the minority of cases. Even in such cases, the detailed mechanism is largly left unspecified.
If "hydration" means "upgrades" an HTML page to be interactive using client-side JS, what then is not "hydration"? Any page with JS would be "hydrating" by that definition. However, people don't talk about "hydration" for, say, traditional server-driven apps (Ruby on Rails, Django, etc.) with "sprinkles of" JS (often with JQuery). So where's the distinction?
I believe this distinction lies in the one-app architecture of the mainstream JS frameworks, where both the server-generated static HTML page and the "hydrated" JS-driven app are written in and managed by the same framework code (React, Vue, Svelte, etc.). In other words, "hydration", however broadly defined, does have its specific scope and context.
With this scope and context in mind, we can better understand what "hydration" does. Like most techniques in technology, "hydration" was invented as a solution to a problem, namely, SEO.
Soon after its initial public release (v0.3) in 2013, React, the first modern JS Single Page App framework, recognized that React apps were having troubles with search engine crawlers that didn't wait for React to populate the contents. This meant that any React app would do poorly on search results as their crawlers would see only its empty root element. One obvious solution to this challenge was to have a server to render the app into HTML and serve that so all its contents were available to the crawlers. This solution was named server-side rendering.4
However, supporting server-side rendering was only a half the story. React also had to somehow turn that server-rendered HTML back to an interactive client-side app intended by its developers. I don't have experience with pre-v16 React, but it appears that the original, pre-v16 solution to this problem was to simply render() the whole DOM for the app from scratch on the client side and replace the server-generated DOM with it if necessary.5
The situation has somewhat changed with React v16, which included an improved server renderer and new ReactDOM.hydrate() API that "will attempt to reuse as much of the existing DOM as possible" (quote from "React v16.0").6 Even so, React still had to perform all the work needed to create the VDOM representation of the app, in a top-down fashion.
And that is, in essense, how all the major JS frameworks currently "hydrates".7
While the status quo has been (and still is) good enough for the majority of web developers, many have long been searching for a better approach.
That said, 2019 seems to be the year that the issue of "hydration cost" came to the forefront. The aforementioned "Rendering on the web" article was published in February 2019, not only introducing relevant concepts but also discussing the limitations of current practices ("one app for the price of two") as well as strategies to optimize rendering performance such as "progressive rehydration" and "partial rehydration". In May 2019, a Google I/O 2019 conference talk with the same name (and a co-author of the article, Jason Miller, as one of the presenters this time) further exposed the collective consciousness of the web developers to these ideas. And we can see that React metaframeworks were being requested to support such strategies later that year (e.g. Next.js & Gatsby).8
In the following years, actual solutions started being built and released. I won't get too deep in discussing all such solutions, but here are some of the most notable open source projects and when they first appeared in public:
- Elder.js is a Svelte metaframework/site generator that supports partial and progressive hydration. Its initical commit was made in August 2020. Elder.js soon reached v1.0 in September 2020.
- Astro is a BYOF (Bring Your Own Framework) metaframework/site generator that supports partial and progressive hydration with any major frontend JS framework (React, Vue, Svelte, Solid, etc.). Its initial commit was made in March 2021. Astro released its first v1.0 beta (with much fanfare🎉) in early April.
- îles is another BYOF metaframework/site generator that supports partial and progressive hydration. It originally supported Vue only but later added multi-framework support (no React). Its initial commit was made in August 2021. îles is under active development and currently at v0.7.
In addition to these new solutions, Marko, a long-time outsider JS framework created by eBay, gets an honorable mention. It has been supporting partial hydration as early as in 2014.
The aforementioned solutions mostly followed the course set by the much discussed strategies, i.e. partial and progressive hydration. This, to be clear, is no small feat.
Meanwhile, Miško Hevery, the creator of AngularJS and an ex-core team member of Angular, had a different thought. Instead of optimizing for less hydration (in a more technical sense), he envisioned bypassing it altogether. In February 2021, still at Google then, Miško Hevery started working on a new framework called Qoot. The original tagline for Qoot was:
An Open-Source framework designed for best possible time to interactive, by focusing on resumability of server-side-rendering of HTML, and fine-grained lazy-loading of code. [hyperlinks removed; emphasis mine]
The project was later renamed to Qwik when Miško Hevery left Google to join BuilderIO in May 2021. Qwik's API has since gone through a series of refinement to improve DX, but the core mechanism has mostly remained unchanged. Indeed, a quick inspection of the documentation files' Git history shows that the original contents as well as their organization have stayed more or less the same.
So what's resumability and how's it different from hydration? I don't pretend to be the best person to explain this (rather technical) point, so I'll just link a relevant page on the official documentation site (which, by the way, only just launched earlier this month!). And here's the executive summary:
A good mental model is that Qwik applications at any point in their lifecycle can be serialized and moved to a different VM instance. (Server to browser). There, the application simply resumes where the serialization stopped. No hydration is required. This is why we say that Qwik applications don't hydrate; they resume.
Then, just recently, Miško Hevery finally declared that "Hydration is Pure Overhead"!
So have we finally conqured hydration? Is it just dying the slow and painful death while the major JS frameworks catch up or simply get replaced by the new generation of frameworks? Will the release of Qwik v1 (or Marko v6) the final nail in its coffin?
Maybe, maybe not.
To fully appreciate the naunce in this perspective, it's worth remembering that even Qwik, the most vocal of all rebels against hydration, had to sometime present its innovation ("resuming", etc.) as a better approach to performing hydration, not an outright alternative. Here are a couple of quotes from its ealier presentations:
Qwik is a stateless framework (all application states are in DOM in the form of strings). Stateless code is easy to serialize, ship over the wire, and resume. It is also what allows components to be rehydrated independently from each other. [quote from "HTML-first, JavaScript last: the secret to web speed!" published in July 2021; emphasis mine]
Above is a list of reasons why we don't have a truly progressive hydration framework yet. Achieving this is the explicit goal of Qwik. [...] Qwik will be a fully progressive hydration framework that will have all of the above primitives built-in and have a corresponding mental model and DX to go with it.
Qwik is a first of a new breed of frameworks that will take progressive hydration to heart and allow even the largest applications to load instantly on even the slowest clients. [quote from "Why Progressive Hydration is Harder than You Think" published in February 2022; emphasis mine]
And it is not uncommon to see the JS framework authors and those who are closely following the topic discuss "the new breed of frameworks" in terms of how they implement next-gen hydration techniques that are better and more efficient than the "traditional" approach of React, Vue, etc.9
In other words, despite Qwik's bold and tenacious attempt to fix the meaning of "hydration" to that of the "traditional" technique ("replayable", "eager", etc.), it is very much possible that the relevant social body that ultimately determines its usage and thereby meaning will lead to simply strengthening the "broad" definition of "hydration". In fact, the very ambiguity of the term "hydration", which has always been a loose metaphor than a literal-descriptive name for what it points to, is very much not in favor of Qwik's effort to move past it.
With that said, language does evolve and a word's usage does change. So with his clout in the industry and confidence in the true innovation of Qwik, Miško Hevery might really pull off the seemingly impossible: changing people's mind and language.
Footnotes
-
Even in mathematics, meaning of a mathematical expression or notation cannot be completely and universally fixed. The same expression may have different values based on the axioms it rests on, and the same notation or symbol may serve multiple roles in different contexts. ↩
-
A glaring omission in this list is MDN Web Docs, considered by many as the source of information on web technology. Unfortunately, it doesn't seem to have any page on hydration as of this writing. ↩
-
By the way, I don't believe this to be true at least in the case of the web.dev article. A co-author of that article, Jason Miller, is also the creator of Preact, a lightweight React alternative with great React compatibility support. On the other hand, the other co-author, Addy Osmani, hasn't authored any well-known JS framwork but surely knows a thing or two about web performance! ↩
-
The first API to support server-side rendering,
React.renderComponentToString(), appeared in React v0.4. This later becameReact.renderToString()inv0.12andReactDOM.renderToString()inv0.14. ↩ -
React didn't always blow up the server-rendered DOM. As Dan Abromov of the React core team stated in stated in this GitHub Issue comment, "[i]f the markup is identical, React will just attach a component instance to the existing DOM." The comment was written in March 2016, so React v0.15. And this approach relied on expensive checksum verification. ↩
-
However, the term "hydration" was already in use well before the release of React 16 in September 2017. Evan You, the creator of Vue.js, spoke of "hydration" as early as in late 2015 in this GitHub Issue comment. Also, beta releases for Vue.js v2 included "hydration" in their release notes in Summer 2016. ↩
-
As expected, Svelte, lacking VDOM, doesn't "hydrate" the same way React or Vue does. But this actually has costed Svelte in performance. For instance, see this GitHub Issue. ↩
-
This is not to suggest that these terms were first invented in 2019. For exasmple, Sebastian Markbåge of the React core team used the term "partial hydration" as early as in October 2017 in this GitHub Issue comment. ↩
-
See this awesome graphic on the relationship between "the new breed of frameworks" and hydration techniques by Ryan Carniato, the creator of SolidJS and a core team member of eBay's Marko. The graphic, published in earlier this month, offers a great summary of how Astro, Qwik, and Marko (as well as React Server Component) implement progressive, partial, and resumable hydration techniques on what level of granularity. ↩
Love the post! Highly encourage you to publish it!
I would love to add a note. As you say in the latest section we struggled with a name for what do we call it. For a while, we were trying to market it as better hydration, more progressive, etc... But the problem was that it just mentally did not fit what hydration is.
Hydration is attaching listeners. But to attach listeners frameworks have to re-run the app and that is the problem. Qwik can just continue execution where the server left off, so there is no re-runing.
So as our understanding of what Qwik is evolved we realized that talking about it in terms of hydration just did not make any sense. You just can't say that a component is hydrated or not in Qwik. There is no clear definition by which you can declare a component hydrated.
So yes, we did use to say that Qwik does better hydration, but we just came to the conclusion, that it just does not do hydration.
Here is a thought experiment. Define hydration in such a way to only include attaching listeners to DOM which came from SSR/SSG but not from the client-side rendered app. Once you make such a definition, you will see you have effectively excluded Qwik from it.
Example:
My argument is that you can't make a definition that will include classical frameworks hydration and Qwik but exclude other apps which we clearly don't consider hydration, such as client rendered applications.