sidharthachatterjee/querying-2.md Secret

## querying-2.md

      
                Raw

        
              querying-2.md
            
          
    Querying 2.0 and Unlocking Better Abstractions
Querying is one of the most important features of Gatsby today. But we also have several limitations today. This document will talk about out work on Querying 2.0 which will feature several improvements over current functionality and unlock things that aren't possible with Gatsby today.
Querying 1.0
Currently, we have page queries and static queries. Page queries support variables but only when passed in as part of page context and static queries need to static and therefore do not support variables.
Current limitations?
The biggest limitation of our current approach is that we are at best able to create leaky abstractions. Let's look at a canonical example, gatsby-image
Imagine a perfectly encapsulated image component. What parameters would it accept?
<Img url="gatsby-astronaut.png" />

gatsby-image looks more like:
<Img fluid={data.placeholderImage.childImageSharp.fluid} />

This is a leaky abstraction. Why? Because to use Img, a user has to:

make the query themselves
know the shape of the data that Img needs
what a childImageSharp is

This stems from the fact that we don't support queries with variables at arbitrary depths in the tree.
Therefore, there is no way to pass in an arbitrary var as the url for the current image.
But why Sid
Let's look at how we run queries today. We use babel to parse user land code and look for queries (page and static). Every time we visit one:

we extract the query
we run it against our GraphQL schema
we write the results back to JSON files
etc

The biggest issue with parsing user land code and extracting out queries from an AST is the loss of surrounding context. To be able to support arbitrary variables in queries, we need to evaluate code, not parse it.
Eval versus Parse
How do we evaluate all JavaScript code surrounding our queries? We already do! Every time we run ReactDOMServer.renderToString, we're literally running all  (in render functions) user land code.
We need to be able to hook into this process. Suspense (and more importantly the React Fiber rewrite) enables pausing (and persisting slices of) renders and continuing when ready. In our case, we'd pause while we run the query and fetch data.
Querying 2.0
Querying 2.0 will have one big feature that will enable everything else. Say hello to useQuery.
useQuery
useQuery will replace useStaticQuery and (maybe) page queries as well. It will be a React hook that will take a query and variables and throw a Promise (similar to react-cache) that will resolve to the result of the query.
It will be callable anywhere in the component tree and will support build time variables (and eventually run time, read on).
Internally, it will throw a Promise that will suspend the render. A top level context provider (gatsby-cache) will contain a map of query (and variable) hash to result.
This will enable us to build non leaky abstractions like:
<Img url="gatsby-astronaut.png" />

Gotchas
Suspense is currently not available for SSR. This is something the React team is working on. However, looking at this deeper, we see that the nuance dictates that SSR is a different animal for general use cases in React and for Gatsby.
The general use cases in React will likely want to optimise for runtime performance (including streaming results etc) whereas Gatsby doesn't really care for that much since we run SSR at build time.
The solution to this is two-pass rendering.
Two Passes
What we need to do is run a pre pass on the component tree first. By pass, I mean traverse the component tree and execute code in render for all the components. In this pass, we will effectively collect a snapshot of queries and surrounding variables.
The second pass will be regular old ReactDOMServer.renderToString.
Eventually Single Pass
Once Suspense support is out in ReactDOMServer, we should be able to remove the first pass and everything should just work.
First class support for Fragments
First class support for fragments (like what Relay does) is something that I think would be very valuable to have in Gatsby. I haven't thought this through this but I imagine a useFragment like hook that will enable fragments to collate upwards in the tree (in the nearest parent that calls the query perhaps) could be useful.
Caching and better reuse
Since we'll have a cache implementation (gatsby-cache), we should be able to do much better caching and ideally not rerun queries in case of multiple instances (static queries that are identical currently run n times, n being the number of their instances)
Queries on demand
This naturally (and might I add, very elegantly) translates into querying on demand. Queries will only be run on page renders. We should still be able to pre-run some queries if we would like to but I think this will be a nice default to have as we scale.
Support for subscriptions
Querying 2.0 should involve a mechanism to enable subscriptions. I haven't thought this through yet but I imagine a @live directive that will:

mark some queries as runnable on the client
connect to a Gatsby cloud service that serves the GraphQL instance for the site

gatsby-cache should be able to transparently support this in the future.
Questions


Runtime here is at build time?
This will largely be an educational issue that we will have to document well. Some users might confuse when something in the render function (in context of a query) is run. Think of a query that takes a time stamp as a variable. The query will still be run at build time of course while the user might expect it to update when in the client.


What about conditional components that might not render at build time?
Hooks cannot be used inside conditionals inside a render function. The only possibility here is whole components not being rendered at build time and being rendered at runtime. A common example of this is a mobile first SSRed website running on desktop. We should be able to lint for this but I'm not sure and this warrants further discussion.


How do you query this post hydration?
Read about subscriptions above.


With incremental builds, we won't run SSR for everything on every build. How do we track query dependencies in that case?
This will be interesting to solve and will need some discussion. With incremental builds, we will need to track dependencies and arrive at an entry point that will have to be SSRed again. Since that entry point will be a parent of the component that includes the query, this should be fine. Also, to begin with, incremental builds will likely track code changes as whole rebuilds so this shouldn't be a problem.


Rough todo

 Complete proof of concept with react-ssr-prepass
 Implement new querying engine
 Deprecate page and static queries
 Write code mods for page and static queries
 Write documentation
 Release