plan.md

Spec: Datasette extras explorer

A single-page HTML+JavaScript tool for interactively exploring the ?_extra= options available on any Datasette JSON endpoint. No build step, no server component, no dependencies that require bundling - one .html file that can be opened from disk or served from anywhere.

How it works

1. The user pastes a URL to a Datasette .json endpoint into an input field - a table page, a row page, or a query page, with or without existing query string parameters or the .json extension. Examples:
   • http://127.0.0.1:7777/fixtures/facetable.json
   • http://127.0.0.1:7777/fixtures/facetable
   • http://127.0.0.1:7777/fixtures/simple_primary_key/1.json
   • http://127.0.0.1:7777/fixtures/-/query.json?sql=select+:greeting+as+msg&greeting=hello
2. The tool fetches that URL with _extra=extras appended and reads the extras key from the response: a list of {name, description, toggle_url, selected} objects. This is the discovery step - the list differs by page type (TABLE / ROW / QUERY scope) and the tool makes no assumptions about which names exist.
3. The discovered extras render as a checkbox list. Toggling checkboxes re-fetches the endpoint with the selected names and displays the resulting JSON - minus the extras key, which exists to drive the UI and would otherwise drown the interesting output.

UI layout

+--------------------------------------------------------------+
| Datasette extras explorer                                    |
| [ URL input................................. ] [ Explore ]   |
+----------------------+---------------------------------------+
| Extras               | Response                              |
|                      |                                       |
| [x] columns          | GET /fixtures/facetable.json?_extra=  |
|     Column names     |     columns,count        [copy URL]   |
|     returned by...   | 200 - 1.2 KB - 45ms                   |
| [x] count            |                                       |
|     Total count of   | {                                     |
|     rows matching... |   "ok": true,                         |
| [ ] count_sql        |   "rows": [...],                      |
|     SQL query used   |   "columns": ["pk", "created", ...],  |
|     to calculate...  |   "count": 15                         |
| [ ] facet_results    | }                                     |
|     ...              |                                       |
+----------------------+---------------------------------------+

• Left panel: one checkbox per discovered extra, with its name in monospace and its description underneath in smaller muted text. Sorted alphabetically. The extras entry itself is not shown (it is always requested, never displayed).
• Right panel: the constructed request line with a copy button, response status / size / elapsed time, then the response JSON pretty-printed with 2-space indentation. Keys contributed by currently-selected extras are visually marked (e.g. a subtle background tint on the line containing the top-level key) so it is obvious what each toggle added; this is computed by comparing top-level keys against the selected extra names, not by diffing responses.

Request construction rules

• Parse the user's URL with the URL API. Preserve every existing query parameter (sql=, named parameters, _size=, _shape= etc.) on every request - the tool only manages the _extra parameter.
• If the pasted URL already contains _extra= values (comma-separated or repeated), those names start out checked.
• Every request includes extras plus the checked names, as a single comma-separated parameter: _extra=extras,columns,count. Requesting extras every time keeps the checkbox list and selected states current and costs little.
• In-flight requests are aborted (AbortController) when a new toggle happens; rapid toggling coalesces to one request via a ~200ms debounce.
• Explorer state (endpoint URL + checked names) is mirrored into the page's location.hash so a session can be bookmarked or shared; on load, a hash takes precedence over the empty input.

Display rules

• The extras key is deleted from the response object before rendering. Everything else renders exactly as returned, pretty-printed.
• JSON is rendered as text in a <pre> with lightweight syntax highlighting (strings/numbers/keys/booleans via a small regex highlighter - no library). Long rows arrays are not truncated; readability of large responses is the user's responsibility via _size=.
• A "raw" toggle shows the unmodified response including the extras block, for the skeptics.

Error handling

• Non-200 responses: show the status code and the response body (Datasette returns JSON error objects with ok: false and error keys - render them in the same JSON view, tinted red).
• Network/CORS failures: show a hint that the target Datasette must either be same-origin or started with --cors (the discovery fetch failing entirely is almost always CORS).
• A checked extra whose key is absent from the response (scope mismatch or an older Datasette) renders its checkbox with a warning marker and tooltip "requested but not returned" rather than erroring.
• A URL whose discovery response has no extras key is reported as "this endpoint does not support ?_extra= - is it a Datasette 1.0a33+ JSON endpoint?".

Implementation requirements

• One self-contained .html file: inline <style> and <script>, vanilla ES2020+, no frameworks, no build step, no external requests other than to the target Datasette.
• fetch() with credentials: "same-origin" so cookie-authenticated Datasette instances work when the tool is served from the same origin.
• Works when opened via file:// against a --cors instance.
• Keyboard accessible: checkboxes are real <input type="checkbox"> elements with <label>s; the URL form submits on Enter.
• No state outside the DOM and the URL hash.

Non-goals

• No authentication UI (tokens, sign-in) - rely on cookies/same-origin.
• No editing of non-_extra parameters - paste a different URL instead.
• No awareness of which extras are "expensive"; the extras listing does not expose that flag. (Possible future Datasette enhancement: include expensive in the ExtrasExtra output, at which point the UI can badge those entries.)
• No handling of ?_shape= variants that change the top-level structure (array, arrayfirst, object) - extras only appear in the default object shape, so the tool strips _shape from pasted URLs and says so.

Acceptance criteria

Tested against the fixtures instance at http://127.0.0.1:7777:

1. Pasting /fixtures/facetable.json lists the TABLE-scope extras (count, facet_results, next_url, ...); checking count shows "count": 15 in the response panel; no extras key is visible.
2. Pasting /fixtures/simple_primary_key/1.json lists the 14 ROW-scope extras; checking foreign_key_tables shows the five inbound references.
3. Pasting the :greeting query URL above lists the 10 QUERY-scope extras; checking query shows params: {"greeting": "hello"}.
4. Checking then unchecking an extra returns the display to its prior state, and the copy-URL button reproduces the current view when opened in a new tab - the copied URL is exactly the URL the tool requested, including the extras name.
5. Requesting an HTML-only extra like filters by hand-editing the URL produces a 200 whose response simply lacks the key, surfaced via the "requested but not returned" marker rather than an error.