Adhoc CLJS dependency generation

depgen.md

Introduction

A conversation started at the Heart of Clojure conference in Belgium on Friday August 2nd 2019.

The group represented project owners from Maria.Cloud, Next.Journal, Klipse and Replete-Web. The projects make significant use of self-hosted CLJS.

This is a proposal to have a service that generates and caches JS files for a specific CLJS dependency (name & version).

Background

The ClojureScript compiler generates JS files for each of an apps stated CLJS dependencies. The runtime environment then loads each of the needed JS files to satisfy the dependency at runtime.

Problem statement

When a dependency is added at runtime

• it adds significant time delays on the user experience
• we don't yet have a shared, generic solution to calculating / generating the correct dependency tree per dep

Suggestions

API / Cache

1. Call a hosted API to generate a lib given an argument such as

{:deps {mvngrp/mvnartefact {:mvn/version "0.6.2"}}}

or

{:deps {gitname {:git/url "https://github.com/org-name/repo-name.git"
                 :sha     "ad5bcac0c2d771f09f69de4edab183b0c2fe437b"}}

2. The API would return

• a URL to the location of the generated files
• a list of the JS files and maybe others (eg source maps)
• meta data such as a SHA / MD5 for each of the generated files

API / Cache - Implementation

• Normalise the EDN
• Look up the normalised dep (or a hash of it) in the cache
  • when cache-hit: return the link to previously generated data
  • when cache-miss: run CLJS to produce the deps, add the data to the cache and return the data

Web workers

Make the call via a web worker rather than the mainline code.

This could improve the perceived performance in the UX.

END

This is my recollection.

Please feel free to edit / comment.

@mfikes Thx, I used to take a similar approach.. now we AOT compile macro namespaces at the same time as the rest of the sources. This is performed by the shadow-cljs :bootstrap target (here, roughly). After sources are resolved for the given entries, a separate pass finds all their dependent macros, before the compile step.

Gents! I modified mhuebert demos a little bit and am now generating dynamic bundles via shadow cljs that I serve via http.
https://github.com/pink-gorilla/kernel-cljs-shadowdeps

The self hosted clojurescript is this project: https://github.com/pink-gorilla/kernel-cljs-shadow