Compiling clojurescript + figwheel without boot nor leiningen. Using leiningen to manage dependencies. Print file size stats (raw, gzip, brotli) for production builds

Makefile

CLJ_NREPL_PORT:=22340
FIGWHEEL_PORT:=22345

CLJS_JAR_VERSION:=1.7.48
CLJS_JAR_URL:=https://github.com/clojure/clojurescript/releases/download/r$(CLJS_JAR_VERSION)/cljs.jar

.PHONY: def_target

def_target : null

# http://blog.jgc.org/2015/04/the-one-line-you-should-add-to-every.html
# Allows:
# make print-SOURCE_FILEE
print-%: ; @echo $*=$($*)

#  We use `lein run` to run the cljs compiler scripts since we need the
#  classpath properly set:
# We could also do a lower level and use the standalone jar of cljs to test
# different versions quickly:
cljs-c-jar-dev :
	java -cp cljs-$(CLJS_JAR_VERSION).jar:scripts:$$(lein with-profile +dev-cljs classpath) \
	  clojure.main scripts/cljs-build.clj dev your-entry.core

cljs-c-dev :
	lein with-profile dev-cljs run -m clojure.main scripts/fw+cljs.clj dev

cljs-c-production-login :
	lein with-profile prod-cljs run -m clojure.main scripts/fw+cljs.clj production login

cljs-c-production-app :
	lein with-profile prod-cljs run -m clojure.main scripts/fw+cljs.clj production app

figwheel :
	lein with-profile dev-cljs repl :start :port $(FIGWHEEL_PORT) 

dl-cljs :
	curl -L $(CLJS_JAR_URL) > cljs-$(CLJS_JAR_VERSION).jar

all-deps-dump :
	echo ";; vim: ft=clojure" | tee deps.dev-clj deps.dev-cljs deps.prod-clj deps.prod-cljs > /dev/null
	lein with-profile dev-clj deps :tree >> deps.dev-clj 2>&1
	lein with-profile dev-cljs deps :tree >> deps.dev-cljs 2>&1
	lein with-profile prod-clj deps :tree >> deps.prod-clj 2>&1
	lein with-profile prod-cljs deps :tree >> deps.prod-cljs 2>&1

OVERVIEW.md

Synopsis:

Compiling clojurescript + figwheel without boot nor leiningen. Using leiningen to manage dependencies (with profiles).

Note:

The below filesnames have || in them. That's because gist doesn't allow subdirectories. But the script are all in a ./scripts/ subdir.

Usage

You can have contious (watching) builds by running make in multiple terminals:

make cljs-c-dev
make cljs-c-production
etc..

You can run figwheel with the following steps:

1. make figwheel
2. Connect to the normal NREPL from your editor (Cursive, Emacs etc)
3. Enter (load-file "scripts/fw+cljs.clj")

Note: You now have access to the clojure REPL on your terminal and a CLJS REPL on top of it with fighweel. This allows you to do some cools stuff, like pulling in new clojurescript(!) dependencies without having to restart your fighweel REPL:

Make sure you have vinyasa setup: https://github.com/zcaudate/vinyasa

Then you can enter (./pull 'your-cljs-lib) in your NREPL (terminal, not your fighweel repl) and use it right away in your cljs code (since the java classpaths has been adjusted and your depencies were dynamically downloaded).

project.clj

(defproject xyz "0.1.0-SNAPSHOT"
  :description "TODO description"

  :dependencies [;; CLJ or common
                 [org.clojure/clojurescript "1.7.122"]
                 [cljsjs/react "0.13.3-1"]
                 [figwheel-sidecar "0.5.3-2" :scope "provided"]]

  ;; Some speedup
  ;; https://github.com/technomancy/leiningen/wiki/Faster
  :jvm-opts ["-Xverify:none"]

  ;; We dont have "one" source path when doing CLJS+CLJ in one project, we always
  ;; specify these in the profiles
  ;;:source-paths ["src" "target/classes"]
  ;; TODO: Temporary workaround for Cursive can't start a REPL with profiles so
  ;; we need to specify everything here:
  ;; The  ^:replace meta keys in the profiles will overwrite this and make it a non-issue.
  ;; Also see Cursive issue 1036
  ;; NOTE: This also gets picked up by Cursive/IntelliJ every time it starts:
  ;; So this is ONLY really set for Cursive ATM:
  :source-paths ["src/clj" "src/cljs" "src/cljc"
                 "env/dev/cljs" "env/dev/clj"
                 "env/prod/cljs" "env/prod/clj"]

  ;; All generated files will be placed in :target-path. In order to avoid
  ;; cross-profile contamination (for instance, uberjar classes interfering
  ;; with development), it's recommended to include %s in in your custom
  ;; :target-path, which will splice in names of the currently active profiles.
  :target-path "target/%s/"

  ;; Directory in which to place AOT-compiled files. Including %s will
  ;; splice the :target-path into this value.
  :compile-path "%s/class-files"

  :clean-targets ^{:protect false} ["resources/public/js/" "target"]

  ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
  :profiles {
             ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
             ;; Dependencies for development
             ;; This profile gets merged into other profiles (see "leiningen composite profiles")
             :dev-cljs-deps
             {
              :dependencies []
              }
             ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
             ;; Env for clojurescript dev environ
             :dev-cljs
             [:dev-cljs-deps
              {:source-paths ^:replace ["src/cljs" "src/cljc" "env/dev/cljs"]
               :env {:is-dev true}}]
             ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
             ;; Env for clojurescript prod environ
             :prod-cljs
             [:dev-cljs-deps
              {:source-paths ^:replace ["src/cljs" "src/cljc" "env/prod/cljs"]
               :env {:is-dev false}}]
             };; profiles
);; defproject

scripts||fw+cljs.clj

(ns my.myfw
  "
  We make this a real namespace so we can develop this file interactively with Cursive.

  Run me so:
  lein with-profile +dev-cljs run -m clojure.main scripts/fw+cljs.clj dev
  lein with-profile +dev-cljs run -m clojure.main scripts/fw+cljs.clj production
  or for figwheel:
  1. Start a normal nrepl with lein:
      lein with-profile +dev-cljs repl :start :port YOUR_PORT
  2. Connect to it from Cursive, Emacs etc...
  3. Enter: (load-file \"scripts/fw+cljs\")

  Assumes you have a local server running which maps requests to:
  http://localhost:10083/dev/compress/resources/public/js/

  For instance for nginx:
    location /dev/compress/resources/public/js/ {
      allow 127.0.0.1;
      alias ../../resources/public/js/;
    }
  Should understand gzip and brotli compression.

  This build script outputs all {raw,gzip,brotli} sizes to the root directory in
  size-hist.your-out-file.js
  "
  (:require [cljs.build.api :as api]
            [clojure.pprint :as pprint]
            [clojure.java.io :as io])
  (:import (java.io File FileFilter)
           (java.util Date Scanner)
           (java.net URL)))
#_(clojure.java.classpath/classpath)

(def build-id
  (keyword (first *command-line-args*)))

(def module-id
  (second *command-line-args*))

(println "Got build id: " build-id)
(println "Current ns: " (str *ns*))

(defn err [msg]
  (println msg)
  (System/exit 1))

(defn ls-match
  "Given the directory handle d, lists all files matching the given
   regex. Returns the Java File instances.
   Example:
   (ls-match (File. \"./dir\") #\"(?i).drl$\")"
  ([^File d]
   (ls-match d #"^[~_.]"))
  ([^File d regex]
   (.listFiles d
               (reify
                 FileFilter
                 (accept [this f]
                   (and
                     (.isFile f)
                     (boolean (re-find regex (.getName f)))))))))

(def externs
  (mapv
    #(.getCanonicalPath ^File %)
    (ls-match (io/file (io/resource "externs")) #"(?i)\.js$")))

(def src-paths-common ["src/cljs" "src/cljc"])
(def src-paths-dev (conj src-paths-common "env/dev/cljs"))
(def src-paths-prod (conj src-paths-common "env/prod/cljs"))

(defn compressed-size [^String url ^String accept-encoding]
  (future
    (.length
      (.next
        (.useDelimiter
          (Scanner.
            (.getInputStream
              (doto
                (.openConnection
                  (URL. url))
                (.setRequestProperty "Accept-Encoding" accept-encoding))))
          "\\Z")))))

(defn log-size-history [^String output-to ^File out-file]
  (future
    (let [url (str "http://localhost:10083/dev/compress/" output-to)
          brotli-size (compressed-size url "br")
          gzip-size (compressed-size url "gzip")
          size (.length out-file)
          stats-str (prn-str {:date (Date.)
                              :size size
                              :gzip-size @gzip-size
                              :brotli-size @brotli-size})]
      (println "DONE: " output-to)
      (println stats-str)
      (spit (str "size-hist." (.getName out-file))
            stats-str
            :append true))))

(defn watch-fn-stats [output-to]
  (fn []
    (let [f (io/file output-to)]
      (log-size-history output-to f))))

(defn output-dir [build-id]
  (str "resources/public/js/" (name build-id) "/compiled/out"))

(defn output-to [build-id module]
  (str "resources/public/js/" (name build-id) "/compiled/" (name module) ".js"))

(defn source-map [out-file]
  (str out-file ".map"))

(defn source-paths [cfg]
  (case cfg
    :dev src-paths-dev
    :production src-paths-prod))

(defn figwheel-cfg [cfg]
  (when (= :dev cfg)
    {:figwheel {:websocket-host "localhost"
                :on-jsload "srs-c.dev/on-js-reload"}}))

(defn compiler-cfg [cfg module]
  (let [out (output-to cfg module)
        prod-false (get {:production false} cfg true)
        prod-true (not prod-false)]
    (->>
      (merge
        {:output-to out
         :output-dir (output-dir cfg)
         :compiler-stats true
         :pretty-print prod-false
         :elide-asserts prod-true
         :source-map (get {:production (source-map out)
                           :dev true} cfg true)
         :closure-defines {"goog.DEBUG" prod-false}
         :externs externs}

        (case cfg
          :dev
          {:main "srs-c.dev"
           ;; :asset-path is a relative URL path not a file system path.
           ;; This is prepended to the goog.require loads. So it's part of a URL:
           :asset-path "/js/dev/compiled/out" ;; See :output-dir
           :anon-fn-naming-policy :mapped
           :optimizations :none
           :source-map-timestamp true}
          ;;;;;;;;;;;;;;;;;;;;;;;;;;
          :production
          {:main (str "srs-c.modules." module)
           :optimizations :advanced
           ;; False is almost no difference in brotli size. Performance?
           :optimize-constants true
           :pseudo-names false
           :watch-fn (watch-fn-stats out)}))
      (into (sorted-map)))))
#_(compiler-cfg :dev "src-c.dev")
#_(compiler-cfg :production "login")

(defn build-config
  [cfg module]
  (merge
    {:source-paths (source-paths cfg)}
    (figwheel-cfg cfg)
    {:compiler (compiler-cfg cfg module)}))
#_(build-config :dev "srs-c.dev")
#_(build-config :production "login")

(defn start-compiler [build]
  (pprint/pprint build)
  (api/watch
    (apply api/inputs (:source-paths build))
    (:compiler build)))

(defn build-cljs [build-id module]
  (if-let [build (build-config build-id module)]
    (start-compiler build)
    (err "No such build id")))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn run-figwheel []
  ;; We may not have figwheel in our classpath if we do cljs compilation:
  (let [fw-api 'figwheel-sidecar.repl-api
        _ (require fw-api)
        start-fw (ns-resolve fw-api 'start-figwheel!)
        cljs-repl (ns-resolve fw-api 'cljs-repl)]
    (start-fw
      {:build-ids ["dev"] ;; vector of build ids to start autobuilding
       :all-builds [(assoc (build-config :dev nil) :id "dev")]
       :figwheel-options
       {:websocket-host "localhost"
        ;; The port of the fighweel server, the browser connects to this:
        :server-port 22341
        ;; The bind IP of the figwheel server
        :server-ip "127.0.0.1"
        :load-warninged-code true
        ;;:nrepl-port 22345
        ;;:nrepl-middleware ["cemerick.piggieback/wrap-cljs-repl"]
        }})
    (cljs-repl)))

(if build-id
  (build-cljs build-id module-id)
  (run-figwheel))

June 11: Updated with new build script which combines cljs builds and figwheel.

1. Now also prints out the file size in advanced build.
2. Full clojure for creating your build maps instead of (inflexible) edn.

June 14: Now prints (and logs) files sizes (raw, gzip, brotli) for production builds.

Probably want to remove watch-fn-stats from the watch key if you don't have a server like that.

Also see:
https://gist.github.com/rauhs/d5bb47c239b58cbf54e1d3d1de221031