Starting from scratch, how might one develop a mental model for an existing Leiningen plugin?
For concreteness, consider MrAnderson:
MrAnderson is a dependency inlining and shadowing tool. It isolates the project's dependencies so they can not interfere with other libraries' dependencies.
After obtaining the source code, let us:
- identify an entry point to begin examination,
- instrument some code to record some appropriate values,
- trigger the code, and then
- examine the results
Once this captured data is examined appropriately, perhaps one will find that some parts of the source code will make more sense than otherwise. If need be, an additional function can be identified, instrumented in a similar manner, executed, and the results examined. This process can be repeated to further refine our mental model.
For simple enough data, direct examination of the captured data in the REPL might be sufficient. Here an attempt will be made to use REBL instead.
In the case of MrAnderson, this can be done by:
git clone https://github.com/benedekfazekas/mranderson
An abbreviated directory layout of the fetched content is:
├── build.sh
├── java-src
│ └── mranderson
│ └── util
│ ├── JjMainProcessor.java
│ ├── JjPackageRemapper.java
│ └── JjWildcard.java
├── project.clj
├── README.md
├── src
│ ├── leiningen
│ │ └── inline_deps.clj
│ └── mranderson
│ ├── core.clj
│ ├── dependency
│ │ ├── resolver.clj
│ │ └── tree.clj
│ ├── move.clj
│ ├── plugin.clj
│ ├── profiles.clj
│ └── util.clj
Things one might note:
README.md- perhaps this has some useful docsproject.clj- may be this is a Leiningen projectsrc/leiningenandsrc/mrandersonsubdirectories - some contained file might have a relevant entry point
According to the Leiningen Plugins section of PLUGINS.md:
Leiningen tasks are simply functions named $TASK in a leiningen.$TASK namespace. So writing a Leiningen plugin is just a matter of creating a project that contains such a function
In the current case, src/leiningen/inline_deps.clj contains:
(ns leiningen.inline-deps
(:require [clojure.edn :as edn]
[me.raynes.fs :as fs]
[mranderson.core :as c]
[mranderson.util :as u])
(:import [java.util UUID]))and:
(defn inline-deps
"Inline and shadow dependencies so they can not interfere with other libraries' dependencies.
Available options:
:project-prefix string Project prefix to use when shadowing
:skip-javaclass-repackage boolean If true Jar Jar Links won't be used to repackage java classes
:prefix-exclusions list List of prefixes that should not be processed in imports
:unresolved-tree boolean Enforces unresolved tree mode"
[{:keys [repositories dependencies source-paths target-path] :as project} & args]
(c/copy-source-files source-paths target-path)
(let [{:keys [pprefix] :as ctx} (lein-project->ctx project args)
paths (initial-paths target-path pprefix)]
(c/mranderson repositories dependencies ctx paths)))That seems to fit the pattern in the aforementioned quote, where $TASK is inline-deps.
Examining inline-deps, it appears that after calling mranderson.core/copy-source-files and some setup, mranderson.core/mranderson is invoked. An abbreviated definition is:
(defn mranderson
",,,some docs elided,,,"
[repositories dependencies {:keys [skip-repackage-java-classes unresolved-tree pname pversion overrides] :as ctx} paths]
(let [source-dependencies (filter u/source-dep? dependencies)
resolved-deps-tree (dr/resolve-source-deps repositories source-dependencies)
overrides (or (and unresolved-tree overrides) {})
unresolved-deps-tree (dr/expand-dep-hierarchy repositories resolved-deps-tree overrides)]
(u/info "retrieve dependencies and munge clojure source files")
(if unresolved-tree
(mranderson-unresolved-deps! unresolved-deps-tree paths ctx)
(mranderson-resolved-deps! resolved-deps-tree unresolved-deps-tree paths ctx))
(when-not (or skip-repackage-java-classes (empty? (u/class-files)))
(class-deps-jar!)
(u/apply-jarjar! pname pversion)
(replace-class-deps!))))The function arguments:
repositoriesdependenciesctxpaths
and the bindings of the let form:
resolved-deps-treeoverridesunresolved-deps-tree
look interesting.
An attempt will be made to capture them for examination by adding the following code immediately after the let bindings:
(tap> {:repositories repositories
:dependencies dependencies
:ctx ctx
:paths paths
:resolved-deps-tree resolved-deps-tree
:overrides overrides
:unresolved-deps-tree unresolved-deps-tree})REBL must be running in the same process as the code that generates the data to capture. To prepare for this, mranderson's project.clj is modified as:
(defproject thomasa/mranderson "0.5.2-SNAPSHOT"
,,,
;; XXX: (1) rebl crashes without this commented out
;;:eval-in :leiningen
,,,
;; XXX: (2) rebl jar
:resource-paths ["REBL.jar"]
;; XXX: (3) socket repl
:jvm-opts ["-Dclojure.server.repl={:port 8888 :accept clojure.core.server/repl}"]
:profiles {:dev {:dependencies [;; XXX: (4) rebl deps
[org.clojure/clojure "1.10.1"] ;; replaced 1.8
[cljfmt "0.6.4"] ;; somehow not made available
[org.clojure/core.async "0.4.490"]
[org.clojure/data.csv "0.1.4"]
[org.clojure/data.json "0.2.3"]
[org.yaml/snakeyaml "1.23"]
[org.openjfx/javafx-fxml "11.0.1"]
[org.openjfx/javafx-controls "11.0.1"]
[org.openjfx/javafx-graphics "11.0.1"]
[org.openjfx/javafx-media "11.0.1"]
[org.openjfx/javafx-swing "11.0.1"]
[org.openjfx/javafx-base "11.0.1"]
[org.openjfx/javafx-web "11.0.1"]
;; XXX: was already here
[leiningen-core "2.9.1"]]}})The modifications include:
- (1) commenting out the
:eval-in :leiningenline -- it's not clear why this is necessary, but it appears to be - (2) adding the
REBL.jarto:resource-paths - (3) enabling a socket repl via
:jvm-opts(optional) - (4) adding REBL-required dependencies to
:profiles :dev :dependenciesfor REBL use at the repl
Note that an appropriate REBL.jar needs to be obtained, named appropriately and placed in an appropriate location. In the above setup, the jar is named REBL.jar and is placed in the project root. (This set of instructions was tested with version 0.9.218.)
The above setup is for using REBL with Java 11. (This set of instructions was tested with AdoptOpenJDK 11.)
Launch the REPL from the project directory:
lein repl
Then either at the terminal or via the socket REPL, load REBL and start its GUI:
user=> (require '[cognitect.rebl :as cr])
nil
user=> (cr/ui) ; GUI should show up, possibly after a brief pause
nil
To trigger the entry point, inline-deps, it must be passed an appropriate project. This can be prepared via leiningen.core.project/read:
user=> (require '[leiningen.core.project :as lcp])
nil
user=> (def p (lcp/read))
#'user/p
Finally, the entry point can be triggered:
user=> (require '[leiningen.inline-deps :as li])
nil
user=> (li/inline-deps p) ; probably takes a while, but don't need to wait for it to complete
()
In REBL's GUI, choose the tap tab, located in the left half of the window (to the right of the browse and out tabs).
The first row should not be blank, displaying something representing the captured data.
Click the Browse button to make the data browsable -- this may cause the browse tab to become active as well.
Now use REBL to examine the captured data.
Use the Nav forward command, bound to the right arrow button near the bottom middle of the window to "drill down" into the selected data.
Once the button has been clicked, the bottom right half of the window should contain rows corresponding to:
repositoriesdependenciesctxpathsresolved-deps-treeoverridesunresolved-deps-tree
Selecting appropriately and using the Nav forward command, we take a brief look at resolved-deps-tree and its associated value:
Continue exploring other values as desired :)
Having explored the captured value(s), it may be that more of the source code will make sense than before. Instrumenting and capturing can be continued to help develop one's mental model.