Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Studying a Leiningen Plugin with REBL

Studying a Leiningen Plugin with REBL

Introduction

Starting from scratch, how might one develop a mental model for an existing Leiningen plugin?

For concreteness, consider MrAnderson:

MrAnderson is a dependency inlining and shadowing tool. It isolates the project's dependencies so they can not interfere with other libraries' dependencies.

After obtaining the source code, let us:

  • identify an entry point to begin examination,
  • instrument some code to record some appropriate values,
  • trigger the code, and then
  • examine the results

Once this captured data is examined appropriately, perhaps one will find that some parts of the source code will make more sense than otherwise. If need be, an additional function can be identified, instrumented in a similar manner, executed, and the results examined. This process can be repeated to further refine our mental model.

For simple enough data, direct examination of the captured data in the REPL might be sufficient. Here an attempt will be made to use REBL instead.

Obtaining Source Code

In the case of MrAnderson, this can be done by:

git clone https://github.com/benedekfazekas/mranderson

An abbreviated directory layout of the fetched content is:

├── build.sh
├── java-src
│   └── mranderson
│       └── util
│           ├── JjMainProcessor.java
│           ├── JjPackageRemapper.java
│           └── JjWildcard.java
├── project.clj
├── README.md
├── src
│   ├── leiningen
│   │   └── inline_deps.clj
│   └── mranderson
│       ├── core.clj
│       ├── dependency
│       │   ├── resolver.clj
│       │   └── tree.clj
│       ├── move.clj
│       ├── plugin.clj
│       ├── profiles.clj
│       └── util.clj

Things one might note:

  • README.md - perhaps this has some useful docs
  • project.clj - may be this is a Leiningen project
  • src/leiningen and src/mranderson subdirectories - some contained file might have a relevant entry point

Identifying an Entry Point

According to the Leiningen Plugins section of PLUGINS.md:

Leiningen tasks are simply functions named $TASK in a leiningen.$TASK namespace. So writing a Leiningen plugin is just a matter of creating a project that contains such a function

In the current case, src/leiningen/inline_deps.clj contains:

(ns leiningen.inline-deps
  (:require [clojure.edn :as edn]
            [me.raynes.fs :as fs]
            [mranderson.core :as c]
            [mranderson.util :as u])
  (:import [java.util UUID]))

and:

(defn inline-deps
  "Inline and shadow dependencies so they can not interfere with other libraries' dependencies.

  Available options:

  :project-prefix           string    Project prefix to use when shadowing
  :skip-javaclass-repackage boolean   If true Jar Jar Links won't be used to repackage java classes
  :prefix-exclusions        list      List of prefixes that should not be processed in imports
  :unresolved-tree          boolean   Enforces unresolved tree mode"
  [{:keys [repositories dependencies source-paths target-path] :as project} & args]
  (c/copy-source-files source-paths target-path)
  (let [{:keys [pprefix] :as ctx} (lein-project->ctx project args)
        paths                     (initial-paths target-path pprefix)]
    (c/mranderson repositories dependencies ctx paths)))

That seems to fit the pattern in the aforementioned quote, where $TASK is inline-deps.

Instrument Some Code

Examining inline-deps, it appears that after calling mranderson.core/copy-source-files and some setup, mranderson.core/mranderson is invoked. An abbreviated definition is:

(defn mranderson
  ",,,some docs elided,,,"

  [repositories dependencies {:keys [skip-repackage-java-classes unresolved-tree pname pversion overrides] :as ctx} paths]
  (let [source-dependencies         (filter u/source-dep? dependencies)
        resolved-deps-tree          (dr/resolve-source-deps repositories source-dependencies)
        overrides                   (or (and unresolved-tree overrides) {})
        unresolved-deps-tree        (dr/expand-dep-hierarchy repositories resolved-deps-tree overrides)]
    (u/info "retrieve dependencies and munge clojure source files")
    (if unresolved-tree
      (mranderson-unresolved-deps! unresolved-deps-tree paths ctx)
      (mranderson-resolved-deps! resolved-deps-tree unresolved-deps-tree paths ctx))
    (when-not (or skip-repackage-java-classes (empty? (u/class-files)))
      (class-deps-jar!)
      (u/apply-jarjar! pname pversion)
      (replace-class-deps!))))

The function arguments:

  • repositories
  • dependencies
  • ctx
  • paths

and the bindings of the let form:

  • resolved-deps-tree
  • overrides
  • unresolved-deps-tree

look interesting.

An attempt will be made to capture them for examination by adding the following code immediately after the let bindings:

    (tap> {:repositories repositories
           :dependencies dependencies
           :ctx ctx
           :paths paths
           :resolved-deps-tree resolved-deps-tree
           :overrides overrides
           :unresolved-deps-tree unresolved-deps-tree})

Trigger Code

REBL must be running in the same process as the code that generates the data to capture. To prepare for this, mranderson's project.clj is modified as:

(defproject thomasa/mranderson "0.5.2-SNAPSHOT"
  ,,,
  ;; XXX: (1) rebl crashes without this commented out
  ;;:eval-in :leiningen
  ,,,
  ;; XXX: (2) rebl jar
  :resource-paths ["REBL.jar"]
  ;; XXX: (3) socket repl
  :jvm-opts ["-Dclojure.server.repl={:port 8888 :accept clojure.core.server/repl}"]
  :profiles {:dev {:dependencies [;; XXX: (4) rebl deps
                                  [org.clojure/clojure "1.10.1"] ;; replaced 1.8
                                  [cljfmt "0.6.4"] ;; somehow not made available
                                  [org.clojure/core.async "0.4.490"]
                                  [org.clojure/data.csv "0.1.4"]
                                  [org.clojure/data.json "0.2.3"]
                                  [org.yaml/snakeyaml "1.23"]
                                  [org.openjfx/javafx-fxml "11.0.1"]
                                  [org.openjfx/javafx-controls "11.0.1"]
                                  [org.openjfx/javafx-graphics "11.0.1"]
                                  [org.openjfx/javafx-media "11.0.1"]
                                  [org.openjfx/javafx-swing "11.0.1"]
                                  [org.openjfx/javafx-base "11.0.1"]
                                  [org.openjfx/javafx-web "11.0.1"]
                                  ;; XXX: was already here
                                  [leiningen-core "2.9.1"]]}})

The modifications include:

  • (1) commenting out the :eval-in :leiningen line -- it's not clear why this is necessary, but it appears to be
  • (2) adding the REBL.jar to :resource-paths
  • (3) enabling a socket repl via :jvm-opts (optional)
  • (4) adding REBL-required dependencies to :profiles :dev :dependencies for REBL use at the repl

Note that an appropriate REBL.jar needs to be obtained, named appropriately and placed in an appropriate location. In the above setup, the jar is named REBL.jar and is placed in the project root. (This set of instructions was tested with version 0.9.218.)

The above setup is for using REBL with Java 11. (This set of instructions was tested with AdoptOpenJDK 11.)

Launch the REPL from the project directory:

lein repl

Then either at the terminal or via the socket REPL, load REBL and start its GUI:

user=> (require '[cognitect.rebl :as cr])
nil
user=> (cr/ui) ; GUI should show up, possibly after a brief pause
nil

rebl-gui To trigger the entry point, inline-deps, it must be passed an appropriate project. This can be prepared via leiningen.core.project/read:

user=> (require '[leiningen.core.project :as lcp])
nil
user=> (def p (lcp/read))
#'user/p

Finally, the entry point can be triggered:

user=> (require '[leiningen.inline-deps :as li])
nil
user=> (li/inline-deps p) ; probably takes a while, but don't need to wait for it to complete
()

Examine Results in REBL

In REBL's GUI, choose the tap tab, located in the left half of the window (to the right of the browse and out tabs).

rebl-tap-tab

The first row should not be blank, displaying something representing the captured data.

rebl-press-browse-button

Click the Browse button to make the data browsable -- this may cause the browse tab to become active as well.

rebl-browse-pane

Now use REBL to examine the captured data.

Use the Nav forward command, bound to the right arrow button near the bottom middle of the window to "drill down" into the selected data.

rebl-nav-forward

Once the button has been clicked, the bottom right half of the window should contain rows corresponding to:

  • repositories
  • dependencies
  • ctx
  • paths
  • resolved-deps-tree
  • overrides
  • unresolved-deps-tree

rebl-nav-forward-pressed

Selecting appropriately and using the Nav forward command, we take a brief look at resolved-deps-tree and its associated value:

rebl-choose-resolved-deps-tree

rebl-chose-resolved-deps-tree

Continue exploring other values as desired :)

Conclusion

Having explored the captured value(s), it may be that more of the source code will make sense than before. Instrumenting and capturing can be continued to help develop one's mental model.

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment