Boilerplate for running Clojure as a shebang script

00_script.clj

#!/bin/sh
#_(

   #_DEPS is same format as deps.edn. Multiline is okay.
   DEPS='
   {:deps {clj-time {:mvn/version "0.14.2"}}}
   '

   #_You can put other options here
   OPTS='
   -J-Xms256m -J-Xmx256m -J-client
   '

exec clojure $OPTS -Sdeps "$DEPS" "$0" "$@"

)

(println "Hello!")

(require '[clj-time.core :as t])

(prn (str (t/now)))

(prn *command-line-args*)

(println (.. (Runtime/getRuntime)
             totalMemory))

Execution time

$ cp script.clj ~/bin/cljtest2
# ~/bin is on my $PATH
$ chmod +x ~/bin/cljtest2
$ time cljtest2 "Yo" "Hey" 1 3 4 - -ff
Hello!
"2019-03-01T17:22:43.564Z"
("Yo" "Hey" "1" "3" "4" "-" "-ff")
268435456

real	0m2.073s
user	0m6.073s
sys	0m0.297s
$ 

What manner of sorcery is this?

I love this.

Strong odds that someone with better shell-fu than me could reads the deps directly from the file!

Puts me in mind of my native literate programming hack with python: https://gist.github.com/0atman/36574328fdb2d390834c1d878ac4c32f

What is the deal with the underscores?

An attempt to explain the sorcery at work here, step by step:

First, this is a shell script. The shell treats a # character up to the end of the line as a comment.

Clojure comments are the ; character up to the next end of line, but there are two other ways to make Clojure ignore some of what it reads.

(1) When the Clojure reader encounters the pair of characters #_, it will then read the next form, which must be legal Clojure syntax, and then discard it. https://clojure.org/guides/weird_characters#_discard

(2) When the Clojure reader encounters the pair of characters #!, it treats the rest of the line after that as a comment, the same as it does for the semicolon ; character. This has been in Clojure since 2009, but is very seldom used in production code. It appears to have been put into Clojure specifically for the use case of ignoring the first line of a Unix/Linux script.

So you use this file as an executable file on a Unix-like or Linux system, which interprets the first two characters #! to mean "the rest of the line is a command to invoke, with this file as input". It says to run /bin/sh on it. https://en.wikipedia.org/wiki/Shebang_(Unix)

/bin/sh reads it, treats lines 1, 2, and 4 as comments because they begin with # characters. line 3 is blank. The shell executes nothing for blank lines.

Lines 5 through 7 assign a value to the variable named DEPS. Its value is a string.

line 9 is a comment again. Lines 10 through 12 assign a value to a variable named OPTS. Its value is also a string.

Line 14 is another command for the shell to execute. exec is a command that means "take the rest of the line as a command to execute, and make this process effectively start over and execute that". That is, without the exec, the shell would execute the command in a new child process, and when it completed, the shell would continue attempting to execute further lines in the file. But because exec replaces the current process with the one that executes the command, the shell process is now gone, and it will never "come back" and try to execute commands from any later lines. https://www.geeksforgeeks.org/exec-command-in-linux-with-examples/

So now what was the shell process is instead the process running the command clojure $OPTS -Sdeps "$DEPS" "$0" "$@".

The clojure command does various things like checking a local cache of dependencies to see if they are already on the local file system, and if not, tries to go to the network and retrieve whatever dependencies are needed. Assuming that all goes well there, the clojure command then starts a Java JVM process with appropriate command line options so that the JVM calls a method in clojure.main that reads the lines of the file, from the beginning, reading them using the Clojure reader and evaluating them, similar to what would happen if you entered them into a Clojure REPL.

As mentioned in numbered item (2) above, the Clojure reader treats everything from #! to the end of a line as a comment, so the first line is a comment for Clojure, too.

When the Clojure reader reads the #_ on line 2, it then reads whatever the next form is, which is the entire parenthesized expression starting on line 2, ending on line 16, since the right paren on line 16 balances the left one on line 2. Everything on the lines between is legal Clojure data.

Why the #_DEPS instead of # DEPS on line 4, and a similar thing on line 9? Because If it were # DEPS, while bash still treats that all as part of a comment, the Clojure reader treats # DEPS as a reader tag: https://clojure.org/guides/weird_characters#tagged_literals By putting an underscore immediately after the # character, the Clojure reader reads DEPS as a Clojure symbol, and discards it.

Once the Clojure reader reads the list on lines 2 through 16, it discards it, because of the #_ immediately before it on line 2.

Now the Clojure reader continues reading expressions and evaluating them in the rest of the file, again, very similarly to how it would do if you were entering them at a REPL.

Mostly the same description I have checked in here: https://github.com/jafingerhut/dotfiles/blob/master/templates/clj-template-README.md but it may have a few more details fleshed out.

I should mention that "$0" means the currently executing script file. I'm passing the current file to clojure to execute. "$@" adds all of the arguments passed to the script.

I love this.

Strong odds that someone with better shell-fu than me could reads the deps directly from the file!

If you are willing to rely on bash (or zsh), you could use

DEPS=$(<deps.edn)

instead.

That is cool. Thanks for the explanations!

So you have to be a little careful about what you put between the parens on lines 2 and 16. For example I had to add quotes around the URL in this line to make the Clojure reader happy:

   #_this shebang genius is from "https://gist.github.com/ericnormand/6bb4562c4bc578ef223182e3bb1e72c5"

Just in case somebody is looking at this and is okay with having separate deps.edn (I do, mainly for cider-jack-in-clj), then this could be simplified to a single string:

":";exec clojure -M -m $(basename $0 .clj)

This should be a first line of a file. From Clojure's point of view, it's a string ":" and then a comment (since it starts with ;), so a no-op practically.

From shell's point of view, if there is no shebang and no pound sign as a first character of a file, then this file is going to be run with /bin/sh. : is an empty command (try it out in your shell), then ; is a command separator and then next command is exec effectively handing control over to clojure.

All this stuff here is to pass -m namespace to clojure, so that it will call function -main in that script. This means you can safely eval this in REPL how many times you want without re-executing initialization code. This is done by $(basename $0 .clj), of course.

Babashka sets "babashka.file" system property to a file being run, which simplifies all that machinery. clojure does nothing similar, so this is the way I came up with. Would love to find out if there is a way like Python's if __name__ == "__main__".

I just figured out a way to make it even more portable by adding a step that install a specific version of Clojure local to the script.

#!/bin/sh
#_(

   #_DEPS is same format as deps.edn. Multiline is okay.
   DEPS='
   {:deps {
           clj-http/clj-http {:mvn/version "3.12.3"}
           cheshire/cheshire {:mvn/version "5.11.0"}
           }}
   '

   #_You can put other options here
   OPTS='
   -J-Xms4m -J-Xmx256m
   '

if [[ ! -x .local/bin/clojure ]]; then
  [[ ! -d .local ]] && mkdir .local
  pushd .local
  curl -O https://download.clojure.org/install/posix-install-1.11.1.1273.sh
  chmod +x posix-install-1.11.1.1273.sh
  ./posix-install-1.11.1.1273.sh -p $PWD
  popd
fi

exec .local/bin/clojure $OPTS -Sdeps "$DEPS" "$0" "$@"
)

(require 
  '[clj-http.client :as client]
  '[clojure.pprint :as pp])

(defn -main [& args]
  (pp/pprint (:body (client/get "https://www.example.com" {}))))

(apply -main *command-line-args*)

So with this small block added, your only dependency is java.