Quirks of Clojure def

def-quirks.clj

; Clojure is not an interpreted language. Every top level form is compiled before it's run.
; If a form attempts to refer to an undefined variable, compilation fails.
(let [] ; ignore the useless 'let' for now - we'll get back to why it's there later.
  (inc a))
;=> Syntax error compiling at (REPL:1:1).
;=> Unable to resolve symbol: a in this context

(def a 1)
; Now that the variable is defined, compilation of the form below works fine
(let []
  (inc a))
;=> 2

; Interestingly, Clojure compiler specifically supports forms where variable definition
; happens inside of it. That's why 'def' is a special form and not a macro.
(let []
  (def b 1) ; this variable is not defined when compilation starts
  (inc b))
;=> 2

; You could think 'def' declares the variable at macroexpansion but that's not the case.
; 'def' really is a special form:
; https://github.com/clojure/clojure/blob/a29f9b911b569b0a4890f320ec8f946329bbe0fd/src/jvm/clojure/lang/Compiler.java#L407
(macroexpand '(def c 1))
;=> (def c 1)
c
;=> Syntax error compiling at (REPL:0:0).
;=> Unable to resolve symbol: c in this context

; The compiler actually checks that you don't use a variable before it's declared.
(let []
  (inc c)
  (def c 1))
;=> Syntax error compiling at (REPL:0:0).
;=> Unable to resolve symbol: c in this context

; Let's see how fool-proof the compiler is.
; What if the form defines a var but an exception happens before it gets a chance?
(let []
  (throw (Exception. "Stop right there"))
  (def c 1)
  (inc c))
;=> Execution error at user/eval7525 (REPL:1).
;=> Stop right there

; Turns out the variable got declared regardless
c
;=> #object[clojure.lang.Var$Unbound 0x460aa24e "Unbound: #'user/c"]

; Weirdly, the variable isn't declared if we replace 'let' with 'do'.
; Turns out 'do' blocks are kind of interpreted after all:
; https://github.com/clojure/clojure/blob/a29f9b911b569b0a4890f320ec8f946329bbe0fd/src/jvm/clojure/lang/Compiler.java#L7166
(do
  (throw (Exception. "Stop right there"))
  (def d 1)
  (inc d))
;=> Execution error at user/eval7529 (REPL:2).
;=> Stop right there
d
;=> Syntax error compiling at (REPL:0:0).
;=> Unable to resolve symbol: d in this context

(ns user) ; make sure we're in user namespace
; While 'def' is a special form, 'ns' is not. The compiler is thus unable to follow
; namespace switches. This shows that you can't actually use 'def' in 'let' the same
; as you would outside of it.
(let []
  (ns hide-e)
  (def e 1) ; You'd expect hide-e/e to be defined
  (inc e)) ; But we actually got user/e
;=> 2
hide-e/e
;=> Syntax error compiling at (REPL:0:0).
;=> No such var: hide-e/e
user/e
;=> 1

; As shown, making 'def' a special form does not avoid subtle problems. You cannot
; take a REPL session, wrap it in a 'let' block and expect it to work the same.
; Unsurprisingly, mutation is hard.

; On the bright side, def-in-let is a rather uncommon pattern, so few will
; ever be affected.

; Why do I bring this up then?
; Because I believe 'def' should have been a regular macro, just like 'ns' is.
; def-in-let is not a compelling case enough to warrant a separate language feature.
; This language feature is leaky anyway, and the whole problem can easily be mitigated
; with explicit declarations ahead of time.
; Removing 'def' as a special form would make for a smaller core language - easier
; to understand, analyse, write macros for, or maintain a compiler.

; One could have a 'def' macro that declares the var ahead of time, but it's not 
; without issues.
(defmacro def' [name val]
  (intern *ns* name) ; declare the variable at macroexpansion time
  `(intern '~(symbol (str *ns*)) '~name ~val)) ; emit code to assign the value at runtime

; def' still supports def-in-let
(let []
  (def' f 1)
  (inc f))
;=> 2

; But it will leave side-effects even if compilation fails (note, this isn't about
; runtime failures anymore)
(def' g (inc h)) ; This will blow up at compilation time
; But g got declared regardless
g
;=> #object[clojure.lang.Var$Unbound 0x8beb0dd "Unbound: #'user/g"]

; Unfortunately, I think it's too late at this stage to turn 'def' into a macro.
; Leaving vars behind is unacceptable, and taking away def-in-let is against
; no breaking changes policy, even if the feature is rarely used.
; It's a shame it wasn't a macro from day one I guess.