nh2/TemplateHaskell-recompilation-problem.md

## TemplateHaskell-recompilation-problem.md

      
    Raw
  

              TemplateHaskell-recompilation-problem.md
            
          
    "The [TH] Recompilation Problem"

Usually, when you use ghci's :reload or ghc --make
(with -O0 to disable unfoldings which are used for cross-module inlining),
after changing implementation code of functions,
GHC will incrementally recompile only the modules you changed,
making for a fast development experience when iterating on
implementation details.
(When you change API like functions types, export lists, etc.,
GHC must naturally recompile more.)
However, this fast incremental building for non-API changes currently
breaks down when TemplateHaskell (or QuasiQuotes) is used.
For example, you may see when changing A.hs:
[118 of 163] Compiling A ( src/A.hs, dist/build/A.o )
[120 of 163] Compiling B ( src/B.hs, dist/build/B.o ) [TH]
[121 of 163] Compiling C ( src/C.hs, dist/build/C.o ) [TH]
[122 of 163] Compiling D ( src/D.hs, dist/build/D.o ) [TH]
[123 of 163] Compiling E ( src/E.hs, dist/build/E.o ) [TH]

where B, C, D, E depend directly or indirectly on A.
This happens because TemplateHaskell allows a downstream module
to look at ("reify") the value of any imported module,
and generate syntax based on it.
For example, if A.hs contains x = 42, then TH used in B
could inspect whether that x is 42 or 3 and generate different
syntax in B depending on it.
This means that GHC must recompile a module if it uses TH and
any imported module changes in any way.
When that happens, GHC prints [TH] as the recompilation reason.
GHC's check for whether an "imported module changes" is currently
very unsophisticated: a file modification time change (touch) is enough.
(Note that as of writing, GHC's check has various other flaws
and inconsistencies. For example, you can "magically" get around
the [TH] recompilation if you cancel (Ctrl-C) GHC at the right time;
if you then resume, GHC will "forget" that it should do the [TH] check.)
It gets worse: Because modules can re-export other modules or their functions,
this check doens't only apply for modules importing a module directly,
but for all modules that depend on it even indirectly.
That means that if you use TH pervasively, touching any file in your
project is likely to result in O(modules in your project) many
recompiles, instead of O(1), destroying the concept of incremental
recompilation.
There are various ideas on how "The [TH] Recompilation Problem"
can be solved (including improving object-code deterministic compilation
and per-splice dependency tracking).
But until that happens, the use of TemplateHaskell should be reduced
as far as possible to allow for incremental compilation,
and where TH is necessary,
it should be put into separate modules that not downstream of
any modules that need to be changed for fast dev iteration.