Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
TemplateHaskell: The [TH] Recompilation Problem

"The [TH] Recompilation Problem"

Usually, when you use ghci's :reload or ghc --make (with -O0 to disable unfoldings which are used for cross-module inlining), after changing implementation code of functions, GHC will incrementally recompile only the modules you changed, making for a fast development experience when iterating on implementation details.

(When you change API like functions types, export lists, etc., GHC must naturally recompile more.)

However, this fast incremental building for non-API changes currently breaks down when TemplateHaskell (or QuasiQuotes) is used.

For example, you may see when changing A.hs:

[118 of 163] Compiling A ( src/A.hs, dist/build/A.o )
[120 of 163] Compiling B ( src/B.hs, dist/build/B.o ) [TH]
[121 of 163] Compiling C ( src/C.hs, dist/build/C.o ) [TH]
[122 of 163] Compiling D ( src/D.hs, dist/build/D.o ) [TH]
[123 of 163] Compiling E ( src/E.hs, dist/build/E.o ) [TH]

where B, C, D, E depend directly or indirectly on A.

This happens because TemplateHaskell allows a downstream module to look at ("reify") the value of any imported module, and generate syntax based on it. For example, if A.hs contains x = 42, then TH used in B could inspect whether that x is 42 or 3 and generate different syntax in B depending on it.

This means that GHC must recompile a module if it uses TH and any imported module changes in any way.

When that happens, GHC prints [TH] as the recompilation reason.

GHC's check for whether an "imported module changes" is currently very unsophisticated: a file modification time change (touch) is enough.

(Note that as of writing, GHC's check has various other flaws and inconsistencies. For example, you can "magically" get around the [TH] recompilation if you cancel (Ctrl-C) GHC at the right time; if you then resume, GHC will "forget" that it should do the [TH] check.)

It gets worse: Because modules can re-export other modules or their functions, this check doens't only apply for modules importing a module directly, but for all modules that depend on it even indirectly.

That means that if you use TH pervasively, touching any file in your project is likely to result in O(modules in your project) many recompiles, instead of O(1), destroying the concept of incremental recompilation.

There are various ideas on how "The [TH] Recompilation Problem" can be solved (including improving object-code deterministic compilation and per-splice dependency tracking).

But until that happens, the use of TemplateHaskell should be reduced as far as possible to allow for incremental compilation, and where TH is necessary, it should be put into separate modules that not downstream of any modules that need to be changed for fast dev iteration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.