It's a bit annoying that in Nix a nix-build causes an entire rebuild every time. For big Haskell projects this sucks! Can we do better? We sure can! Here's a default.nix that has caching:
{ mkDerivation, aeson, aeson-pretty, base, bifunctors, bytestring
, classy-prelude, comparable-key, containers, either, errors
, hashable, mono-traversable, safe, semigroups, split, stdenv, text
, time, unordered-containers, utf8-string, vector
, dist ? null # <-- Not a Haskell dependency, but a directory
}:
mkDerivation {
pname = "circuithub-prelude";
version = "0.0.28";
src = ./.;
libraryHaskellDepends = [
aeson aeson-pretty base bifunctors bytestring classy-prelude
comparable-key containers either errors hashable mono-traversable
safe semigroups split text time unordered-containers utf8-string
vector
];
homepage = "https://github.com/circuithub/circuithub-prelude";
description = "CircuitHub's Prelude that builds on top of the ClassyPrelude";
license = stdenv.lib.licenses.mit;
# We will output to $out as normal, but also to $dist
outputs = [ "out" "dist" ];
# Before we can build, we restore the previous dist/ directory
preConfigure = ''
# Before we do anything, capture the MD5 sums of all source files.
# We will compare against this in subsequent builds.
mkdir -p $dist
find . -type f | xargs md5sum | sort > $dist/MD5SUMS
# Restore the old dist/ directory, with its MD5SUMS
mkdir -p dist
cp -r ${dist}/* dist/ # */
chmod +w -R dist/
touch dist/MD5SUMS
# Touch any files whose MD5SUM has changed since the last build
join $dist/MD5SUMS dist/MD5SUMS -v 1 | cut -d' ' -f 2 | while read filename; do
echo "$filename" has changed
touch "$filename" || true
done
# Touch all dist/ files to be 2 hours in the past.
# Note that source code will be last modified in 1970 *by default*
# but changed to the current time by the loop above.
find dist/build -print | while read filename; do
touch -d "$(date -R -r "$filename") - 2 hours" "$filename"
done
'';
# We need to store dist/ in the $dist output.
preInstall = "cp -R dist/build $dist/";
doHaddock = false;
}This is one of our projects at CircuitHub. Notice that we have a new non-Haskell input - dist. This is generated by using multiple outputs whenever we do a build. Here's what it all looks like:
# ollie at nixos-desktop in ~/work/circuithub on git:master ✖︎ [19:11:28]
→ nix-build -E '(import prelude/shell.nix { nixpkgs = import /home/ollie/nixpkgs {}; }).dist' ~/work/circuithub
these derivations will be built:
/nix/store/90m4km9fa52hpnmwhn3r4ifxa7c72rjg-circuithub-prelude-0.0.28.drv
building path(s) ‘/nix/store/wwj3ldjzd0kwyjx6zm67fbiawvzna04m-circuithub-prelude-0.0.28’, ‘/nix/store/zgiclk9wqzjhlsi2fpfbk634kgvkms5b-circuithub-prelude-0.0.28-dist’
setupCompilerEnvironmentPhase
Build with /nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3.
unpacking sources
unpacking source archive /nix/store/r17yfr51nfqdgg8w6ip41fbbj1zpmk7m-prelude
source root is prelude
patching sources
compileBuildDriverPhase
setupCompileFlags: -package-db=/tmp/nix-build-circuithub-prelude-0.0.28.drv-0/package.conf.d -j8 -threaded
[1 of 1] Compiling Main ( /nix/store/4mdp8nhyfddh7bllbi7xszz7k9955n79-Setup.hs, /tmp/nix-build-circuithub-prelude-0.0.28.drv-0/Main.o )
Linking Setup ...
configuring
./default.nix has changed
./Setup has changed
configureFlags: --verbose --prefix=/nix/store/wwj3ldjzd0kwyjx6zm67fbiawvzna04m-circuithub-prelude-0.0.28 --libdir=$prefix/lib/$compiler --libsubdir=$pkgid --with-gcc=gcc --package-db=/tmp/nix-build-circuithub-prelude-0.0.28.drv-0/package.conf.d --ghc-option=-optl=-Wl,-rpath=/nix/store/wwj3ldjzd0kwyjx6zm67fbiawvzna04m-circuithub-prelude-0.0.28/lib/ghc-7.10.3/circuithub-prelude-0.0.28 --enable-split-objs --disable-library-profiling --disable-executable-profiling --enable-shared --enable-library-vanilla --enable-executable-dynamic --enable-tests
Configuring circuithub-prelude-0.0.28...
Flags chosen: dev=False
Dependency aeson >=0.7.0.6: using aeson-0.11.1.4
Dependency aeson-pretty >=0.7.1: using aeson-pretty-0.7.2
Dependency base >=4.7.0.2 && <5: using base-4.8.2.0
Dependency bifunctors >=4.1.1.1: using bifunctors-5.2
Dependency bytestring >=0.10.4.0: using bytestring-0.10.6.0
Dependency classy-prelude >=0.12.3: using classy-prelude-0.12.7
Dependency comparable-key >=0.0.1: using comparable-key-0.0.1
Dependency containers >=0.5.5.1: using containers-0.5.6.2
Dependency either >=4.3.0.1: using either-4.4.1
Dependency errors >=1.4.7 && <3.0: using errors-2.1.2
Dependency hashable >=1.2.2.0: using hashable-1.2.4.0
Dependency mono-traversable >=0.6.1: using mono-traversable-0.10.2
Dependency safe >=0.3.8: using safe-0.3.9
Dependency semigroups >=0.15.2: using semigroups-0.18.1
Dependency split >=0.2.2 && <0.3: using split-0.2.3
Dependency text >=1.1.1.3: using text-1.2.2.1
Dependency time >=1.4.2: using time-1.5.0.1
Dependency unordered-containers >=0.2.5.0: using unordered-containers-0.2.7.0
Dependency utf8-string >=0.3.8: using utf8-string-1.0.1.1
Dependency vector >=0.10.9.1 && <0.12: using vector-0.11.0.0
Using Cabal-1.22.5.0 compiled by ghc-7.10
Using compiler: ghc-7.10.3
Using install prefix:
/nix/store/wwj3ldjzd0kwyjx6zm67fbiawvzna04m-circuithub-prelude-0.0.28
Binaries installed in:
/nix/store/wwj3ldjzd0kwyjx6zm67fbiawvzna04m-circuithub-prelude-0.0.28/bin
Libraries installed in:
/nix/store/wwj3ldjzd0kwyjx6zm67fbiawvzna04m-circuithub-prelude-0.0.28/lib/ghc-7.10.3/circuithub-prelude-0.0.28
Private binaries installed in:
/nix/store/wwj3ldjzd0kwyjx6zm67fbiawvzna04m-circuithub-prelude-0.0.28/libexec
Data files installed in:
/nix/store/wwj3ldjzd0kwyjx6zm67fbiawvzna04m-circuithub-prelude-0.0.28/share/x86_64-linux-ghc-7.10.3/circuithub-prelude-0.0.28
Documentation installed in:
/nix/store/wwj3ldjzd0kwyjx6zm67fbiawvzna04m-circuithub-prelude-0.0.28/share/doc/x86_64-linux-ghc-7.10.3/circuithub-prelude-0.0.28
Configuration files installed in:
/nix/store/wwj3ldjzd0kwyjx6zm67fbiawvzna04m-circuithub-prelude-0.0.28/etc
No alex found
Using ar found on system at:
/nix/store/4bqj6wzfq81lhqysh224lqbsi533cyin-binutils-2.26-dev/bin/ar
No c2hs found
No cpphs found
Using gcc version 5.3.0 given by user at:
/nix/store/kz3avmqpq66xi8rfxsj9c9dvfva661s3-gcc-wrapper-5.3.0/bin/gcc
Using ghc version 7.10.3 found on system at:
/nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3/bin/ghc
Using ghc-pkg version 7.10.3 found on system at:
/nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3/bin/ghc-pkg
No ghcjs found
No ghcjs-pkg found
No greencard found
Using haddock version 2.16.1 found on system at:
/nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3/bin/haddock
No happy found
Using haskell-suite found on system at: haskell-suite-dummy-location
Using haskell-suite-pkg found on system at: haskell-suite-pkg-dummy-location
No hmake found
Using hpc version 0.67 found on system at:
/nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3/bin/hpc
Using hsc2hs version 0.67 found on system at:
/nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3/bin/hsc2hs
Using hscolour version 1.24 found on system at:
/nix/store/pgy4pkf5wm516x2h657gdalqxkdj4wr8-hscolour-1.24/bin/HsColour
No jhc found
Using ld found on system at:
/nix/store/kz3avmqpq66xi8rfxsj9c9dvfva661s3-gcc-wrapper-5.3.0/bin/ld
No lhc found
No lhc-pkg found
No pkg-config found
Using strip version 2.26 found on system at:
/nix/store/4bqj6wzfq81lhqysh224lqbsi533cyin-binutils-2.26-dev/bin/strip
Using tar found on system at:
/nix/store/fv8lbg8jk2amwsxzrw82r028r8fm3rl3-gnutar-1.28/bin/tar
No uhc found
building
Building circuithub-prelude-0.0.28...
Preprocessing library circuithub-prelude-0.0.28...
In-place registering circuithub-prelude-0.0.28...
running tests
Package has no test suites.
No compilation happened, because I haven't changed the source code! Let's try changing it:
echo "fortyTwo = 42" >> prelude/CHPrelude.hs
and now we run nix-build...
# ollie at nixos-desktop in ~/work/circuithub on git:master ✖︎ [19:20:10]
→ nix-build -E '(import prelude/shell.nix { nixpkgs = import /home/ollie/nixpkgs {}; }).dist' ~/work/circuithub
these derivations will be built:
/nix/store/ijcr7lg2802qmpzvpipjqicmdn3fz1i6-circuithub-prelude-0.0.28.drv
building path(s) ‘/nix/store/ihy7pfdxyzxhhv3ndrq5dbgl6hnps8mk-circuithub-prelude-0.0.28-dist’, ‘/nix/store/xg13lmk5ph456a31kw31kmbk0rsldm71-circuithub-prelude-0.0.28’
setupCompilerEnvironmentPhase
Build with /nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3.
unpacking sources
unpacking source archive /nix/store/lfwg3ymwa7gag3mdpf2k07gjdxbfci20-prelude
source root is prelude
patching sources
compileBuildDriverPhase
setupCompileFlags: -package-db=/tmp/nix-build-circuithub-prelude-0.0.28.drv-0/package.conf.d -j8 -threaded
[1 of 1] Compiling Main ( /nix/store/4mdp8nhyfddh7bllbi7xszz7k9955n79-Setup.hs, /tmp/nix-build-circuithub-prelude-0.0.28.drv-0/Main.o )
Linking Setup ...
configuring
./Setup has changed
./CHPrelude.hs has changed
configureFlags: --verbose --prefix=/nix/store/xg13lmk5ph456a31kw31kmbk0rsldm71-circuithub-prelude-0.0.28 --libdir=$prefix/lib/$compiler --libsubdir=$pkgid --with-gcc=gcc --package-db=/tmp/nix-build-circuithub-prelude-0.0.28.drv-0/package.conf.d --ghc-option=-optl=-Wl,-rpath=/nix/store/xg13lmk5ph456a31kw31kmbk0rsldm71-circuithub-prelude-0.0.28/lib/ghc-7.10.3/circuithub-prelude-0.0.28 --enable-split-objs --disable-library-profiling --disable-executable-profiling --enable-shared --enable-library-vanilla --enable-executable-dynamic --enable-tests
Configuring circuithub-prelude-0.0.28...
Flags chosen: dev=False
Dependency aeson >=0.7.0.6: using aeson-0.11.1.4
Dependency aeson-pretty >=0.7.1: using aeson-pretty-0.7.2
Dependency base >=4.7.0.2 && <5: using base-4.8.2.0
Dependency bifunctors >=4.1.1.1: using bifunctors-5.2
Dependency bytestring >=0.10.4.0: using bytestring-0.10.6.0
Dependency classy-prelude >=0.12.3: using classy-prelude-0.12.7
Dependency comparable-key >=0.0.1: using comparable-key-0.0.1
Dependency containers >=0.5.5.1: using containers-0.5.6.2
Dependency either >=4.3.0.1: using either-4.4.1
Dependency errors >=1.4.7 && <3.0: using errors-2.1.2
Dependency hashable >=1.2.2.0: using hashable-1.2.4.0
Dependency mono-traversable >=0.6.1: using mono-traversable-0.10.2
Dependency safe >=0.3.8: using safe-0.3.9
Dependency semigroups >=0.15.2: using semigroups-0.18.1
Dependency split >=0.2.2 && <0.3: using split-0.2.3
Dependency text >=1.1.1.3: using text-1.2.2.1
Dependency time >=1.4.2: using time-1.5.0.1
Dependency unordered-containers >=0.2.5.0: using unordered-containers-0.2.7.0
Dependency utf8-string >=0.3.8: using utf8-string-1.0.1.1
Dependency vector >=0.10.9.1 && <0.12: using vector-0.11.0.0
Using Cabal-1.22.5.0 compiled by ghc-7.10
Using compiler: ghc-7.10.3
Using install prefix:
/nix/store/xg13lmk5ph456a31kw31kmbk0rsldm71-circuithub-prelude-0.0.28
Binaries installed in:
/nix/store/xg13lmk5ph456a31kw31kmbk0rsldm71-circuithub-prelude-0.0.28/bin
Libraries installed in:
/nix/store/xg13lmk5ph456a31kw31kmbk0rsldm71-circuithub-prelude-0.0.28/lib/ghc-7.10.3/circuithub-prelude-0.0.28
Private binaries installed in:
/nix/store/xg13lmk5ph456a31kw31kmbk0rsldm71-circuithub-prelude-0.0.28/libexec
Data files installed in:
/nix/store/xg13lmk5ph456a31kw31kmbk0rsldm71-circuithub-prelude-0.0.28/share/x86_64-linux-ghc-7.10.3/circuithub-prelude-0.0.28
Documentation installed in:
/nix/store/xg13lmk5ph456a31kw31kmbk0rsldm71-circuithub-prelude-0.0.28/share/doc/x86_64-linux-ghc-7.10.3/circuithub-prelude-0.0.28
Configuration files installed in:
/nix/store/xg13lmk5ph456a31kw31kmbk0rsldm71-circuithub-prelude-0.0.28/etc
No alex found
Using ar found on system at:
/nix/store/4bqj6wzfq81lhqysh224lqbsi533cyin-binutils-2.26-dev/bin/ar
No c2hs found
No cpphs found
Using gcc version 5.3.0 given by user at:
/nix/store/kz3avmqpq66xi8rfxsj9c9dvfva661s3-gcc-wrapper-5.3.0/bin/gcc
Using ghc version 7.10.3 found on system at:
/nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3/bin/ghc
Using ghc-pkg version 7.10.3 found on system at:
/nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3/bin/ghc-pkg
No ghcjs found
No ghcjs-pkg found
No greencard found
Using haddock version 2.16.1 found on system at:
/nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3/bin/haddock
No happy found
Using haskell-suite found on system at: haskell-suite-dummy-location
Using haskell-suite-pkg found on system at: haskell-suite-pkg-dummy-location
No hmake found
Using hpc version 0.67 found on system at:
/nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3/bin/hpc
Using hsc2hs version 0.67 found on system at:
/nix/store/dpwrxddbffl2462frml0myz3aknfypyp-ghc-7.10.3/bin/hsc2hs
Using hscolour version 1.24 found on system at:
/nix/store/pgy4pkf5wm516x2h657gdalqxkdj4wr8-hscolour-1.24/bin/HsColour
No jhc found
Using ld found on system at:
/nix/store/kz3avmqpq66xi8rfxsj9c9dvfva661s3-gcc-wrapper-5.3.0/bin/ld
No lhc found
No lhc-pkg found
No pkg-config found
Using strip version 2.26 found on system at:
/nix/store/4bqj6wzfq81lhqysh224lqbsi533cyin-binutils-2.26-dev/bin/strip
Using tar found on system at:
/nix/store/fv8lbg8jk2amwsxzrw82r028r8fm3rl3-gnutar-1.28/bin/tar
No uhc found
building
Building circuithub-prelude-0.0.28...
Preprocessing library circuithub-prelude-0.0.28...
[22 of 22] Compiling CHPrelude ( CHPrelude.hs, dist/build/CHPrelude.o )
CHPrelude.hs:42:1: Warning:
Top-level binding with no type signature: fortyTwo :: Integer
CHPrelude.hs:42:12: Warning:
Defaulting the following constraint(s) to type ‘Integer’
(Num a0) arising from the literal ‘42’
In the expression: 42
In an equation for ‘fortyTwo’: fortyTwo = 42
In-place registering circuithub-prelude-0.0.28...
running tests
Woohoo! Recompilation, and only for what was necessary.