hdgarrood/large-haskell-projects-on-travis.md

## large-haskell-projects-on-travis.md

      
    Raw
  

              large-haskell-projects-on-travis.md
            
          
    Haskell projects with lots of dependencies on Travis CI

If your Haskell project has lots of dependencies, you can find yourself
reaching the timeout while compiling them - the free service has a timeout of
50 minutes. How do we address this?
First, a quick reminder of build statuses in Travis CI:
All builds finish with one of four possible statuses: succeeded, failed,
errored, or (manually) cancelled. The difference between a failed and errored
build is that a build is marked as errored if any of the build steps in the
install phase fails, whereas a failed build is where one of the build steps
in the build phase fails.
The general idea is to address the issue of builds timing out by allowing the
next build to pick up where the previous one left off, by ensuring that the
cache is preserved between the builds.
This solution is based on a few observations:

Storing the build cache does count as part of the build as far as timeouts
are concerned - if we want the build cache to be stored, we need to make
sure the build finishes with a little time to spare for uploading the new
cache.
The build cache is stored on succeeded and failed builds, but not errored
builds. (Note: this may change in the future, see:
travis-ci/travis-ci#4472)
If a build is restarted, it starts with the same version of the cache as
it did the first time. So if we want to pick up where we left off, we can't
use the 'Restart build' button in Travis' web UI, we need to trigger a new
build by pushing a new commit.
Once the cache is set up, builds are relatively quick. It is fairly rare
that so many dependencies must be rebuilt that we do hit the timeout - the
only occasions I am aware of that might cause this are switching Stackage
snapshots or GHC versions. So it's acceptable if it's a little bit of a faff
to get the cache set up again, as long as it's easy.

So how does it work? The key is a handy tool called timeout, which is part of
GNU coreutils. This tool can run a command but kill it if it fails to complete
in a specified amount of time.
One thing to be careful of is that subsequent build steps are still run even if
one build step fails. So I recommend writing a separate bash script and having
your build phase in .travis.yml look like this:
script:
  - ./travis-script.sh

Then, inside travis.script.sh, have something like this:
#!/bin/bash

set -x

# Install GHC and build dependencies
timeout 40m stack --no-terminal --jobs=1 --install-ghc build --only-dependencies
ret=$?
case "$ret" in
  0)
    # continue
    ;;
  124)
    echo "Timed out while installing dependencies."
    echo "Try building again by pushing a new commit."
    exit 1
    ;;
  *)
    echo "Failed to install dependencies; stack exited with $ret"
    exit "$ret"
    ;;
esac

# Build your project
stack --no-terminal --jobs=1 build --pedantic
For Travis OSX builds, you can run brew update && brew install coreutils and
then the timeout tool will be available as gtimeout. I'm not sure if the
brew update is necessary but the Travis docs recommend it.