Haskell projects with lots of dependencies on Travis CI
If your Haskell project has lots of dependencies, you can find yourself reaching the timeout while compiling them - the free service has a timeout of 50 minutes. How do we address this?
First, a quick reminder of build statuses in Travis CI:
All builds finish with one of four possible statuses: succeeded, failed,
errored, or (manually) cancelled. The difference between a failed and errored
build is that a build is marked as errored if any of the build steps in the
install phase fails, whereas a failed build is where one of the build steps
build phase fails.
The general idea is to address the issue of builds timing out by allowing the next build to pick up where the previous one left off, by ensuring that the cache is preserved between the builds.
This solution is based on a few observations:
- Storing the build cache does count as part of the build as far as timeouts are concerned - if we want the build cache to be stored, we need to make sure the build finishes with a little time to spare for uploading the new cache.
- The build cache is stored on succeeded and failed builds, but not errored builds. (Note: this may change in the future, see: https://github.com/travis-ci/travis-ci/issues/4472)
- If a build is restarted, it starts with the same version of the cache as it did the first time. So if we want to pick up where we left off, we can't use the 'Restart build' button in Travis' web UI, we need to trigger a new build by pushing a new commit.
- Once the cache is set up, builds are relatively quick. It is fairly rare that so many dependencies must be rebuilt that we do hit the timeout - the only occasions I am aware of that might cause this are switching Stackage snapshots or GHC versions. So it's acceptable if it's a little bit of a faff to get the cache set up again, as long as it's easy.
So how does it work? The key is a handy tool called
timeout, which is part of
GNU coreutils. This tool can run a command but kill it if it fails to complete
in a specified amount of time.
One thing to be careful of is that subsequent build steps are still run even if
one build step fails. So I recommend writing a separate bash script and having
build phase in
.travis.yml look like this:
script: - ./travis-script.sh
travis.script.sh, have something like this:
#!/bin/bash set -x # Install GHC and build dependencies timeout 40m stack --no-terminal --jobs=1 --install-ghc build --only-dependencies ret=$? case "$ret" in 0) # continue ;; 124) echo "Timed out while installing dependencies." echo "Try building again by pushing a new commit." exit 1 ;; *) echo "Failed to install dependencies; stack exited with $ret" exit "$ret" ;; esac # Build your project stack --no-terminal --jobs=1 build --pedantic
For Travis OSX builds, you can run
brew update && brew install coreutils and
timeout tool will be available as
gtimeout. I'm not sure if the
brew update is necessary but the Travis docs recommend it.