Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Auto-deploying built products to gh-pages with Travis

Auto-deploying built products to gh-pages with Travis

This is a set up for projects which want to check in only their source files, but have their gh-pages branch automatically updated with some compiled output every time they push.

Create a compile script

You want a script that does a local compile to e.g. an out/ directory. Let's call this compile.sh for our purposes, but for your project it might be npm build or gulp make-docs or anything similar.

The out/ directory should contain everything you want deployed to gh-pages. That almost always includes an index.html.

Check this script in to your project.

Create a deploy script

Create a deploy script, call it deploy.sh, with the following contents:

#!/bin/bash
set -e # Exit with nonzero exit code if anything fails

SOURCE_BRANCH="master"
TARGET_BRANCH="gh-pages"

function doCompile {
  ./compile.sh
}

# Pull requests and commits to other branches shouldn't try to deploy, just build to verify
if [ "$TRAVIS_PULL_REQUEST" != "false" -o "$TRAVIS_BRANCH" != "$SOURCE_BRANCH" ]; then
    echo "Skipping deploy; just doing a build."
    doCompile
    exit 0
fi

# Save some useful information
REPO=`git config remote.origin.url`
SSH_REPO=${REPO/https:\/\/github.com\//git@github.com:}
SHA=`git rev-parse --verify HEAD`

# Clone the existing gh-pages for this repo into out/
# Create a new empty branch if gh-pages doesn't exist yet (should only happen on first deply)
git clone $REPO out
cd out
git checkout $TARGET_BRANCH || git checkout --orphan $TARGET_BRANCH
cd ..

# Clean out existing contents
rm -rf out/**/* || exit 0

# Run our compile script
doCompile

# Now let's go have some fun with the cloned repo
cd out
git config user.name "Travis CI"
git config user.email "$COMMIT_AUTHOR_EMAIL"

# If there are no changes to the compiled out (e.g. this is a README update) then just bail.
if [ -z `git diff --exit-code` ]; then
    echo "No changes to the output on this push; exiting."
    exit 0
fi

# Commit the "changes", i.e. the new version.
# The delta will show diffs between new and old versions.
git add .
git commit -m "Deploy to GitHub Pages: ${SHA}"

# Get the deploy key by using Travis's stored variables to decrypt deploy_key.enc
ENCRYPTED_KEY_VAR="encrypted_${ENCRYPTION_LABEL}_key"
ENCRYPTED_IV_VAR="encrypted_${ENCRYPTION_LABEL}_iv"
ENCRYPTED_KEY=${!ENCRYPTED_KEY_VAR}
ENCRYPTED_IV=${!ENCRYPTED_IV_VAR}
openssl aes-256-cbc -K $ENCRYPTED_KEY -iv $ENCRYPTED_IV -in deploy_key.enc -out deploy_key -d
chmod 600 deploy_key
eval `ssh-agent -s`
ssh-add deploy_key

# Now that we're all set up, we can push.
git push $SSH_REPO $TARGET_BRANCH

This deploy script depends on an encrypted deploy key file, deploy_key.enc, which we will discuss shortly. The basic idea is that Travis stores a couple environment variables for you, not checked in to your repo, which allow the script to decrypt deploy_key.enc, which is checked in to your repo.

Sign up for Travis and add your project

Get an account at https://travis-ci.org/. Turn on Travis for your repository in question, using the Travis control panel.

Get encrypted credentials

The trickiest part of all this is that you want to give Travis the ability to run your deploy script and push changes to gh-pages, without checking in the necessary credentials to your repo. The solution for this is to use Travis's encrypted file support.

NOTE: an earlier version of this guide recommended generating a GitHub personal access token and encrypting that. Although this is simpler, it is not a good idea in general, since it means any of your repository's collaborators would be able to edit the Travis build script to email them your access token, thus giving them access to all your repositories. The repository-specific deploy key approach is safer.

First, generate a new SSH key. You should not reuse existing SSH keys, and you should not add the SSH key to your GitHub account.

Next, add that deploy key to your repository at https://github.com/<your name>/<your repo>/settings/keys.

Now use the Travis client to encrypt the generated deploy key. The result should look something like this:

$ travis encrypt-file deploy_key
encrypting deploy_key for domenic/travis-encrypt-file-example
storing result as deploy_key.enc
storing secure env variables for decryption

Please add the following to your build script (before_install stage in your .travis.yml, for instance):

    openssl aes-256-cbc -K $encrypted_0a6446eb3ae3_key -iv $encrypted_0a6446eb3ae3_key -in super_secret.txt.enc -out super_secret.txt -d

Pro Tip: You can add it automatically by running with --add.

Make sure to add deploy_key.enc to the git repository.
Make sure not to add deploy_key to the git repository.
Commit all changes to your .travis.yml.

Make note of that encryption label, here "0a6446eb3ae3". This can be public information; it just says which environment variables to use on the Travis server when decrypting this file.

You should follow the instructions and commit deploy_key.enc to the repository. You should also add deploy_key to your .gitignore, or delete it. Ignore the bits about .travis.yml, however; we're going to do that part all custom-like.

Create your .travis.yml file

With all this in hand, you can create a .travis.yml file. It should look like this:

language: generic # don't install any environment

script: bash ./deploy.sh
env:
  global:
  - ENCRYPTION_LABEL: "<.... encryption label from previous step ....>"
  - COMMIT_AUTHOR_EMAIL: "you@example.com"

The $COMMIT_AUTHOR_EMAIL variable will be used in the commits to gh-pages that Travis makes on your behalf.

If your compile script depends on certain environment features, you might want to set up the environment using Travis's built-in abilities, e.g. by changing the language lines like so:

language: node_js
node_js:
- stable

(In this case, by default Travis will install the latest stable Node.js, then run npm install.)

Finishing up

At this point you should have 3-4 files checked in to your repo: compile.sh, deploy.sh, deploy_key.enc, and .travis.yml. If you've also told Travis about your repo, then the first time you push to GitHub with these changes, it will automatically compile and deploy your source!

See it in action

I use basically this exact setup for https://domenic.github.io/zones/. The relevant files are:

(I have inlined the compile script into deploy.sh, so there is no separate compile.sh.)

I'm trying to use this method, but a little confused about what I should be putting in the compile.sh. Would you share what you put in that file?

@drewrwilson You can put anything you'd like, as long as it outputs to out/. e.g., jekyll -d out/.

Another useful example for a static site could be find . -iname "*.js" -o -iname "*.html*" -o -iname "*.css" -type f | parallel htmlcompressor --compress-js --compress-css -o {} {}. (You would need to copy/compile and cd to the out/ directory before running that.)

gnzlbg commented Sep 15, 2015

What scopes should my github token have?

What scopes should my github token have?

I think it only needs public_repo (or repo if you're deploying a private repository).

Nice writeup. Similar to http://benlimmer.com/2013/12/26/automatically-publish-javadoc-to-gh-pages-with-travis-ci/ but it's good to have multiples to look at.

PinkyJie commented Oct 9, 2015

This is exactly what I need, thanks!

Any way to deploy only on pushes to master (or new tags)?

Edit: found some one-liners, but I prefer a bit more verbose:

if [ "$TRAVIS_TAG" = "" ]
then
   echo "Not a tag, not deploying"
   exit 0
else
   echo "==> Building and deploying tag $TRAVIS_TAG <=="
fi

Can I deploy only after a successful build on master?

Can you pls look at this question: http://stackoverflow.com/questions/34183506/yeoman-project-in-travis-ci-failed-to-deploy-dist-directory-to-s3. My deployment is failing , can anybody suggest ?

@ghpabs Instead of overriding the compile step I use after_success, which (as the name suggests) will only run after success:

after_success: "./travis.sh"
dKab commented Jan 11, 2016

should I create gh-pages branch in advance or this command git push --force --quiet "https://${GH_TOKEN}@${GH_REF}" master:gh-pages > /dev/null 2>&1 will create it for me if it doesn't exist?

timwis commented Jan 12, 2016

Very helpful writeup! For those interested, I ended up putting the executing in the deploy section of my travis config so that I could still use the script part of travis for running my tests, and control which branch was deployed rather than deploying everything (including pull requests):

https://github.com/timwis/vizwit/blob/master/.travis.yml#L12-L18

My deploy.sh file also defaults to the local git repo's origin remote url, so you can execute the deploy script manually

https://github.com/timwis/vizwit/blob/master/deploy.sh

You can generate a personal access token in https://github.com/settings/tokens instead of https://github.com/settings/applications.

Or you could just use this Ruby gem that comes with a Rake task that does all this.

https://github.com/jirutka/rake-jekyll

@domenic I'm sure you're a busy man :) but it would be cool if you can update the following part

First, generate a new SSH key. You should not reuse existing SSH keys, and you should not add the SSH key to your GitHub account.

Next, add that deploy key to your repository at https://github.com///settings/keys.

To be more explanatory, and just include something along the lines of:

First, generate a new SSH key, ..., this will not be for your profile keys, but the relevant repository where you want to use gh-pages. Go to your repository settings, https://github.com/USERNAME/PROJECT/settings/keys and add your new key as a deploy key.

Thanks for this gist!

@domenic thank you very much for this! But I have one question) It doesn't work right.

[gh-pages d260bac] Deploy to GitHub Pages: 2351f4184099cb644c8678ad604c4ca80d7228e4
 1 file changed, 2 insertions(+), 2 deletions(-)
iv undefined
The command "bash ./deploy.sh" exited with 1.

I see this message in travis console(

stefanbc commented Jun 8, 2016

Adding to what @BlackEdder said. I've ran into some issues while trying to run the deploy.sh file because it didn't have permissions to run. So I ended up using it like this:

after_success: 
  - chmod a+x deploy.sh
  - ./deploy.sh

And it's working fine like this.

You definitely need to be more careful with your tokens here... any git push or git pull needs to ensure they don't leak out. You should just pipe all output of these commands to /dev/null or store the output in a variable, and never print the whole thing...

This will definitely work, but it's kind of a round-about way of deploying. It might give you extra control if you need it, but it might be simpler to just use the deploy feature of travis:

`deploy:

provider: releases
api_key: "GITHUB OAUTH TOKEN"
file: "FILE TO UPLOAD"
skip_cleanup: true
on:
tags: true
`

https://docs.travis-ci.com/user/deployment/releases

wings27 commented Jul 7, 2016 edited

Very helpful, although I encountered this problem: the travis CI paused, asking for a passphrase when executing "ssh-add deploy_key".
My passphrase is empty. But I do have to hit ENTER to continue, which is impossible within the travis CI interface.
Ten minutes later, the build was terminated due to timeout.
So is there a way that I can skip entering the passphrase? Simulating pressing ENTER perhaps?

...
Agent pid 3037
Enter passphrase for deploy_key:
(10 minutes later...)
No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.
The build has been terminated

same problem as @wings27

RyanNHG commented Jul 17, 2016

@wings27 and @ShikherVerma

I had the same issue because I used a passphrase when generating my public key.

A quick search for the error took me to this stack overflow post:
http://stackoverflow.com/questions/36684631/how-to-do-a-custom-deploy-using-ssh-with-travis-ci

This is really useful, thanks! You might want to incorporate the following change

-if [ -z `git diff --exit-code` ]; then
+if git diff --quiet ; then

Git's --exit-code option does not cause git to output an exit code, and therefore you cannot test it with -z. The --quiet option (which implied --exit-code) is the one you want.

Also, unless the deploy_key.enc is copied into out, you probably want to decode it by specifying it in the parent directory, as ../deploy_key.enc.

adbre commented Aug 2, 2016

Really great script and article!

But the script will abort saying "No changes" if the new version only adds new files (git status only has untracked files).
The following change will detect both modifications and new untracked files.

-if [ -z `git diff --exit-code` ]; then
+if [ $(git status --porcelain | wc -l) -lt 1 ]; then

Also, git will print a warning regarding the git add . command. The syntax is obsolete and should be replaced with git add -A .

I'm with @dspinellis on this:

Also, unless the deploy_key.enc is copied into out, you probably want to decode it by specifying it in the parent directory, as ../deploy_key.enc.

That's definitely important!

thymikee commented Aug 8, 2016

I think it's worth to mention a distinct difference between private and public key:

Next, add a public deploy_key.pub to your repository at https://github.com/<your name>/<your repo>/settings/keys.

Now use the Travis client to encrypt the generated private deploy_key. The result should look something like this:

@awinogradov I had the same problem as you, putting the openssl command in before_install fixed it!

We can add --add flag to the command travis encrypt-file id_rsa (I used version: 1.8.2), sotravis will put the command automatically in the .travis.yml file. No need to decrypt the file in deploy.sh. And if the flag is used move the ssh-add etc, somewhere before executing the command cd out.

aullman commented Aug 26, 2016

I had the same problem as @dspinellis. I was getting the following error:

./deploy_key.enc: No such file or directory
140184029640352:error:02001002:system library:fopen:No such file or directory:bss_file.c:398:fopen('deploy_key.enc','r')

So I changed to point to ../deploy.enc so:

openssl aes-256-cbc -K $ENCRYPTED_KEY -iv $ENCRYPTED_IV -in ../deploy_key.enc -out deploy_key -d

@domenic, @RyanNHG, @wings27 and @ShikherVerma

Did you found a solution for entering the passphrase?
I followed all instructions and generated a ssh key with no passphrase ('').
Travis still tries to enter a passphrase:

Enter passphrase for deploy_key:

Always times out after 10 minutes.

jilulu commented Sep 5, 2016

@marcobiedermann Hi. I followed the instructions here when generating the ssh key, on first attempt I gave the key a passphrase which is nonempty. And I encountered the exact same problem as you did.
Then on second attempt, I simply hit enter when being prompted for a passphrase. Push the changes, and the travis deployment procedure succeeded.

I'm not sure whether the ssh generation process is platform dependent. I'm using zsh under macos 11.6.

has2k1 commented Oct 16, 2016 edited

I thought there was glaring security issue with this method but then I saw this in the documentation.

Please note that encrypted environment variables are not available for pull requests from forks.

@JimTheMan

This will definitely work, but it's kind of a round-about way of deploying. It might give you extra control if you need it, but it might be simpler to just use the deploy feature of travis:

How do you use the deploy feature of travis to push changes to gh-pages? The instructions you linked to are about pushing build results as binaries to releases and only when a new tag is added. Is there some other way to have it push pages to gh-pages and without tags?

greggman commented Nov 23, 2016 edited

As @dspinellis pointed out I needed to use git diff --quiet instead of git diff --exit-code. In my case because the --exit-code passes the entire diff output to bash and bash choked on it.

@stefanbc you can chmod u+x deploy.sh in your local repo then add and commit deploy.sh to the repo. Git tracks permissions (or at least the executable bit)

I think a better way to clean out the existing contents is to switch into the repository and do this

# Clean out existing contents
git ls-files | xargs rm -rf

Also, I think performing the following check before pushing (while not perfect) is probably a good idea

git fetch origin master

if [ "$(git rev-parse origin/master)" == "$SHA" ]; then
  git push "$SSH_REPO" "$TARGET_BRANCH"
else
    echo "The commit this build was started for is not the one on master. Doing nothing."
fi
letmaik commented Jan 11, 2017 edited

@jilulu Have the same problem, asking for passphrase even though none was defined. I created the key using the git bash under Windows. As a test I also generated one using PuTTYgen, but same problem. Maybe this is related: https://ubuntuforums.org/showthread.php?t=1828015

EDIT: It seems as if the encryption (or decryption) of the private key produces garbage. When I output the key content while running in Travis CI after decrypting I get just nonsense.

@jilulu @letmaik @domenic, @RyanNHG, @wings27 and @ShikherVerma I have generated ssh keys using the travis CLI to use it as a deployment keys for ssh, but my deployment script gets struct at ssh-add deploy_key ,due to Enter PassPhrase line ,I didnt keep any passphrase for the key but the ssh-add is asking for a password ,according other online sources the script should automatically run , why is my travis build getting struck. here is over all problem view in an image
screenshot from 2017-01-20 01-18-09

Hey, I was having an error using this. I put all of my stuff in the description of the issue. It is right over here.

popomore commented Feb 7, 2017

Good post, but contains some typo

- openssl aes-256-cbc -K $encrypted_0a6446eb3ae3_key -iv $encrypted_0a6446eb3ae3_key -in super_secret.txt.enc -out super_secret.txt -d
+ openssl aes-256-cbc -K $encrypted_0a6446eb3ae3_key -iv $encrypted_0a6446eb3ae3_iv -in super_secret.txt.enc -out super_secret.txt -d
andiwinata commented Feb 21, 2017 edited

hey I was having trouble with Enter Passphrase: too, even though I generated the key without any passphrase,
so I ended up with replacing

eval `ssh-agent -s`
ssh-add deploy_key

with

cp deploy_key ~/.ssh/id_rsa

and it seems to be working fine now

bfritz commented Feb 28, 2017

@domenic Recommend not writing private key without passphrase to disk. Something like this might be better:

eval `ssh-agent -s`
openssl aes-256-cbc -K $ENCRYPTED_KEY -iv $ENCRYPTED_IV -in deploy_key.enc -d | ssh-add -
doxxx commented Mar 14, 2017

Travis appears to have a deployment provider specifically for GitHub Pages now: https://docs.travis-ci.com/user/deployment/pages/

What if we don't want to overwrite older releases on GitHub pages? I'd like to keep hosting multiple compiled versions for testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment