Auto-deploying Doxygen documentation to gh-pages with Travis CI

auto-deploy_documentation.md

Auto-deploying Doxygen documentation to gh-pages with Travis CI

This explains how to setup for GitHub projects which automatically generates Doxygen code documentation and publishes the documentation to the gh-pages branch using Travis CI. This way only the source files need to be pushed to GitHub and the gh-pages branch is automatically updated with the generated Doxygen documentation.

Sign up for Travis CI and add your project

Get an account at Travis CI. Turn on Travis for your repository in question, using the Travis control panel.

Create a clean gh-pages branch

To create a clean gh-pages branch, with no commit history, from the master branch enter the code below in the Git Shell. This will create a gh-pages branch with one file, the README.md in it. It doesn't really matter what file is uploaded in it since it will be overwritten when the automatically generated documentation is published to the gh-pages branch.

cd /path/to/repository
git checkout --orphan gh-pages
git rm -rf .
echo "My gh-pages branch" > README.md
git add .
git commit -a -m "Clean gh-pages branch"
git push origin gh-pages

Create a generate documentation and deploy script

This script will generate the Doxygen documentation using the DOXYFILE for configuration and publish it to the gh-pages branch. Below is an example script that I use. This script will create a file called doxygen.log in the root of the gh-pages branch, containing the output of Doxygen. Besides the log it, of course, also generates the documentation itself. It is importand that the Doxygen configuration file has the OUTPUT_DIRECTORY set to empty and the INPUT starting with $(TRAVIS_BUILD_DIR). This way the script will know where the output of Doxygen is going to be located and can find the input files.

Script name: generateDocumentationAndDeploy.sh Script location: root of the repository

#!/bin/sh
################################################################################
# Title         : generateDocumentationAndDeploy.sh
# Date created  : 2016/02/22
# Notes         :
__AUTHOR__="Jeroen de Bruijn"
# Preconditions:
# - Packages doxygen doxygen-doc doxygen-latex doxygen-gui graphviz
#   must be installed.
# - Doxygen configuration file must have the destination directory empty and
#   source code directory with a $(TRAVIS_BUILD_DIR) prefix.
# - An gh-pages branch should already exist. See below for mor info on hoe to
#   create a gh-pages branch.
#
# Required global variables:
# - TRAVIS_BUILD_NUMBER : The number of the current build.
# - TRAVIS_COMMIT       : The commit that the current build is testing.
# - DOXYFILE            : The Doxygen configuration file.
# - GH_REPO_NAME        : The name of the repository.
# - GH_REPO_REF         : The GitHub reference to the repository.
# - GH_REPO_TOKEN       : Secure token to the github repository.
#
# For information on how to encrypt variables for Travis CI please go to
# https://docs.travis-ci.com/user/environment-variables/#Encrypted-Variables
# or https://gist.github.com/vidavidorra/7ed6166a46c537d3cbd2
# For information on how to create a clean gh-pages branch from the master
# branch, please go to https://gist.github.com/vidavidorra/846a2fc7dd51f4fe56a0
#
# This script will generate Doxygen documentation and push the documentation to
# the gh-pages branch of a repository specified by GH_REPO_REF.
# Before this script is used there should already be a gh-pages branch in the
# repository.
# 
################################################################################

################################################################################
##### Setup this script and get the current gh-pages branch.               #####
echo 'Setting up the script...'
# Exit with nonzero exit code if anything fails
set -e

# Create a clean working directory for this script.
mkdir code_docs
cd code_docs

# Get the current gh-pages branch
git clone -b gh-pages https://git@$GH_REPO_REF
cd $GH_REPO_NAME

##### Configure git.
# Set the push default to simple i.e. push only the current branch.
git config --global push.default simple
# Pretend to be an user called Travis CI.
git config user.name "Travis CI"
git config user.email "travis@travis-ci.org"

# Remove everything currently in the gh-pages branch.
# GitHub is smart enough to know which files have changed and which files have
# stayed the same and will only update the changed files. So the gh-pages branch
# can be safely cleaned, and it is sure that everything pushed later is the new
# documentation.
rm -rf *

# Need to create a .nojekyll file to allow filenames starting with an underscore
# to be seen on the gh-pages site. Therefore creating an empty .nojekyll file.
# Presumably this is only needed when the SHORT_NAMES option in Doxygen is set
# to NO, which it is by default. So creating the file just in case.
echo "" > .nojekyll

################################################################################
##### Generate the Doxygen code documentation and log the output.          #####
echo 'Generating Doxygen code documentation...'
# Redirect both stderr and stdout to the log file AND the console.
doxygen $DOXYFILE 2>&1 | tee doxygen.log

################################################################################
##### Upload the documentation to the gh-pages branch of the repository.   #####
# Only upload if Doxygen successfully created the documentation.
# Check this by verifying that the html directory and the file html/index.html
# both exist. This is a good indication that Doxygen did it's work.
if [ -d "html" ] && [ -f "html/index.html" ]; then

    echo 'Uploading documentation to the gh-pages branch...'
    # Add everything in this directory (the Doxygen code documentation) to the
    # gh-pages branch.
    # GitHub is smart enough to know which files have changed and which files have
    # stayed the same and will only update the changed files.
    git add --all

    # Commit the added files with a title and description containing the Travis CI
    # build number and the GitHub commit reference that issued this build.
    git commit -m "Deploy code docs to GitHub Pages Travis build: ${TRAVIS_BUILD_NUMBER}" -m "Commit: ${TRAVIS_COMMIT}"

    # Force push to the remote gh-pages branch.
    # The ouput is redirected to /dev/null to hide any sensitive credential data
    # that might otherwise be exposed.
    git push --force "https://${GH_REPO_TOKEN}@${GH_REPO_REF}" > /dev/null 2>&1
else
    echo '' >&2
    echo 'Warning: No documentation (html) files have been found!' >&2
    echo 'Warning: Not going to push the documentation to GitHub!' >&2
    exit 1
fi

Create the Travis CI file

This script will handle the build and call the previous created generateDocumentationAndDeploy script.

Script name: .travis.yml Script location: root of the repository

# This will run on Travis' 'new' container-based infrastructure
sudo: false

# Blacklist
branches:
  except:
    - gh-pages

# Environment variables
env:
  global:
    - GH_REPO_NAME: <your_repo>
    - DOXYFILE: $TRAVIS_BUILD_DIR/<path_to_your_doxygen_configuration_file>
    - GH_REPO_REF: github.com/<your_name>/<your_repo>.git

# Install dependencies
addons:
  apt:
    packages:
      - doxygen
      - doxygen-doc
      - doxygen-latex
      - doxygen-gui
      - graphviz

# Build your code e.g. by calling make
script:
  - make

# Generate and deploy documentation
after_success:
  - cd $TRAVIS_BUILD_DIR
  - chmod +x generateDocumentationAndDeploy.sh
  - ./generateDocumentationAndDeploy.sh

Provide Travis CI with encrypted credentials for publishing to gh-pages

A tricy part of this setup is to provide Travis CI with the ability to deploy to gh-pages without making the required credentials public. The solution for this is to use a personal acces token for GitHub and use Travis' encryption support.

Generating a personal access token for GitHub

0. In GitHub, go to settings and Personal access tokens;
1. Click Generate a new token and fill in the description. E.g. <your_repo> Travis CI Documentation;
2. As scope select public_repo;
3. Click Generate token;
4. Copy the generated token.

Adding the secure variable to Travis CI

To add this personal access token to Travis CI there are two methods. In this example I've used the fist method, since it is the easiest one.

Method 1: Using the Repository Settings

Similar to the Travis CI documentation

0. Go to the repository in Travis CI and go to settings;
1. In the Environment Variables section give the variable a Name, in this case GH_REPO_TOKEN;
2. Paste the copied token as Value;
3. Click on Add;
4. Make sure the Display value in build log switch is OFF.

Method 2: Using the Encrypted Variables

Follow the method described by the Travis CI documentation

0. Install the Travis CI client;
1. Run the following command travis encrypt GH_REPO_TOKEN=<copied_personal_acces_github_token>;
2. This will give a very long line like secure: <encrypted_token>
3. Copy this line and add it to the environment variables in the .travis.yml file as shown below.

env:
  global:
  - secure: <encrypted_token>

See it in action

I use this setup for one of my projects. The documentation of that project is hosted on the gh-pages branch.

The relevant files for this setup are:

• generateDocumentationAndDeploy.sh
• .travis.yml

Feel free to come with any suggestions for this setup!

This is a great manual, really useful for C++ developers, who (to my experience) are less fluent with CI/CD things than, say, frontend devs.
There's one thing, however:

branches:
  except:
    - gh-pages

This will make the documentation update from each branch, which may be wrong on an active project with several feature branches.
The safer way might be:

branches:
  only:
    - master

Works well. However, 'rm -rf *' should be replaced by another command to avoid errors due to the argument list being too long.

Edit: what I ended up using is:

CURRENTCOMMIT=`git rev-parse HEAD`
git reset --hard `git rev-list HEAD | tail -n 1` # Reset working tree to initial commit
git reset --soft $CURRENTCOMMIT # Move HEAD back to where it was

It should be noted that Github Access Tokens are pretty awful in that they don't allow you to restrict access to a single repository. Its best to create a new github user with the minimal required privileges and create the access token from there.

A better way than Access Tokens would probably be to generate - and encrypt - an SSH key which you can then add as a deploy key to the GitHub project in question.
That way the key is only valid for a single project, though considering the size of it you might have to check the encrypted private key in and store a decryption key for it in the secret on Travis instead.

For instance, https://github.com/domenic/zones seems to do this.

It should be noted that Github Access Tokens are pretty awful in that they don't allow you to restrict access to a single repository.

that's correct, escaping Github Access Tokens into the wild is a security risk, since the tokens can be used to push to any repository managed by the token originator.

however, encrypting the tokens with travis encrypt is a lesser risk, since it is possible to bind the token to a given repository (with -r name/repo), so the encrypted token cannot be used with other repositories.

create a new github user with the minimal required privileges and create the access token from there.

the Github Access Tokens can also be created with minimal privileges.

suggestion:

• update the page with travis encrypt -r <name>/<repo> GH_REPO_TOKEN=<copied_personal_acces_github_token>

I would recommend not doing git add ., since that will add any files not in git, like docs/html/*, etc. git add README.md would be better.

I would recommend a few changes to the script:

 GH_REPO_ORG=`echo $TRAVIS_REPO_SLUG | cut -d "/" -f 1`
 GH_REPO_NAME=`echo $TRAVIS_REPO_SLUG | cut -d "/" -f 2`
 GH_REPO_REF="github.com/$GH_REPO_ORG/$GH_REPO_NAME.git"

That would skip the need for passing the GitHub name and location. That way you only need GH_REPO_TOKEN and DOXYFILE.

Relative links in the Doxyfile are broken in this version of the script, I recommend:

 # Create a clean working directory for this script.
 # Get the current gh-pages branch
 git clone -b gh-pages https://git@$GH_REPO_REF code_docs
 cd code_docs

Since that way you are the same number of folders down from the original /docs folder.

I configured my repository using your great tips, my configuration was very simple because there is a builtin integration between Github pages and Travis CI.

My .travis.yml:

# This will run on Travis' 'new' container-based infrastructure
sudo: false

# Blacklist
branches:
  only:
    - master

# Install dependencies
addons:
  apt:
    packages:
      - doxygen
      - doxygen-doc
      - doxygen-latex
      - doxygen-gui
      - graphviz

# Build your code e.g. by calling make
script:
  - cd docs
  - doxygen collab.dox

# Deploy using travis builtin GitHub Pages support
deploy:
  provider: pages
  skip_cleanup: true
  local_dir: $TRAVIS_BUILD_DIR/docs/html
  github_token: $GITHUB_API_KEY
  on:
    branch: master

So the only Travis configuration needed was the variable GITHUB_API_KEY as you indicated before.

Thanks a lot for this Gist, it helped me a lot. Only after your tips I got my project working like a charm. 🥇
My repository using this solution: http://github.com/amadeubarbosa/openbus-collaboration-service

May I ask what license the scripts contained in the documentation are? I'd like to use it in a project of mine, but to do so, I need to know if the licenses are compatible.

Thanks!

What I am looking to avoid is adding the output of my generated documentation to git. I don't want to use Gigabytes of repository diskspace on github while that isn't needed at all. So, I don't think this solution will work for me, as it is still just doing that: committing to gh-pages and pushing that to github. That doesn't seem very nice towards the free service they provide.

I modified this for my purposes using a different documentation system but I also found that committing to master when there aren't any new changes to the documentation causes the build to fail because the auto-commit results in a no changes to commit which is a non-zero exit code. To get around with that I skip trying to commit if there are no changes with

if [[ ! `git status --porcelain` ]]

See my version.

@amadeubarbosa thanks, this is actually the way to go (no need for custom shell scripts)

1. Add clean gh-pages branch to your repository

git checkout --orphan gh-pages
git rm -rf .
echo "my gh-pages branch" > README.md
git add .
git commit -a -m "clean gh-pages branch"
git push origin gh-pages

2. Add Doxyfile to master branch of your repository
3. Sign up for Travis CI and enable your project
4. Generate personal access tokens with public_repo scope
5. Add this token in Travis CI -> Your Project -> Settings -> Environment Variables as GH_REPO_TOKEN
6. Add .travis.yml to master branch of your repository:

sudo: false

branches:
  only:
    - master

addons:
  apt:
    packages:
      - doxygen

script:
  - doxygen Doxyfile

deploy:
  provider: pages
  skip_cleanup: true
  local_dir: docs/html
  github_token: $GH_REPO_TOKEN
  on:
    branch: master

Troubleshoot if needed watching Travis CI output for your project.

Your documentation will be available at:
https://[your github user].github.io/[your github project]/

Working example / documentation