Skip to content

Instantly share code, notes, and snippets.

@rogeruiz

rogeruiz/g-devops-presents-workflows-and-separation-of-responsibilities-for-circleci.markdown

Last active March 9, 2018 14:18
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rogeruiz/be6558855c38290ecee725bfcd3d6b5a to your computer and use it in GitHub Desktop.
Save rogeruiz/be6558855c38290ecee725bfcd3d6b5a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
g-devops-presents-workflows-and-separation-of-responsibilities-for-circleci.markdown

#g-devops Presents: Workflows and Separations of Responsibilities for CircleCI

To increase the speed of your software development through faster feedback, shorter reruns, and more efficient use of resources, configure Workflows.

  • CircleCI version 2.0 documentation on Orchestrating Workflows

Disclaimer This talk is about making sure we’re using CircleCI version 2.0 the way it is used at 18F and blessed by the #g-devops folks. Office/Working hours are weekly at 4pm EST on Tuesdays. The next meeting is March 6th, 2018.

This talk meant to be prescriptive like over-the-counter medicine. There are many different brands both designer and generic and ultimately it comes down to how your project team agrees on the way things are supposed to be done. If you can't get consensus or you're a team of one, you can follow these helpful steps.

Introduction

  _      __         __    _____
 | | /| / /__  ____/ /__ / _/ /__ _    _____
 | |/ |/ / _ \/ __/  '_// _/ / _ \ |/|/ (_-<
 |__/|__/\___/_/_/_/\_\/_//_/\___/__,__/___/  __  _
 ___ ____  ___/ / / __/__ ___  ___ ________ _/ /_(_)__  ___  ___
/ _ `/ _ \/ _  / _\ \/ -_) _ \/ _ `/ __/ _ `/ __/ / _ \/ _ \(_-<
\_,_/_//_/\_,_/ /___/\__/ .__/\_,_/_/  \_,_/\__/_/\___/_//_/___/___  __________
 ___  / _/ _______  ___/_/______ _______  ___   (_)__    / ___/  _/_/_/ ___/ _ \
/ _ \/ _/ / __/ _ \/ _ \/ __/ -_) __/ _ \(_-<  / / _ \  / /___/ /_/_// /__/ // /
\___/_/   \__/\___/_//_/\__/\__/_/ /_//_/___/ /_/_//_/  \___/___/_/  \___/____/

We're going to be focusing on a couple of things like

  • CircleCI 2.0 Brief Overview
  • Jobs
    • Docker
    • Environment
    • Steps
      • Checking out code
      • Caching
      • Workspaces
      • Running arbitrary commands
  • Workflows
  • YAML Anchors

CircleCI 2.0 Brief Overview

  ______         __    _________                   _             ___   ___
 / ___(_)_______/ /__ / ___/  _/ _  _____ _______ (_)__  ___    |_  | / _ \
/ /__/ / __/ __/ / -_) /___/ /  | |/ / -_) __(_-</ / _ \/ _ \  / __/_/ // /
\___/_/_/  \__/_/\__/\___/___/  |___/\__/_/ /___/_/\___/_//_/ /____(_)___/

CircleCI 2.0 is the latest release of the CircleCI CI/CD Software as a Service. It includes quite a few updates from CircleCI 1.0.

  • First-class Docker Container Support (everything runs in a container)
  • Addition of Workflows and Workspaces
  • Addition of Scheduled triggers for Workflows

While keeping all of the other already great features.

  • SSH Access
  • Parallelism
  • Slack & GitHub integration

Read more about 2.0 features here. https://circleci.com/docs/2.0/

CircleCI v2.0 Jobs

 ██████╗██╗██████╗  ██████╗██╗     ███████╗ ██████╗██╗
██╔════╝██║██╔══██╗██╔════╝██║     ██╔════╝██╔════╝██║██╗
██║     ██║██████╔╝██║     ██║     █████╗  ██║     ██║╚═╝
██║     ██║██╔══██╗██║     ██║     ██╔══╝  ██║     ██║██╗
╚██████╗██║██║  ██║╚██████╗███████╗███████╗╚██████╗██║╚═╝
 ╚═════╝╚═╝╚═╝  ╚═╝ ╚═════╝╚══════╝╚══════╝ ╚═════╝╚═╝

     ██╗ ██████╗ ██████╗ ███████╗
     ██║██╔═══██╗██╔══██╗██╔════╝
     ██║██║   ██║██████╔╝███████╗
██   ██║██║   ██║██╔══██╗╚════██║
╚█████╔╝╚██████╔╝██████╔╝███████║
 ╚════╝  ╚═════╝ ╚═════╝ ╚══════╝

version: 2
jobs:
  job_name_1:
    docker:
    - image: circleci/python:latest-node-browsers
    environment:
      DEBUG: 'true'
    steps:
    - checkout:
        path: /app/
    - run:
        name: Create an artifact to cache
        command: |
          echo "cache me" >> artifacts_to_cache
    - save_cache:
        key: app_name-{{ checksum "some_file" }}-{{ checksum "some_other_file" }}
        paths: ['artifacts_to_cache']
    - persist_to_workspace:
        root: /app/
        paths: ['*']
    - run:
        name: A human-readable name
        command: |
          echo "Job Name 1 has run"

Jobs - Configuration

CircleCI v2.0 Jobs are comprised of configuration. There are a lot of different properties that can be added under the jobs: map, but I'm only going to cover a few of them today.

  • jobs.${JOB_NAME}.docker
  • jobs.${JOB_NAME}.environment
  • jobs.${JOB_NAME}.steps

For more information refer to the documentation.

Docker

     ██╗ ██████╗ ██████╗ ███████╗
     ██║██╔═══██╗██╔══██╗██╔════╝██╗
     ██║██║   ██║██████╔╝███████╗╚═╝
██   ██║██║   ██║██╔══██╗╚════██║██╗
╚█████╔╝╚██████╔╝██████╔╝███████║╚═╝
 ╚════╝  ╚═════╝ ╚═════╝ ╚══════╝

██████╗  ██████╗  ██████╗██╗  ██╗███████╗██████╗
██╔══██╗██╔═══██╗██╔════╝██║ ██╔╝██╔════╝██╔══██╗
██║  ██║██║   ██║██║     █████╔╝ █████╗  ██████╔╝
██║  ██║██║   ██║██║     ██╔═██╗ ██╔══╝  ██╔══██╗
██████╔╝╚██████╔╝╚██████╗██║  ██╗███████╗██║  ██║
╚═════╝  ╚═════╝  ╚═════╝╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝
  • jobs.${JOB_NAME}.docker
    • Required image property jobs.${JOB_NAME}.docker.image
    • Optional name property jobs.${JOB_NAME}.docker.name
    • Optional entrypoint property jobs.${JOB_NAME}.docker.entrypoint
    • Optional command property jobs.${JOB_NAME}.docker.command
    • Optional user property jobs.${JOB_NAME}.docker.user
    • Optional environment property jobs.${JOB_NAME}.docker.environment
    • Optional auth property jobs.${JOB_NAME}.docker.auth
    • Optional aws_auth property jobs.${JOB_NAME}.docker.aws_auth
    # ...
    docker:
    - image: circleci/python:latest-node-browsers
    # ...

For more information refer to the documentation and make sure you checkout the list of official CircleCI images

Environment

     ██╗ ██████╗ ██████╗ ███████╗
     ██║██╔═══██╗██╔══██╗██╔════╝██╗
     ██║██║   ██║██████╔╝███████╗╚═╝
██   ██║██║   ██║██╔══██╗╚════██║██╗
╚█████╔╝╚██████╔╝██████╔╝███████║╚═╝
 ╚════╝  ╚═════╝ ╚═════╝ ╚══════╝

███████╗███╗   ██╗██╗   ██╗██╗██████╗  ██████╗ ███╗   ██╗███╗   ███╗███████╗███╗   ██╗████████╗
██╔════╝████╗  ██║██║   ██║██║██╔══██╗██╔═══██╗████╗  ██║████╗ ████║██╔════╝████╗  ██║╚══██╔══╝
█████╗  ██╔██╗ ██║██║   ██║██║██████╔╝██║   ██║██╔██╗ ██║██╔████╔██║█████╗  ██╔██╗ ██║   ██║
██╔══╝  ██║╚██╗██║╚██╗ ██╔╝██║██╔══██╗██║   ██║██║╚██╗██║██║╚██╔╝██║██╔══╝  ██║╚██╗██║   ██║
███████╗██║ ╚████║ ╚████╔╝ ██║██║  ██║╚██████╔╝██║ ╚████║██║ ╚═╝ ██║███████╗██║ ╚████║   ██║
╚══════╝╚═╝  ╚═══╝  ╚═══╝  ╚═╝╚═╝  ╚═╝ ╚═════╝ ╚═╝  ╚═══╝╚═╝     ╚═╝╚══════╝╚═╝  ╚═══╝   ╚═╝

Set any environment variables you might need for a particular job. Note: this will override any environment variables that are set in the CircleCI web interface that match.

    # ...
    environment:
      DEBUG: 'true'
    # ...

For more information refer to the documentation

Jobs - Steps

     ██╗ ██████╗ ██████╗ ███████╗
     ██║██╔═══██╗██╔══██╗██╔════╝██╗
     ██║██║   ██║██████╔╝███████╗╚═╝
██   ██║██║   ██║██╔══██╗╚════██║██╗
╚█████╔╝╚██████╔╝██████╔╝███████║╚═╝
 ╚════╝  ╚═════╝ ╚═════╝ ╚══════╝

███████╗████████╗███████╗██████╗ ███████╗
██╔════╝╚══██╔══╝██╔════╝██╔══██╗██╔════╝
███████╗   ██║   █████╗  ██████╔╝███████╗
╚════██║   ██║   ██╔══╝  ██╔═══╝ ╚════██║
███████║   ██║   ███████╗██║     ███████║
╚══════╝   ╚═╝   ╚══════╝╚═╝     ╚══════╝

CircleCI v2.0 Jobs are comprised of Steps. There are a lot common steps that are leveraged throughout 18F projects. Let's go over them:

  • jobs.${JOB_NAME}.steps.checkout
  • jobs.${JOB_NAME}.steps.save_cache
  • jobs.${JOB_NAME}.steps.restore_cache
  • jobs.${JOB_NAME}.steps.persist_to_workspace
  • jobs.${JOB_NAME}.steps.attach_workspace
  • jobs.${JOB_NAME}.steps.run
Checking out code
███████╗████████╗███████╗██████╗ ███████╗
██╔════╝╚══██╔══╝██╔════╝██╔══██╗██╔════╝██╗
███████╗   ██║   █████╗  ██████╔╝███████╗╚═╝
╚════██║   ██║   ██╔══╝  ██╔═══╝ ╚════██║██╗
███████║   ██║   ███████╗██║     ███████║╚═╝
╚══════╝   ╚═╝   ╚══════╝╚═╝     ╚══════╝

 ██████╗██╗  ██╗███████╗ ██████╗██╗  ██╗ ██████╗ ██╗   ██╗████████╗
██╔════╝██║  ██║██╔════╝██╔════╝██║ ██╔╝██╔═══██╗██║   ██║╚══██╔══╝
██║     ███████║█████╗  ██║     █████╔╝ ██║   ██║██║   ██║   ██║
██║     ██╔══██║██╔══╝  ██║     ██╔═██╗ ██║   ██║██║   ██║   ██║
╚██████╗██║  ██║███████╗╚██████╗██║  ██╗╚██████╔╝╚██████╔╝   ██║
 ╚═════╝╚═╝  ╚═╝╚══════╝ ╚═════╝╚═╝  ╚═╝ ╚═════╝  ╚═════╝    ╚═╝
  • jobs.${JOB_NAME}.steps.checkout
    • Optional path property jobs.${JOB_NAME}.steps.checkout.path

The most common one is a Git checkout of your code. You can supply a path for this step as well. Note: this uses ssh to checkout your code.

    # ...
    steps:
    - checkout:
        path: /app/
    # ...

For more information refer to the documentation

Caching dependencies and artifacts between Builds
███████╗████████╗███████╗██████╗ ███████╗
██╔════╝╚══██╔══╝██╔════╝██╔══██╗██╔════╝██╗
███████╗   ██║   █████╗  ██████╔╝███████╗╚═╝
╚════██║   ██║   ██╔══╝  ██╔═══╝ ╚════██║██╗
███████║   ██║   ███████╗██║     ███████║╚═╝
╚══════╝   ╚═╝   ╚══════╝╚═╝     ╚══════╝

███████╗ █████╗ ██╗   ██╗███████╗     ██████╗ █████╗  ██████╗██╗  ██╗███████╗
██╔════╝██╔══██╗██║   ██║██╔════╝    ██╔════╝██╔══██╗██╔════╝██║  ██║██╔════╝
███████╗███████║██║   ██║█████╗      ██║     ███████║██║     ███████║█████╗
╚════██║██╔══██║╚██╗ ██╔╝██╔══╝      ██║     ██╔══██║██║     ██╔══██║██╔══╝
███████║██║  ██║ ╚████╔╝ ███████╗    ╚██████╗██║  ██║╚██████╗██║  ██║███████╗
╚══════╝╚═╝  ╚═╝  ╚═══╝  ╚══════╝     ╚═════╝╚═╝  ╚═╝ ╚═════╝╚═╝  ╚═╝╚══════╝

██████╗ ███████╗███████╗████████╗ ██████╗ ██████╗ ███████╗     ██████╗ █████╗  ██████╗██╗  ██╗███████╗
██╔══██╗██╔════╝██╔════╝╚══██╔══╝██╔═══██╗██╔══██╗██╔════╝    ██╔════╝██╔══██╗██╔════╝██║  ██║██╔════╝
██████╔╝█████╗  ███████╗   ██║   ██║   ██║██████╔╝█████╗      ██║     ███████║██║     ███████║█████╗
██╔══██╗██╔══╝  ╚════██║   ██║   ██║   ██║██╔══██╗██╔══╝      ██║     ██╔══██║██║     ██╔══██║██╔══╝
██║  ██║███████╗███████║   ██║   ╚██████╔╝██║  ██║███████╗    ╚██████╗██║  ██║╚██████╗██║  ██║███████╗
╚═╝  ╚═╝╚══════╝╚══════╝   ╚═╝    ╚═════╝ ╚═╝  ╚═╝╚══════╝     ╚═════╝╚═╝  ╚═╝ ╚═════╝╚═╝  ╚═╝╚══════╝

There are times when working with your project that you'd like to cache files either between jobs or builds. This will allow you to speed up your builds if they ever need to fetch anything from the web including your source code. This is super useful for dependencies for example.

  • jobs.${JOB_NAME}.steps.save_cache
    • Required key property jobs.${JOB_NAME}.steps.save_cache.key
    • Required paths property jobs.${JOB_NAME}.steps.save_cache.paths
  app_deps_job:
    # ...
    steps:
    # ...
    - run:
        name: Install and update depedencies
        command: |
          npm install
    # ...
    - save_cache:
        key: app_name-{{ checksum "package.json" }}-{{ checksum "npm-shrinkwrap.json" }}
        paths: ['node_modules']
    # ...
  • jobs.${JOB_NAME}.steps.restore_cache
    • Required key property or keys list jobs.${JOB_NAME}.steps.save_cache.key(s)
  app_test_job:
    # ...
    steps:
    # ...
    - restore_cache:
        keys:
          - app_name-{{ checksum "package.json" }}-{{ checksum "npm-shrinkwrap.json" }}
          - app_name- # will fetch the latest cache that matches this prefix, if it can't find the file above
    - run:
        name: Run tests
        command: |
          npm test
    # ...

For more examples refer to the documentation

Persisting files between Jobs
███████╗████████╗███████╗██████╗ ███████╗
██╔════╝╚══██╔══╝██╔════╝██╔══██╗██╔════╝██╗
███████╗   ██║   █████╗  ██████╔╝███████╗╚═╝
╚════██║   ██║   ██╔══╝  ██╔═══╝ ╚════██║██╗
███████║   ██║   ███████╗██║     ███████║╚═╝
╚══════╝   ╚═╝   ╚══════╝╚═╝     ╚══════╝

██████╗ ███████╗██████╗ ███████╗██╗███████╗████████╗    ████████╗ ██████╗     ██╗    ██╗ ██████╗ ██████╗ ██╗  ██╗███████╗██████╗  █████╗  ██████╗███████╗
██╔══██╗██╔════╝██╔══██╗██╔════╝██║██╔════╝╚══██╔══╝    ╚══██╔══╝██╔═══██╗    ██║    ██║██╔═══██╗██╔══██╗██║ ██╔╝██╔════╝██╔══██╗██╔══██╗██╔════╝██╔════╝
██████╔╝█████╗  ██████╔╝███████╗██║███████╗   ██║          ██║   ██║   ██║    ██║ █╗ ██║██║   ██║██████╔╝█████╔╝ ███████╗██████╔╝███████║██║     █████╗
██╔═══╝ ██╔══╝  ██╔══██╗╚════██║██║╚════██║   ██║          ██║   ██║   ██║    ██║███╗██║██║   ██║██╔══██╗██╔═██╗ ╚════██║██╔═══╝ ██╔══██║██║     ██╔══╝
██║     ███████╗██║  ██║███████║██║███████║   ██║          ██║   ╚██████╔╝    ╚███╔███╔╝╚██████╔╝██║  ██║██║  ██╗███████║██║     ██║  ██║╚██████╗███████╗
╚═╝     ╚══════╝╚═╝  ╚═╝╚══════╝╚═╝╚══════╝   ╚═╝          ╚═╝    ╚═════╝      ╚══╝╚══╝  ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝╚══════╝╚═╝     ╚═╝  ╚═╝ ╚═════╝╚══════╝

 █████╗ ████████╗████████╗ █████╗  ██████╗██╗  ██╗    ██╗    ██╗ ██████╗ ██████╗ ██╗  ██╗███████╗██████╗  █████╗  ██████╗███████╗
██╔══██╗╚══██╔══╝╚══██╔══╝██╔══██╗██╔════╝██║  ██║    ██║    ██║██╔═══██╗██╔══██╗██║ ██╔╝██╔════╝██╔══██╗██╔══██╗██╔════╝██╔════╝
███████║   ██║      ██║   ███████║██║     ███████║    ██║ █╗ ██║██║   ██║██████╔╝█████╔╝ ███████╗██████╔╝███████║██║     █████╗
██╔══██║   ██║      ██║   ██╔══██║██║     ██╔══██║    ██║███╗██║██║   ██║██╔══██╗██╔═██╗ ╚════██║██╔═══╝ ██╔══██║██║     ██╔══╝
██║  ██║   ██║      ██║   ██║  ██║╚██████╗██║  ██║    ╚███╔███╔╝╚██████╔╝██║  ██║██║  ██╗███████║██║     ██║  ██║╚██████╗███████╗
╚═╝  ╚═╝   ╚═╝      ╚═╝   ╚═╝  ╚═╝ ╚═════╝╚═╝  ╚═╝     ╚══╝╚══╝  ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝╚══════╝╚═╝     ╚═╝  ╚═╝ ╚═════╝╚══════╝

This is super useful when creating jobs that build your project and then passing the workspace to a job that deploys the project. A deployment job shouldn't be checking code out of your repository as that code probably still needs to go through a build step.

  • jobs.${JOB_NAME}.steps.persist_to_workspace
    • Required root property jobs.${JOB_NAME}.steps.persist_to_workspace.root
    • Required paths list jobs.${JOB_NAME}.steps.persist_to_workspace.paths
  app_build_job:
    # ...
    steps:
    # ...
    - checkout
    - run:
        name: Build project
        command: |
          npm build
    # ...
    - persist_to_workspace:
        root: .
        paths: ['*']
    # ...
  • jobs.${JOB_NAME}.steps.attach_workspace
    • Required at property jobs.${JOB_NAME}.steps.persist_to_workspace.at
  app_deploy_job:
    # ...
    steps:
    # ...
    - attach_workspace:
        at: .
    - run:
        name: Login to cloud.gov
        command: |
          cf login -u $CF_DEPLOYER_USER -p $CF_DEPLOYER_PASSWORD -o $CF_ORG -s $CF_SPACE
    - run:
        name: Deploy application
        command: |
          cf push -f manifest.yml
    # ...

For more information refer to the documentation

Security, workspaces, and incident response
███████╗███████╗ ██████╗██╗   ██╗██████╗ ██╗████████╗██╗   ██╗
██╔════╝██╔════╝██╔════╝██║   ██║██╔══██╗██║╚══██╔══╝╚██╗ ██╔╝
███████╗█████╗  ██║     ██║   ██║██████╔╝██║   ██║    ╚████╔╝
╚════██║██╔══╝  ██║     ██║   ██║██╔══██╗██║   ██║     ╚██╔╝
███████║███████╗╚██████╗╚██████╔╝██║  ██║██║   ██║      ██║
╚══════╝╚══════╝ ╚═════╝ ╚═════╝ ╚═╝  ╚═╝╚═╝   ╚═╝      ╚═╝

 █████╗ ██╗     ███████╗██████╗ ████████╗
██╔══██╗██║     ██╔════╝██╔══██╗╚══██╔══╝
███████║██║     █████╗  ██████╔╝   ██║
██╔══██║██║     ██╔══╝  ██╔══██╗   ██║
██║  ██║███████╗███████╗██║  ██║   ██║
╚═╝  ╚═╝╚══════╝╚══════╝╚═╝  ╚═╝   ╚═╝

Don’t copy files containing sensitive information to Workspaces as these get archived and uploaded to a CircleCI data-store. Make sure for these sort of jobs you are excluding files that may contain sensitive information so they aren't stored anywhere.

Generic Run Step
███████╗████████╗███████╗██████╗
██╔════╝╚══██╔══╝██╔════╝██╔══██╗██╗
███████╗   ██║   █████╗  ██████╔╝╚═╝
╚════██║   ██║   ██╔══╝  ██╔═══╝ ██╗
███████║   ██║   ███████╗██║     ╚═╝
╚══════╝   ╚═╝   ╚══════╝╚═╝

██████╗ ██╗   ██╗███╗   ██╗
██╔══██╗██║   ██║████╗  ██║
██████╔╝██║   ██║██╔██╗ ██║
██╔══██╗██║   ██║██║╚██╗██║
██║  ██║╚██████╔╝██║ ╚████║
╚═╝  ╚═╝ ╚═════╝ ╚═╝  ╚═══╝

The bread-and-butter of all CircleCI steps. This step allows you to run arbitrary commands that are specific to your project. You can leverage the expanded command property to run multiple commands in the same shell or you can use the short-form to run scripts that exist in your project.

  • jobs.${JOB_NAME}.steps.run
    • Can be used in short-form where the value will be used as both the name and command properties.
    • Required command property jobs.${JOB_NAME}.steps.run.command
    • Optional name property jobs.${JOB_NAME}.steps.run.name
    • Optional shell property jobs.${JOB_NAME}.steps.run.shell
    • Optional environment property jobs.${JOB_NAME}.steps.run.environment
    • Optional background property jobs.${JOB_NAME}.steps.run.background
    • Optional working_directory property jobs.${JOB_NAME}.steps.run.working_directory
    • Optional no_output_timeout property jobs.${JOB_NAME}.steps.run.no_output_timeout
    • Optional when property jobs.${JOB_NAME}.steps.run.when
    # ...
    - run:
        name: Create an artifact to cache
        command: |
          touch artifacts_to_cache
          echo "cache me" >> artifacts_to_cache
    # ...
    - run:
        name: Run a CI specific script
        command: |
          cd ci/
          ./exec_many_commands
    # ...

For more information refer to the documentation

CircleCI v2.0 Workflows

 ██████╗██╗██████╗  ██████╗██╗     ███████╗ ██████╗██╗
██╔════╝██║██╔══██╗██╔════╝██║     ██╔════╝██╔════╝██║██╗
██║     ██║██████╔╝██║     ██║     █████╗  ██║     ██║╚═╝
██║     ██║██╔══██╗██║     ██║     ██╔══╝  ██║     ██║██╗
╚██████╗██║██║  ██║╚██████╗███████╗███████╗╚██████╗██║╚═╝
 ╚═════╝╚═╝╚═╝  ╚═╝ ╚═════╝╚══════╝╚══════╝ ╚═════╝╚═╝

██╗    ██╗ ██████╗ ██████╗ ██╗  ██╗███████╗██╗      ██████╗ ██╗    ██╗███████╗
██║    ██║██╔═══██╗██╔══██╗██║ ██╔╝██╔════╝██║     ██╔═══██╗██║    ██║██╔════╝
██║ █╗ ██║██║   ██║██████╔╝█████╔╝ █████╗  ██║     ██║   ██║██║ █╗ ██║███████╗
██║███╗██║██║   ██║██╔══██╗██╔═██╗ ██╔══╝  ██║     ██║   ██║██║███╗██║╚════██║
╚███╔███╔╝╚██████╔╝██║  ██║██║  ██╗██║     ███████╗╚██████╔╝╚███╔███╔╝███████║
 ╚══╝╚══╝  ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝╚═╝     ╚══════╝ ╚═════╝  ╚══╝╚══╝ ╚══════╝

CircleCI v2.0 introduced Workflows which are really useful for trying together different jobs that you define in the jobs: section of your CircleCI configuration file.

If you have jobs that consist of multiple separate steps, you can have these jobs run in parallel during by having them fan-out and then fan-in for a final deployment step.

For more information refer to the blog post and the documentation.

workflows:
  version: 2
  workflow_name_1:
    jobs:
    - pre_job_name_1
    - job_name_1:
        requires: ['pre_job_name_1']
    - job_name_2:
        requires: ['pre_job_name_1']
    - job_name_3:
        requires:
        - pre_job_name_1
        - job_name_2
        filters:
          branches:
            only: ['master']
  workflow_name_2:
    jobs:
    - job_name_4
    - job_name_5:
        requires: ['pre_job_name_2']
    - job_name_6:
        requires:
        - pre_job_name_2
        - job_name_4
  schedule_workflow_name_1:
    jobs:
    - pre_job_name_1
    - job_name_1:
        requires: ['pre_job_name_1']
    - job_name_2:
        requires: ['pre_job_name_1']
    - job_name_3:
        requires:
        - pre_job_name_1
        - job_name_2
    triggers:
    - schedule:
        cron: "0 12 * * *"
        filters:
          branches:
            only: ['master']

YAML-soup and CircleCI v2.0's new API

I found a very big UX issue in CircleCI v2.0. It's not terrible, but it can be intimidating. Because of how powerful CircleCI v2.0 is, there's a really big chance that you're going to create configuration files that are over 500 lines of YAML for large projects.

It's important to leverage the power of the &ANCHOR syntax found in YAML to make sure you're not repeating yourself.

workflows:
  version: 2
  workflow_name_1: &DEFAULT_WORKFLOW
    jobs:
    - pre_job_name_1
    - job_name_1:
        requires: ['pre_job_name_1']
    - job_name_2:
        requires: ['pre_job_name_1']
    - job_name_3:
        requires:
        - pre_job_name_1
        - job_name_2
        filters:
          branches:
            only: ['master']
  workflow_name_2:
    jobs:
    - pre_job_name_2
    - job_name_4
    - job_name_5:
        requires: ['pre_job_name_2']
    - job_name_6:
        requires:
        - pre_job_name_2
        - job_name_4
  schedule_workflow_name_1:
    <<: *DEFAULT_WORKFLOW
    triggers:
    - schedule:
        cron: "0 12 * * *"
        filters:
          branches:
            only: ['master']

Thanks

To all the folks at 18F that worked on migrating us over to CircleCI as this talk wouldn't be relevant without them. Thanks to #g-devops for early feedback, and Adam Kendall for proposing this talk in the first place. And thanks to all of you to listen me yammer about CircleCI and YAML for a bit.

Inspired by the following configuration files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment