Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

Abstract CI/CD Pipeline Design

This is an abstract CI/CD pipeline design that optimizes for blue/green deployments, repeatable builds and bakes in the ability to rapidly deploy a hotfix in emergencies.

Parameters

Parameter Default Description
version HEAD Git tag or commit SHA to be built and deployed.
target stage Enumeration: { dev | stage | prod }
skip_slow_scans false Enable builds to optionally complete faster by skipping some slow scans.

Stages

  • Prepare Workspace
    • Clean workspace
    • Check out $version
  • Unit Tests
  • Build
  • Push artifacts to Nexus/S3
  • Deploy (initial)
    • Pull artifact from Nexus/S3
    • Push to $target with version-suffixed domain/route
  • Integration Tests
    • If $skip_slow_scans, skip X, Y and/or Z
  • Deploy (cutover)
    • Point unsuffixed domain/route to newly deployed instance
    • Terminate previous instance

Deployment Infrastructure

Build Artifacts

Build artifacts would ideally be pushed to some central repository. This example just uses some S3 bucket, e.g.:

myproj-build-artifacts/<component_name>/myproj-<component_name>-<version>-<commit_sha>.tar.gz

Domains/Routing

To accomplish the blue/green deployment, each project component requires at least two DNS entries:

1. Fully-qualified application hostname

<component_name>-<commit_sha>.<target>.myproj.somehost.com

This entry enables us to run integration tests on the incoming instance to make sure it actually works before tearing down the previous instance.

The CI server updates this DNS record during each build.

2. Short "alias" to the fully-qualified hostname

<component_name>.<target>.myproj.somehost.com

This entry is effectively a pointer to the latest deployed instance of a component (identified by its commit SHA-suffixed hostname), allowing for "zero downtime" deployments (in theory). It will always point to a running instance (either the previous instance just before it gets torn down) or the incoming instance (once the alias record gets updated to point to it).

The CI server updates this DNS record during each build if and only if the integration tests for the incoming instance all pass.

Releases

Given some project myproj that is to be deployed directly onto one or more EC2 instances:

Repository/Pipeline Name Params Description
myproj-infrastructure
  • $target

Creates a deployment target environment (e.g., dev, qa, stage, prod) and provisions all of the raw resources needed to run the system (e.g., VPC, Route53, S3, EC2, RDS, etc)

All infrastructure should be tagged for push-button wholesale teardown of everything inside the deployment target environment.

myproj-api
  • $target
  • $version
myproj-ui
  • $target
  • $version
myproj-microservice-a
  • $target
  • $version
myproj-microservice-b
  • $target
  • $version
myproj-release
  • $target
  • $api_version
  • $ui_version
  • $msa_version
  • $msb_version

Triggers and waits for each of the above builds and optionally flip some arbitrary switch at the end of it all (maybe set and push some git tags to the repos?).

Given the same set of parameters, the build system should be capable of deploying an exact replica each .

@dbazile

This comment has been minimized.

Copy link
Owner Author

dbazile commented Dec 12, 2017

Issues raised with this model:

  • Needs "escape hatch" verbiage for when we need to diverge from the model
  • "What if you have 2 apps that are B/Ging simultaneously - how do you test and which do you use?"
@dbazile

This comment has been minimized.

Copy link
Owner Author

dbazile commented Dec 12, 2017

More issues raised:

  • Need to expand on secrets management
  • Need to expand on B/G deployment
@dbazile

This comment has been minimized.

Copy link
Owner Author

dbazile commented Dec 12, 2017

Quote from Patrick on "sciency" deps:

the only way ive seen people do this right is to vendor it themselves and own it as a first class citizen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.