dbazile/abstract pipeline.md

## abstract pipeline.md

      
    Raw
  

              abstract pipeline.md
            
          
    Abstract CI/CD Pipeline Design

This is an abstract CI/CD pipeline design that optimizes for
blue/green deployments,
repeatable builds and
bakes in the ability to rapidly deploy a hotfix in emergencies.
Parameters


Parameter
Default
Description


version
HEAD
Git tag or commit SHA to be built and deployed.


target
stage
Enumeration: { dev | stage | prod }


skip_slow_scans
false
Enable builds to optionally complete faster by skipping some slow scans.


Stages


Prepare Workspace

Clean workspace
Check out $version


Unit Tests
Build
Push artifacts to Nexus/S3
Deploy (initial)

Pull artifact from Nexus/S3
Push to $target with version-suffixed domain/route


Integration Tests

If $skip_slow_scans, skip X, Y and/or Z


Deploy (cutover)

Point unsuffixed domain/route to newly deployed instance
Terminate previous instance


## deployment infrastructure.md

      
    Raw
  

              deployment infrastructure.md
            
          
    Deployment Infrastructure

Build Artifacts

Build artifacts would ideally be pushed to some central repository.  This example just uses some S3 bucket, e.g.:
myproj-build-artifacts/<component_name>/myproj-<component_name>-<version>-<commit_sha>.tar.gz

Domains/Routing

To accomplish the blue/green deployment, each project component requires
at least two DNS entries:
1. Fully-qualified application hostname

<component_name>-<commit_sha>.<target>.myproj.somehost.com

This entry enables us to run integration tests on the incoming instance
to make sure it actually works before tearing down the previous instance.
The CI server updates this DNS record during each build.
2. Short "alias" to the fully-qualified hostname

<component_name>.<target>.myproj.somehost.com

This entry is effectively a pointer to the latest deployed instance of a
component (identified by its commit SHA-suffixed hostname), allowing for
"zero downtime" deployments (in theory).  It will always point to a
running instance (either the previous instance just before it gets torn down)
or the incoming instance (once the alias record gets updated to point to it).
The CI server updates this DNS record during each build if and only if
the integration tests for the incoming instance all pass.

  
## releases.md

      
    Raw
  

              releases.md
            
          
    Releases

Given some project myproj that is to be deployed directly onto one or more EC2 instances:


Repository/Pipeline Name
Params
Description


myproj-infrastructure
$target
Creates a deployment target environment (e.g., dev, qa, stage, prod) and provisions all of the raw resources needed to run the system (e.g., VPC, Route53, S3, EC2, RDS, etc)
All infrastructure should be tagged for push-button wholesale teardown of everything inside the deployment target environment.


myproj-api
$target
$version


myproj-ui
$target
$version


myproj-microservice-a
$target
$version


myproj-microservice-b
$target
$version


myproj-release
$target
$api_version
$ui_version
$msa_version
$msb_version
Triggers and waits for each of the above builds and optionally flip some arbitrary switch at the end of it all (maybe set and push some git tags to the repos?).
Given the same set of parameters, the build system should be capable of deploying an exact replica each .
Parameter	Default	Description
`version`	`HEAD`	Git tag or commit SHA to be built and deployed.
`target`	`stage`	Enumeration: `{ dev \| stage \| prod }`
`skip_slow_scans`	`false`	Enable builds to optionally complete faster by skipping some slow scans.
Repository/Pipeline Name	Params	Description
`myproj-infrastructure`	`$target`	Creates a deployment target environment (e.g., `dev`, `qa`, `stage`, `prod`) and provisions all of the raw resources needed to run the system (e.g., VPC, Route53, S3, EC2, RDS, etc) All infrastructure should be tagged for push-button wholesale teardown of everything inside the deployment target environment.
`myproj-api`	`$target` `$version`
`myproj-ui`	`$target` `$version`
`myproj-microservice-a`	`$target` `$version`
`myproj-microservice-b`	`$target` `$version`
`myproj-release`	`$target` `$api_version` `$ui_version` `$msa_version` `$msb_version`	Triggers and waits for each of the above builds and optionally flip some arbitrary switch at the end of it all (maybe set and push some git tags to the repos?). Given the same set of parameters, the build system should be capable of deploying an exact replica each .