pipeline-cd-design

pipeline-cd-design.md

Pipeline CD Design

1. CD on K8S using existing tools

Drone

There're some Drone plugins for integration with K8S:

Drone-Kubernetes: upgrade a Kubernetes deployment with a newer version of an image.

Do as kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1

Drone-Kube: apply a Kubernetes manifest file.

Do as kubectl apply -f deployment.yaml

Helm: install or upgrade a helm release.

Do as helm upgrade --install RELEASE CHART

Jenkins

Kubernetes CD Plugins: Similar to kubectl apply

ElasticBox Jenkins Kubernetes CI/CD Plug-in: deploy & delete Helm charts

To sum up:

they provide support for kubectl set image, kubectl apply, helm upgrade/delete

2. Features of Pipeline CD part

We can provide the following functionalities by adding more step types to do deployment on K8S:

1. Upgrade workload images (similar to kubectl set image)
2. Apply(create or upgrade) workloads
3. Apply(create or upgrade) loadBalancers (similar to kubectl apply)
4. Upgrade Helm/compose catalog( Hold on now untill related part is implemented)

For consistent user experience, other than reading some yaml config files from the repo( it can be an advanced option though), in step settings we use the same UI as workload/catalog deploy/edit pages.

Prototype images are shown in the following link:

https://4kmzpd.axshare.com/#g=1&p=new-pipeline , password: rancher@123

Upgrade workload images

1. Use selectors to choose workloads to upgrade
2. Specify image to upgrade

Apply workloads

1. Similar UI of Deploy Workload page
2. Can choose an existing workload as the template or start from scratch.

Apply ingress

1. Similar UI of Add Ingress page
2. Can choose an existing ingress as the template or start from scratch.

Apply Helm Catalog

1. similar UI of Launch App page
2. Can choose an existing Helm release as the template or start from scratch.

3. Access Control

1. A pipeline can do deployment to its own project namespaces.
2. What resources can be deployed should be in accord with the pipeline creator's project permissions. For example, a pipeline can upgrade workload images or workloads if the creator has Manage Workload Project permission.

4. Questions

1. Do we need to provide support to split CI and CD workflow?

   Say, team A setup a CI pipeline , team B setup a CD pipeline that is not triggered by github webhook(some generic webhooks maybe). For now, everything starts from the source code repo.

2. Need to investigate more to see how the same set of agents respects for different permissions of the pipelines.

question number 4.1 I would say, yes at some point. It would be nice if a docker hub image push or something could trigger a deployment. I wonder though if thats more of just a webhook type that watches a deployment. Similar to 1.6

we can just develop webhooks again as a feature. That can be used for CD as well.

Other questions:

1. Do we want to support cluster level pipelines? If so, which project does it get created in?
2. If not, can project level pipelines deploy to other projects or only same project. I prefer same to keep things simple.
3. If cluster-wide projects are supported, I assume it can be used to deploy to any projects assuming right permissions?
4. What's the difference between Apply Workload and Upgrade Workload? Only the update of images?
5. Isn't Update to Helm more CI?
6. Do we need Apply Ingress? Or what is that for?
7. What is the the deployment stage called? It can't be "Run a script".
8. When you upgrade a workload, shouldn't they be able to set some upgrade policy (rolling-upgrade vs blue/green vs others?)