Skip to content

Instantly share code, notes, and snippets.

@vfarcic
Last active November 9, 2023 17:40
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save vfarcic/eabe08e8e147fb2ce51afc520efc0cef to your computer and use it in GitHub Desktop.
Save vfarcic/eabe08e8e147fb2ce51afc520efc0cef to your computer and use it in GitHub Desktop.
# Source: https://gist.github.com/eabe08e8e147fb2ce51afc520efc0cef
##########################################################################
# GitOps Across A Large Number Of Kubernetes Clusters With Rancher Fleet #
# https://youtu.be/JcOTHFDgMXM #
##########################################################################
# Additional Info:
# - Rancher Fleet: https://fleet.rancher.io/
# - What Is GitOps And Why Do We Want It?: https://youtu.be/qwyRJlmG5ew
# - Argo CD - Applying GitOps Principles To Manage Production Environment In Kubernetes: https://youtu.be/vpWQeoaiRM4
# - Flux CD v2 With GitOps Toolkit - Kubernetes Deployment And Sync Mechanism: https://youtu.be/R6OeIgb7lUI
# - Kustomize - How to Simplify Kubernetes Configuration Management: https://youtu.be/Twtbg6LFnAg
# - GitHub CLI - How to manage repositories more efficiently: https://youtu.be/BII6ZY2Rnlc
#########
# Setup #
#########
# Create a management, a dev, and 2 production clusters
# Create kubeconfig files for each of those clusters and store the absolute paths to those files into the following environment variables:
# - KUBECONFIG_MAN
# - KUBECONFIG_DEV
# - KUBECONFIG_PROD01
# - KUBECONFIG_PROD02
# Watch https://youtu.be/BII6ZY2Rnlc if you are not familiar with GitHub CLI
gh repo fork \
https://github.com/vfarcic/rancher-fleet-demo \
--clone
cd rancher-fleet-demo
kubectl --kubeconfig $KUBECONFIG_MAN \
create namespace production
# Replace `[...]` with the GitHub organization or user
export GITHUB_ORG=[...]
export REPO_URL=https://github.com/$GITHUB_ORG/rancher-fleet-demo
cat repo-kustomize.yaml \
| sed -e "s@repo: .*@repo: $REPO_URL@g" \
| tee repo-kustomize.yaml
cat repo-base.yaml \
| sed -e "s@repo: .*@repo: $REPO_URL@g" \
| tee repo-base.yaml
####################################
# Install Rancher Fleet Controller #
####################################
export MANAGEMENT_CLUSTER=$(kubectl \
--kubeconfig $KUBECONFIG_MAN \
config current-context)
kubectl --kubeconfig $KUBECONFIG_MAN \
config view \
--raw --minify --flatten \
--output jsonpath="{.clusters[?(@.name == \"$MANAGEMENT_CLUSTER\")].cluster.certificate-authority-data}" \
| base64 -d \
| tee ca.pem
export SERVER=$(kubectl \
--kubeconfig $KUBECONFIG_MAN \
config view \
--raw --minify --flatten \
--output jsonpath="{.clusters[?(@.name == \"$MANAGEMENT_CLUSTER\")].cluster.server}")
curl -fL --cacert ca.pem $SERVER/version
helm upgrade --install fleet-crd \
https://github.com/rancher/fleet/releases/download/v0.3.7/fleet-crd-0.3.7.tgz \
--kubeconfig $KUBECONFIG_MAN \
--namespace cattle-fleet-system \
--create-namespace \
--wait
helm upgrade --install fleet \
https://github.com/rancher/fleet/releases/download/v0.3.7/fleet-0.3.7.tgz \
--kubeconfig $KUBECONFIG_MAN \
--namespace cattle-fleet-system \
--create-namespace \
--set apiServerURL=$SERVER \
--set-file apiServerCA=ca.pem \
--wait
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace fleet-system \
get pods
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace fleet-system \
logs --selector app=fleet-controller
kubectl \
--kubeconfig $KUBECONFIG_MAN \
create namespace clusters
cat token.yaml
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace clusters \
apply --filename token.yaml
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace clusters \
get secret token \
--output jsonpath="{.data.values}" \
| base64 --decode \
| tee values.yaml
################################
# Install Rancher Fleet Agents #
################################
helm upgrade --install \
fleet-agent https://github.com/rancher/fleet/releases/download/v0.3.7/fleet-agent-0.3.7.tgz \
--namespace fleet-system \
--kubeconfig $KUBECONFIG_DEV \
--create-namespace \
--set-string labels.provider=civo \
--set-string labels.cluster=ck \
--set-string labels.env=dev \
--values values.yaml \
--wait
kubectl \
--kubeconfig $KUBECONFIG_DEV \
--namespace fleet-system \
get pods
kubectl \
--kubeconfig $KUBECONFIG_DEV \
--namespace fleet-system \
logs --selector app=fleet-agent
helm upgrade --install \
fleet-agent https://github.com/rancher/fleet/releases/download/v0.3.7/fleet-agent-0.3.7.tgz \
--namespace fleet-system \
--kubeconfig $KUBECONFIG_PROD01 \
--create-namespace \
--set-string labels.provider=azure \
--set-string labels.cluster=aks \
--set-string labels.env=production \
--values values.yaml \
--wait
helm upgrade --install \
fleet-agent https://github.com/rancher/fleet/releases/download/v0.3.7/fleet-agent-0.3.7.tgz \
--namespace fleet-system \
--kubeconfig $KUBECONFIG_PROD02 \
--create-namespace \
--set-string labels.provider=aws \
--set-string labels.cluster=eks \
--set-string labels.env=production \
--values values.yaml \
--wait
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace clusters \
get clusters.fleet.cattle.io
cat cluster-group.yaml
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace clusters \
apply --filename cluster-group.yaml
#############################################
# GitOps In A Local Cluster Using Kustomize #
#############################################
ls -1 kustomize/
ls -1 kustomize/base
ls -1 kustomize/overlays/production
cat repo-kustomize.yaml
kubectl \
--kubeconfig $KUBECONFIG_MAN \
apply --filename repo-kustomize.yaml
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace fleet-local \
get fleet
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace fleet-local \
describe gitrepo devops-toolkit
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace production \
get all
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace fleet-system logs \
--selector app=fleet-controller
kubectl \
--kubeconfig $KUBECONFIG_MAN \
delete --filename repo-kustomize.yaml
########################################
# GitOps In A Local Cluster Using Helm #
########################################
cat repo-helm.yaml
kubectl \
--kubeconfig $KUBECONFIG_MAN \
apply \
--filename repo-helm.yaml
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace fleet-local \
get fleet
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace production \
get all
#################################################
# GitOps In A Remote Cluster With Rancher Fleet #
#################################################
cat repo.yaml
kubectl --kubeconfig $KUBECONFIG_MAN \
apply \
--filename repo.yaml
kubectl \
--kubeconfig $KUBECONFIG_MAN \
--namespace clusters \
get fleet
kubectl --kubeconfig $KUBECONFIG_DEV \
get all
######################################
# Drift-detection With Rancher Fleet #
######################################
kubectl --kubeconfig $KUBECONFIG_DEV \
edit deployment \
devops-toolkit-helm-devops-toolkit
# Change `spec.template.spec.containers[0].image` to `vfarcic/devops-toolkit-series:4.0.0`
# Save the changes (e.g., `:wq`)
kubectl --kubeconfig $KUBECONFIG_DEV \
describe deployment \
devops-toolkit-helm-devops-toolkit
kubectl --kubeconfig $KUBECONFIG_DEV \
describe deployment \
devops-toolkit-helm-devops-toolkit
###########################################
# Multi-Cluster GitOps With Rancher Fleet #
###########################################
cat repo.yaml
kubectl --kubeconfig $KUBECONFIG_PROD01 \
get all
kubectl --kubeconfig $KUBECONFIG_PROD02 \
get all
cat helm/fleet.yaml
# Open `helm/fleet.yaml` and change the `production` target value of `helm.values.image.tag` to `4.0.0`
git add .
git commit -m "4.0.0"
git push
kubectl --kubeconfig $KUBECONFIG_PROD01 \
describe deployment devops-toolkit-helm-devops-toolkit
kubectl --kubeconfig $KUBECONFIG_PROD02 \
describe deployment devops-toolkit-helm-devops-toolkit
###########
# Destroy #
###########
cat helm/fleet.yaml \
| sed -e "s@tag: 4.*@tag: 2.7.0@g" \
| tee helm/fleet.yaml
git add .
git commit -m "Revert"
git push
# Destroy the clusters
@srk112255
Copy link

Getting error while creating fleet-crd and fleet due to namespace issue, after mentioning cattle-fleet-system it works fine :

helm upgrade --install fleet-crd
https://github.com/rancher/fleet/releases/download/v0.3.7/fleet-crd-0.3.7.tgz
--kubeconfig $KUBECONFIG_MAN
--namespace cattle-fleet-system
--create-namespace
--wait

helm upgrade --install fleet
https://github.com/rancher/fleet/releases/download/v0.3.7/fleet-0.3.7.tgz
--kubeconfig $KUBECONFIG_MAN
--namespace cattle-fleet-system
--create-namespace
--set apiServerURL=$SERVER
--set-file apiServerCA=ca.pem
--wait

@vfarcic
Copy link
Author

vfarcic commented Jul 17, 2023

I haven't used Rancher Fleet in a while and they probably changed something in the meantime. Thanks for the fix @srk112255. I applied it to the gist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment