Skip to content

Instantly share code, notes, and snippets.

@sgnn7
Last active June 16, 2020 13:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sgnn7/6520f2bce5792ef6dd5302b50e6945ed to your computer and use it in GitHub Desktop.
Save sgnn7/6520f2bce5792ef6dd5302b50e6945ed to your computer and use it in GitHub Desktop.
Upgrading Conjur OSS Helm chart from `v1.3.8` to `v2.0.0`

Upgrade

Info here assumes you are in the base of https://github.com/cyberark/conjur-oss-helm-chart repo

Save config values

This assumes that only conjur is in the specified namespace. If not, manually set the helm_chart_name variable

ns="<REPLACE_YOUR_DEPLOYMENT_NAMESPACE>"

helm_chart_name=$(helm list --namespace $ns -q)
authenticators=$(kubectl --namespace=$ns get secret ${helm_chart_name}-conjur-authenticators -o jsonpath="{.data.key }" | base64 --decode)
datakey=$(kubectl --namespace=$ns get secret ${helm_chart_name}-conjur-data-key -o jsonpath="{.data.key }" | base64 --decode)

Save database to local machine

postgres_old_pod=$(kubectl --namespace $ns get pods -l "app=conjur-oss-postgres" -o jsonpath="{.items[0].metadata.name}")
kubectl exec -it --namespace $ns \
  $postgres_old_pod -- pg_dump -U postgres -c -C --column-inserts --inserts -f /dbdump.tar -F tar
kubectl cp --namespace $ns \
  $postgres_old_pod:dbdump.tar dbdump.tar

Uninstall old chart

WARNING: This will remove your old certificates!

WARNING: This will possibly change your external service IP!

helm uninstall --namespace $ns $helm_chart_name

Install new chart version from repo

This new deployment is unusable in this state as a regular deployment (which is intentional). The upgrade later will enable it. If using your own external database, set it here with --set.

ns="<REPLACE_YOUR_DEPLOYMENT_NAMESPACE>"
helm_chart_name=conjur-oss
helm install $helm_chart_name \
  --set dataKey="$datakey" \
  --set replicaCount=0 \
  --namespace $ns ./conjur-oss

Restore the database

We use the template1 part of the connection string to delete and recreate the database. This assumes that database names have not changed between upgrades. Replace postgres in the sed command if your connection string used a different database name.

postgres_new_pod=$(kubectl --namespace $ns get pods -l "app=conjur-oss-postgres" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace $ns cp ./dbdump.tar $postgres_new_pod:/dbdump.tar
pg_restore_connection_string=$(kubectl --namespace=$ns get secret ${helm_chart_name}-conjur-database-url -o jsonpath="{.data.key}" | base64 --decode | sed 's/postgres?/template1?/')
kubectl exec -it --namespace $ns \
  $postgres_new_pod -- pg_restore -C -c -d "$pg_restore_connection_string" /dbdump.tar 
kubectl exec -it --namespace $ns \
  $postgres_new_pod -- rm -rf /dbdump.tar

Redeploy helm chart with updated replicaCount

Note: If using your own external database, please ensure that you add that variable here too and any other relevant settings.

helm upgrade --namespace=$ns \
  $helm_chart_name \
  --reuse-values \
  --set replicaCount="1" \
  --set authenticators="$authenticators" \
  --set reuseDataKey=true \
  ./conjur-oss
@diverdane
Copy link

LOOKS AWESOME!!!
A couple of minor suggestions:
(1) For Save database to local machine, can replace:

postgres_old_pod=$(kubectl --namespace $ns get pods | grep postgres | awk '{print $1}')

with:

postgres_old_pod=$(kubectl --namespace $ns get pods -l "app=conjur-oss-postgres" -o jsonpath="{.items[0].metadata.name}")

(2) Same for postgres_new_pod in Restore the database
(3) Can we move Uninstall old chart to last, and have them deploy new cluster in a separate namespace?

@diverdane
Copy link

(4) The 'kubectl cp ...command warns about leading/`, so can replace:

kubectl cp --namespace $ns \
  $postgres_old_pod:/dbdump.tar dbdump.tar

with:

kubectl cp --namespace $ns \
  $postgres_old_pod:dbdump.tar dbdump.tar

@izgeri
Copy link

izgeri commented Jun 12, 2020

can we be clear that "Install new chart version from repo" should specify the same conjur version as the original deploy? if we don't list this requirement, and the new version has a different DB schema, will this process still work?

@sgnn7
Copy link
Author

sgnn7 commented Jun 16, 2020

@diverdane: I fixed up #1, #2, and #4 but since #3 requires more time on this than I have right now, I will leave that for later

@izgeri: This should in theory work for newer versions too since the DB migration is properly run on conjur start. However if the chart changes heavily we would have problems with these rather-fragile steps so this should only be used for this specific helm chart version switch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment