Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save jostmart/2da4ced110f004960e4a91ebffdb3c1a to your computer and use it in GitHub Desktop.
Save jostmart/2da4ced110f004960e4a91ebffdb3c1a to your computer and use it in GitHub Desktop.
Restore Rancher 2 cluster/node agents on clusters

Restore Rancher 2 cluster/node agents on clusters

This is an unsupported scenario, see rancher/rancher#14731 when there is an official solution.

When cattle-cluster-agent and/or cattle-node-agent are accidentally deleted, or when server-url/cacerts are changed.

Generate definitions

  • Generate API token in the UI (user -> API & Keys) and save the Bearer token
  • Find the clusterid in the Rancher UI (format is c-xxxxx), its in the address bar when the cluster is selected
  • Generate agent definitions (needs curl, jq)
# Rancher URL
RANCHERURL="https://rancher.mydomain.com"
# Cluster ID
CLUSTERID="c-xxxxx"
# Token
TOKEN="token-xxxxx:xxxxx"
# Valid certificates
curl -s -H "Authorization: Bearer ${TOKEN}" "${RANCHERURL}/v3/clusterregistrationtokens?clusterId=${CLUSTERID}" | jq -r '.data[] | select(.name != "system") | .command'
# Self signed certificates
curl -s -k -H "Authorization: Bearer ${TOKEN}" "${RANCHERURL}/v3/clusterregistrationtokens?clusterId=${CLUSTERID}" | jq -r '.data[] | select(.name != "system") | .insecureCommand'

Apply definitions

The generated command needs to be executed using kubectl configured with a kubeconfig to talk to the cluster. See the gists below to retrieve the kubeconfig:

  1. Generate kubeconfig on node with controlplane role
docker run --rm --net=host -v $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl:/etc/kubernetes/ssl:ro --entrypoint bash $(docker inspect $(docker images -q --filter=label=io.cattle.agent=true) --format='{{index .RepoTags 0}}' | tail -1) -c 'kubectl --kubeconfig /etc/kubernetes/ssl/kubecfg-kube-node.yaml get configmap -n kube-system full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .currentState.certificatesBundle.\"kube-admin\".config | sed -e "/^[[:space:]]*server:/ s_:.*_: \"https://127.0.0.1:6443\"_"' > kubeconfig_admin.yaml
  1. Apply definitions (replace with the command returned from generating the definitions)
docker run --rm --net=host -v $PWD/kubeconfig_admin.yaml:/root/.kube/config --entrypoint bash $(docker inspect $(docker images -q --filter=label=io.cattle.agent=true) --format='{{index .RepoTags 0}}' | tail -1) -c 'curl --insecure -sfL https://xxx/v3/import/dl75kfmmbp9vj876cfsrlvsb9x9grqhqjd44zvnfd9qbh6r7ks97sr.yaml | kubectl apply -f -'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment