Recover cluster.rkestate file from controlplane node

restore-rkestate-file.md

Recover cluster.rkestate file from controlplane node

RKE

Run on controlplane node, uses any found hyperkube image

k8s 1.19 and higher

docker run --rm --net=host -v $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl:/etc/kubernetes/ssl:ro --entrypoint bash $(docker inspect $(docker images -q --filter=label=org.opencontainers.image.source=https://github.com/rancher/hyperkube.git) --format='{{index .RepoTags 0}}' | tail -1) -c 'kubectl --kubeconfig /etc/kubernetes/ssl/kubecfg-kube-node.yaml -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .' > cluster.rkestate

k8s 1.18 and lower

docker run --rm --net=host -v $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl:/etc/kubernetes/ssl:ro --entrypoint bash $(docker inspect $(docker images -q --filter=label=org.label-schema.vcs-url=https://github.com/rancher/hyperkube.git) --format='{{index .RepoTags 0}}' | tail -1) -c 'kubectl --kubeconfig /etc/kubernetes/ssl/kubecfg-kube-node.yaml -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .' > cluster.rkestate

Rancher v2.2.x

Run on controlplane node, uses rancher/rancher-agent image

docker run --rm --net=host -v $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl:/etc/kubernetes/ssl:ro --entrypoint bash $(docker inspect $(docker images -q --filter=label=io.cattle.agent=true) --format='{{index .RepoTags 0}}' | tail -1) -c 'kubectl --kubeconfig /etc/kubernetes/ssl/kubecfg-kube-node.yaml get configmap -n kube-system full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .' > cluster.rkestate

Thanks @superseb, it saved my day.

@superseb do you think we could add this to the support-tools repository?

@oxr463 Yeah we can, it is in progress of being added to RKE CLI here rancher/rke#2258 but it's not in any released version yet.

@superseb.... You really saved my day.

Hello everybody, I know I'm a bit late on this thread, but... I'm unable to retrieve worker nodes' informations, this way.
I managed to loose both cluster.yml and cluster.rkestate ...

Hello everybody, I know I'm a bit late on this thread, but... I'm unable to retrieve worker nodes' informations, this way.
I managed to loose both cluster.yml and cluster.rkestate ...

I don't think there is anything to recover the cluster.yml file.

Hello everybody, I know I'm a bit late on this thread, but... I'm unable to retrieve worker nodes' informations, this way.
I managed to loose both cluster.yml and cluster.rkestate ...

I don't think there is anything to recover the cluster.yml file.

kubectl get configmap -n kube-system full-cluster-state -o "jsonpath={.data.full-cluster-state}" | python -c 'import json, yaml, sys; yaml.safe_dump(json.load(sys.stdin).get("currentState", []).get("rkeConfig",[]), sys.stdout)'

Hello everybody, I know I'm a bit late on this thread, but... I'm unable to retrieve worker nodes' informations, this way.
I managed to loose both cluster.yml and cluster.rkestate ...

I don't think there is anything to recover the cluster.yml file.

kubectl get configmap -n kube-system full-cluster-state -o "jsonpath={.data.full-cluster-state}" | python -c 'import json, yaml, sys; yaml.safe_dump(json.load(sys.stdin).get("currentState", []).get("rkeConfig",[]), sys.stdout)'

Does that work for non rancher-managed clusters? Like say a local cluster provisioned with RKE?

Does that work for non rancher-managed clusters? Like say a local cluster provisioned with RKE?

Thats what I tested it with yep, whatever runs rke up should have it, not tried a downstream cluster at all.

Hello, more-than-late on this thread, I'd like to contribute "my 2 cents" (I'm, in fact, using it on a downstream cluster).

# use the desired context (kubie is a great tool!)
kubie ctx mycontext
# get kubeconfig file (this one has only *one* context inside)
kubectl config view --flatten > kube_config_cluster.yml
# get cluster.yml (this one references *master nodes only*)
kubectl get configmap -n kube-system full-cluster-state -o "jsonpath={.data.full-cluster-state}" | python3 -c 'import json, yaml, sys; yaml.safe_dump(json.load(sys.stdin).get("currentState", []).get("rkeConfig",[]), sys.stdout)' > cluster.yml
# get rkestate
rke util get-state-file
# backup etcd
rke etcd snapshot-save --name 20220323-mycluster

I'm using kubectl config view --flatten initially to obtain the kubeconfig file "thru Rancher server", while using rke util get-kubeconfig afterwards will obtain one that bypasses Rancher (direct cluster access).

this was excellent, thank you!