Rancher/RKE etcd snapshot intro

rancher-rke-etcd-snapshot-intro.md

Rancher/RKE etcd snapshot intro

Brief description of what is done where/how/what etc

Repositories/tools involved:

• https://github.com/rancher/rke-tools The repository containing the the source for the rke-tools image which is used for all snapshot operations. See https://github.com/rancher/rke-tools/blob/master/main.go#L36 for available flags.
• https://github.com/rancher/rke RKE has an etcd subcommand to run operations on etcd. See https://github.com/rancher/rke/blob/master/cmd/etcd.go for options
• https://github.com/rancher/rancher Rancher manages etcd snapshots in its own fashion, as we have the option to store snapshot data in Rancher vs having no state stored in RKE.
• https://github.com/rancher/kontainer-engine Kontainer-engine is the tool to provision k8s clusters like GKE/AKS/EKS, but it also contains an RKE driver which takes care of the cluster calls regarding RKE/custom clusters in Rancher.

Rancher

In Rancher, "everything" is done using controllers. The etcdbackup controller takes care of the snapshots, see pkg/controllers/management/etcdbackup/etcdbackup.go. Functions in here link to function of the kontainer-engine rke driver (https://github.com/rancher/kontainer-engine/blob/master/drivers/rke/rke_driver.go) which then usually calls a cmd function in RKE.

Example of taking a snapshot:

• Create in pkg/controllers/management/etcdbackup/etcdbackup.go
• calls c.etcdSaveWithBackoff
• calls c.backupDriver.ETCDSave
• ETCDSave is defined in the kontainer driver in kontainer-engine, which then calls the function defined in the driver that is used (in this case, RKE)
• In drivers/rke_driver.go the function ETCDSave is defined, which then calls cmd.SnapshotSaveEtcdHosts from rke/cmd

The config for the cluster (and thus for the backups/snapshots) is stored and can be shown using Edit as YAML or View in API for the cluster. These values are used when snapshots are taken/cleaned.

Main differences with RKE

In RKE, there is no state to be stored, so we run etcd-rolling-backups on the nodes to make sure periodic snapshots are made. In Rancher, this is handled by the controller. For one-time snapshots, RKE CLI can be used for RKE, in Rancher, it's an API call which then follows the same trail as mentioned above. Keep in mind that rke-tools is still the link in all operations, as this is used to any operation for etcd.