RKE1 to RKE2 Windows Migration Guidance
- 1. Background
- 2. Guiding Migrations of Workloads to RKE2 Windows
RKE1 Windows Workload scheduling is based on taints and tolerations.
Every Linux node in an RKE1 Windows cluster, regardless of the role assigned to it, will have have a default taint that prevents workloads to be scheduled on it unless the workload has a toleration configured. This is a major design feature for RKE1 Windows clusters which were designed to only run Windows workloads.
Default RKE1 Linux Node NoSchedule Taint
apiVersion: v1
kind: Node
spec:
...
taints:
- effect: NoSchedule
key: cattle.io/os
value: linuxRKE1 Linux NoSchedule Toleration for Workloads
The following toleration would allow an end-user workload to schedule on any Linux node of an RKE1 Windows cluster. These tolerations are used for various core Rancher services and workloads.
apiVersion: apps/v1
kind: Deployment
spec:
...
template:
...
spec:
tolerations:
- effect: NoSchedule
key: cattle.io/os
operator: Equal
value: linuxAligning with best practices, any end-user workloads being ran on Linux nodes would be scheduled on those with the worker role only.
apiVersion: apps/v1
kind: Deployment
spec:
...
template:
...
spec:
tolerations:
- effect: NoSchedule
key: cattle.io/os
operator: Equal
value: linux
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- preference:
matchExpressions:
- key: node-role.kubernetes.io/worker
operator: In
values:
- "true"
weight: 100
...Based on feedback and requests for hybrid workload support, RKE2 Windows was designed to support both Linux and Windows workloads by default. RKE2 scheduling relies on node selectors by default. This is a marked change from RKE1 as taints and tolerations were not incorporated into RKE2. Node selectors were a critical part of RKE1 Windows clusters, which makes for an easy migration of your workloads.
Pre-Migration RKE1 Windows Deployment
apiVersion: apps/v1
kind: Deployment
spec:
...
template:
...
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/os
operator: NotIn
values:
- linuxMigrated RKE2 Windows Deployment Using NodeAffinity
apiVersion: apps/v1
kind: Deployment
...
spec:
...
template:
...
spec:
...
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/os
operator: In
values:
- windowsImportant Things to Keep in Mind when Leveraging Node Selectors and Node Affinity
- If you specify both nodeSelector and nodeAffinity, both must be satisfied for the Pod to be scheduled onto a node.
- If you specify multiple matchExpressions associated with a single nodeSelectorTerms, then the Pod can be scheduled onto a node only if all the matchExpressions are satisfied.
Pre-Migration RKE1 Windows Cluster Linux-Only Deployment Targeting RKE1 Linux Worker Nodes
apiVersion: apps/v1
kind: Deployment
spec:
...
template:
...
spec:
tolerations:
- effect: NoSchedule
key: cattle.io/os
operator: Equal
value: linux
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: node-role.kubernetes.io/worker
operator: In
values:
- "true"
Migrated RKE2 Hybrid Cluster Linux-Only Deployment Targeting RKE2 Linux Worker Nodes Using Node Selectors
apiVersion: apps/v1
kind: Deployment
spec:
...
template:
...
spec:
nodeSelector:
kubernetes.io/os: "linux"
node-role.kubernetes.io/worker: "true"Migrated RKE2 Hybrid Cluster Linux-Only Deployment Targeting RKE2 Linux Worker Nodes Using Node Affinity
apiVersion: apps/v1
kind: Deployment
spec:
...
template:
...
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: node-role.kubernetes.io/worker
operator: In
values:
- "true"
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/os
operator: In
values:
- linux- Windows Server 2019 LTSC ✔️ Will reach Mainstream EOL on Jan 9, 2024 and Extended EOL on Jan 9, 2029
- Windows Server 20H2 SAC ❗ Will reach EOL on Aug 9, 2022
- Windows Server 2004 SAC ❌ EOL Reached on Dec 14, 2021
- Windows Server 1909 SAC ❌ EOL Reached on May 11, 2021
- Windows Server 1903 SAC ❌ EOL Reached on Dec 8, 2020
- Windows Server 1809 SAC ❌ EOL Reached on Nov 10, 2020
- Windows Server 2019 LTSC ✔️ Will reach Mainstream EOL on Jan 9, 2024 and Extended EOL on Jan 9, 2029
- Windows Server 2022 LTSC ✔️ Will reach Mainstream EOL on Oct 13, 2026 and Extended EOL on Oct 13, 2031
References
Windows Server 2022 LTSC Lifecycle
Windows Server 2019 LTSC Lifecycle
NB. All versions listed below are SLA Supported per the Rancher v2.6.5 Support Matrix. Any version not listed should be assumed as being EOL and not supported under SLA by SUSE
RKE1 vs RKE2 Windows Cluster Supported Kubernetes Versions
| Kubernetes Versions | RKE1 | RKE2 |
|---|---|---|
| 1.18 | ✔️ | |
| 1.19 | ✔️ | |
| 1.20 | ✔️ | |
| 1.21 | ✔️ | |
| 1.22 | ✔️ | ✔️ |
| 1.23 | ✔️ | |
| 1.24 | ✔️ | |
| 1.25+ | ✔️ |
1.4.2. Rancher 2.5 vs Rancher 2.6 Supported Kubernetes Versions for Provisioning RKE1 and RKE2 Windows Clusters
| Rancher Versions | Kubernetes Versions | RKE1 | RKE2 |
|---|---|---|---|
| 2.5 - RKE1 Provisioning | 1.18 1.19 1.20 | ✔️ | |
| 2.6 - RKE1 Provisioning | 1.18 1.19 1.20 1.21 1.22 | ✔️ | |
| 2.6 - RKE2 Provisioning | 1.22 1.23 1.24 1.25+ | ✔️ |
Referencing the tables in Rancher 2.5 vs. Rancher 2.6 Support Matrix for Windows Clusters and Rancher 2.5 vs Rancher 2.6 Supported Kubernetes Versions for Provisioning RKE1 and RKE2 Windows Clusters, you will find the overlap in Kubernetes versions between RKE1 and RKE2 occurs in 1.22. This will be the base version required to migrate RKE1 Windows workloads when following the Rancher recommended approach.
- Upgrade the Rancher version to v2.6.5+
- Upgrade the RKE1 Windows downstream cluster(s) to RKE1 v1.22 using the latest available patch version
- Provision a new RKE2 Windows downstream cluster using RKE2 v1.22 using the matching patch version that the RKE1 Windows cluster is at
- Begin the migration of the Windows workloads from RKE1 to RKE2 clusters
- Perform validation tests to ensure that there has been no functionality loss or change when migrating your application from RKE1 to RKE2
- After successful validation tests have occurred, you can opt to upgrade your RKE2 1.22.x cluster to a new minor version such as 1.23 or 1.24.
- Provision a new RKE2 Windows downstream cluster using RKE2 v1.22 using the matching patch version that the RKE1 Windows cluster is at
- Begin the migration of the Windows workloads from RKE1 to RKE2 clusters
- Perform validation tests to ensure that there has been no functionality loss or change when migrating your application from RKE1 to RKE2
- After successful validation tests have occurred, you can opt to upgrade your RKE2 1.22.x cluster to a new minor version such as 1.23 or 1.24.
- Provision a new RKE2 Windows downstream cluster using RKE2 v1.23 or v1.24
- Begin the migration of the Windows workloads from RKE1 to RKE2 clusters
- Perform validation tests to ensure that there has been no functionality loss or change when migrating your application from RKE1 to RKE2