Skip to content

Instantly share code, notes, and snippets.

@jostmart
Forked from rgl/rke2-windows.md
Created October 4, 2022 19:50
Show Gist options
  • Save jostmart/ddb92d91390a80e84cc13de39e6fca15 to your computer and use it in GitHub Desktop.
Save jostmart/ddb92d91390a80e84cc13de39e6fca15 to your computer and use it in GitHub Desktop.
rke2 windows notes

Notes

Reference

Install Server Nodes

Enter the first server node and bootstrap the rke2 managed kubernetes cluster.

Install the rke2 binaries:

apt-get update
apt-get install -y curl
curl -sfL https://get.rke2.io | \
    INSTALL_RKE2_VERSION='v1.21.5+rke2r1' \
    sh -

Create the rke2-server service configuration file:

install -d -m 700 /etc/rancher/rke2
install /dev/null -m 600 /etc/rancher/rke2/config.yaml
cat >/etc/rancher/rke2/config.yaml <<'EOF'
cni: calico
node-taint: CriticalAddonsOnly=true:NoExecute
EOF

Start the rke2-server service:

systemctl status rke2-server.service
systemctl enable rke2-server.service
systemctl start rke2-server.service
# wait for the token file to be created.
while [ ! -f /var/lib/rancher/rke2/server/node-token ]; do sleep 5; done
# show the configuration that should be added to the other nodes
# /etc/rancher/rke2/config.yaml file:
cat <<EOF
token: $(cat /var/lib/rancher/rke2/server/node-token)
server: https://$(ip addr show eth0 | perl -n -e '/inet (\d+(\.\d+)+)\/\d+/ && print $1'):9345
EOF
# in another shell you can tail the logs with:
#   journalctl -u rke2-server.service

Symlink the utilities and setup the environment variables to use them:

ln -fs /var/lib/rancher/rke2/bin/{kubectl,crictl,ctr} /usr/local/bin/
cat >/etc/profile.d/01-rke2.sh <<'EOF'
export CONTAINERD_ADDRESS=/run/k3s/containerd/containerd.sock
export CONTAINERD_NAMESPACE=k8s.io
export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
EOF
source /etc/profile.d/01-rke2.sh
kubectl get nodes -o wide

Install Linux Worker Nodes

Enter the worker node and add it to the cluster as follows.

Install the rke2 binaries:

apt-get update
apt-get install -y curl
curl -sfL https://get.rke2.io | \
    INSTALL_RKE2_VERSION='v1.21.5+rke2r1' \
    INSTALL_RKE2_TYPE='agent' \
    sh -

Configure the rke2-agent service to connect to the kubernetes cluster:

install -d -m 700 /etc/rancher/rke2
install /dev/null -m 600 /etc/rancher/rke2/config.yaml
cat >/etc/rancher/rke2/config.yaml <<'EOF'
token: <TODO set the token from controller node /var/lib/rancher/rke2/server/node-token>
server: https://<TODO set the controller node host or ip address>:9345
EOF

Start the rke2-agent service:

systemctl status rke2-agent.service
systemctl enable rke2-agent.service
systemctl start rke2-agent.service
journalctl -u rke2-agent.service -f

Symlink the utilities and setup the environment variables to use them:

# NB kubectl should not be available in worker nodes as rke2 does not
#    install a kubeconfig.
ln -fs /var/lib/rancher/rke2/bin/{crictl,ctr} /usr/local/bin/
cat >/etc/profile.d/01-rke2.sh <<'EOF'
export CONTAINERD_ADDRESS=/run/k3s/containerd/containerd.sock
export CONTAINERD_NAMESPACE=k8s.io
export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
EOF
source /etc/profile.d/01-rke2.sh
crictl pods list

Install Windows Worker Nodes

Enter the worker node and add it to the cluster as follows.

Add support for running containters:

Install-WindowsFeature Containers
Restart-Computer

Configure the rke2 service to connect to the kubernetes cluster:

New-Item -Type Directory c:/etc/rancher/rke2 -Force | Out-Null
# TODO configure the c:/etc/rancher/rke2 permissions to be available for Administrators and SYSTEM.
Set-Content -Path c:/etc/rancher/rke2/config.yaml -Value @'
server: https://<server>:9345
token: <token from server node>
'@

Install the rke2 binaries:

Invoke-WebRequest `
    -Uri https://raw.githubusercontent.com/rancher/rke2/master/install.ps1 `
    -Outfile install.ps1
.\install.ps1 `
    -Version 'v1.21.5+rke2r1' `
    -Type agent

Add rke2 to the current user powershell profile:

if (!(Test-Path (Split-Path -Parent $PROFILE))) {
    mkdir -Force (Split-Path -Parent $PROFILE) | Out-Null
}
Add-Content $PROFILE @'

# RKE2.
$env:PATH += ';c:\var\lib\rancher\rke2\bin;c:\usr\local\bin'
$env:CONTAINERD_NAMESPACE = 'k8s.io'
$env:CRI_CONFIG_FILE = 'c:\var\lib\rancher\rke2\agent\etc\crictl.yaml'
'@
. $PROFILE

Allow access to the Kubelet port in the firewall:

# see https://github.com/rancher/rke2/issues/1762
New-NetFirewallRule `
    -Name 'Kubelet-TCP-In' `
    -DisplayName 'Kubelet' `
    -Direction 'Inbound' `
    -LocalPort 10250 `
    -Enabled True `
    -Protocol 'TCP' `
    | Out-Null

Install and start the rke2 service:

rke2.exe agent service --add
Start-Service rke2

NB You will loose network access to the machine for a brief period. You should be able to re-connect after a while.

Show the HNS network:

Import-Module c:\var\lib\rancher\rke2\bin\hns.psm1
Get-HnsNetwork

Use

From a Server node, lets try to run some containers.

kubectl run --restart=Never --image=mcr.microsoft.com/windows/nanoserver:1809 --rm -it test-nanoserver
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment