Setting up oidc-login for scaleway kapsule against azure ad

scaleway-azure-ad-oidc-setup.md

The following sets up an oidc flow using kubelogin aka oidc-login. Make sure to install the tool before continuing.

First set up a bunch of variables

# We're using an email claim, but it should be possible to get other claims to work
# See 
export AAD_USER_IDENTIFICATION=mads@reload.dk

# Get this from the the azure ad groups in the console
export AAD_GROUP_OBJECT_ID=8211e035-e025-4352-bbdd-fef907663fed

# Get this from the azure ad overview page
export TENANT_ID=7235d751-8bbc-4ec9-97ba-f541c34cc434

# Get these from the app registration
# App registration details
#  - Web Redirect uri: http://localhost:8000
#  - Token configuration
#     - added group claim (use an enterprise app and "assigned to application" if you have more than 200)
#     - added optional id claim: upn, email (including turning on Microsoft graph)
#  - API permission: granted consent for the entire organization (otherwise you will just be asked during login)
export CLIENT_ID="6fd7cb8d-d1ef-4db4-b404-8e12be749103"
export CLIENT_SECRET="55U8Q~hmrpxc-fN9MZVOevXM6L3AtN5VMfZ9lbtk"

# Get this from scw k8s cluster list
export CLUSTER_ID=c652b633-ae85-438e-b3e9-74e3bad117d8

# Download the kubeconfig, get the name from it
export CONTEXT_NAME="admin@k8s-sleepy-williamson"
export KUBECONFIG="/Users/danquah/Downloads/kubeconfig-k8s-sleepy-williamson.yaml"

Then configure the cluster - it will cause the control-plane to restart so it might take a minute or so

# Reconfigure the cluster to accept jwt tokens issued from the app registrations.
# The claims must match the once Azure issues, see the last comment on debugging.
# I've added the prefixes to show that they are there.
scw k8s cluster update \
 ${CLUSTER_ID} \
 open-id-connect-config.client-id=${CLIENT_ID} \
 open-id-connect-config.issuer-url=https://login.microsoftonline.com/${TENANT_ID}/v2.0 \
 open-id-connect-config.username-claim=email \
 open-id-connect-config.username-prefix= \
 open-id-connect-config.groups-claim.0=groups \
 open-id-connect-config.groups-prefix=

Then setup the role bindings. I'm doing two to demonstrate both a straight user binding and a group - one of these would suffice.

cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: aad-user-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: ${AAD_USER_IDENTIFICATION}
EOF

cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: aad-group-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: ${AAD_GROUP_OBJECT_ID}
EOF

Then add a new user to your kube-context and switch to it. The old admin user will still be in there and you can get back to it by running the last command again with the name of the which you can get from kubectl config get-users.

# Create a new user to authenticate as, update the context to use it
kubectl config set-credentials "azure-user" \
  --exec-api-version=client.authentication.k8s.io/v1beta1 \
  --exec-command=kubectl \
  --exec-arg=oidc-login \
  --exec-arg=get-token \
  --exec-arg=--oidc-issuer-url=https://login.microsoftonline.com/${TENANT_ID}/v2.0 \
  --exec-arg=--oidc-client-id=${CLIENT_ID} \
  --exec-arg=--oidc-client-secret=${CLIENT_SECRET}

# Switch to the user.
kubectl config set-context --current=true --user=azure-user

Run a kubectl command that contacts the api, eg kubectl get nodes, oidc-login should now trigger a login flow via a browser and return you.

Debugging

Should you get an access denied,

kubectl get nodes
error: You must be logged in to the server (Unauthorized)

You should tripple-check the ClusterRoleBinding, and verify that the token oidc-login has fetched contains the expected claims. You can find the token in oidc-logins cache under ~/.kube/cache/oidc-login/ - use something like <https://jwt.ms > to see the claims.

Should you need to trigger a new login flow, just delete the token cache

rm ~/.kube/cache/oidc-login/*