Skip to content

Instantly share code, notes, and snippets.

@abhisek

abhisek/README.md

Last active June 12, 2024 09:17
Show Gist options
  • Save abhisek/eead07d505ea2c8d5109acbbb8e5d5a9 to your computer and use it in GitHub Desktop.
Save abhisek/eead07d505ea2c8d5109acbbb8e5d5a9 to your computer and use it in GitHub Desktop.
Download ZIP
dex IDP Google Auth Kubernetes Storage using minikube
Raw
README.md

Start minikube with specific kubernetes version and OIDC auth

mkdir -p ~/.minikube/files/etc
echo 127.0.0.1 dex.example.com > ~/.minikube/files/etc/hosts

minikube start --kubernetes-version=1.30 \
  --extra-config=apiserver.oidc-issuer-url=https://dex.example.com:32000 \
  --extra-config=apiserver.oidc-username-claim=email \
  --extra-config=apiserver.oidc-groups-claim=groups \
  --extra-config=apiserver.oidc-ca-file=/var/lib/minikube/certs/ca.crt \
  --extra-config=apiserver.oidc-client-id=example-app

Note: We are re-using the minikube generated ca.crt for OIDC server auth. We will use the same ca.crt and ca.key for generating certificate for dex. Also /var/lib/minikube/certs are mounted under all pods.

Copy the generated CA cert and key from minikube

mkdir ./examples/k8s/ssl
docker cp minikube:/var/lib/minikube/certs/ca.key ./examples/k8s/ssl/
docker cp minikube:/var/lib/minikube/certs/ca.crt ./examples/k8s/ssl/

Note: Assuming minikube is the container name created by minikube start ..

Generate cert for use by dex server

cat <<_EOF > examples/k8s/ssl/req.cnf
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name

[req_distinguished_name]

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[alt_names]
DNS.1 = dex.example.com
DNS.2 = dex.dex.svc.cluster.local
DNS.3 = host.minikube.internal
DNS.4 = control-plane.minikube.internal
IP.1 = 127.0.0.1
_EOF
openssl genrsa -out examples/k8s/ssl/key.pem 2048
openssl req -new -key examples/k8s/ssl/key.pem -out examples/k8s/ssl/csr.pem -subj "/CN=kube-ca" -config examples/k8s/ssl/req.cnf
openssl x509 -req -in examples/k8s/ssl/csr.pem -CA examples/k8s/ssl/ca.crt -CAkey examples/k8s/ssl/ca.key -CAcreateserial -out examples/k8s/ssl/cert.pem -days 10 -extensions v3_req -extfile examples/k8s/ssl/req.cnf

Create Cluster Secret for dex key

kubectl create ns dex
kubectl -n dex delete secrets dex.dex.tls
kubectl -n dex create secret tls dex.dex.tls --cert=examples/k8s/ssl/cert.pem --key=examples/k8s/ssl/key.pem

Create Cluster Secret for Google OAuth2 Credentials

kubectl -n dex create secret \
    generic google-client \
    --from-literal=client-id=$GOOGLE_OAUTH2_CLIENT_ID \
    --from-literal=client-secret=$GOOGLE_OAUTH2_CLIENT_SECRET

Note: Google OAuth2 credentials must allow redirect URL https://dex.example.com:32000/callback

Create dex deployment in the cluster

Note: Modified dex deployment YAML given below is used for Google connector

kubectl apply -f examples/k8s/dex.yaml

Accessing dex using example-app

There are some gotcha here. kube-apiserver need to access dex from outside the cluster but dex needs to be inside the cluster to be able to access Kubernetes storage API.

Add following to /etc/hosts

127.0.0.1 dex.example.com

Setup port forwarding using kubectl

kubectl port-forward -n dex deployments/dex 32000:5556

Run example-app

go run example-app/main.go example-app/templates.go \
  --issuer https://dex.example.com:32000 \
  --issuer-root-ca ./k8s/ssl/ca.crt

Navigate tp http://127.0.0.1:5555 and fetch an ID token

Access Kubernetes API Server using dex ID Token

export DEX_IDTOKEN="..."

Get kube-apiserver URL

kubectl cluster-info

Access using token

curl -H "Authorization: $DEX_IDTOKEN" https://127.0.0.1:55153/api/v1/nodes

Access using kubectl oidc-login

kubectl oidc-login get-token \
  --oidc-issuer-url=https://dex.example.com:32000 \
  --oidc-client-id=kubernetes \
  --oidc-extra-scope=email \
  --oidc-extra-scope=groups \
  --oidc-extra-scope=profile \
  --oidc-extra-scope=offline_access \
  --force-refresh \
  --insecure-skip-tls-verify

Modified dex Kubernetes Manifest

---
apiVersion: v1
kind: Namespace
metadata:
  name: dex
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: dex
  name: dex
  namespace: dex
spec:
  replicas: 3
  selector:
    matchLabels:
      app: dex
  template:
    metadata:
      labels:
        app: dex
    spec:
      serviceAccountName: dex # This is created below
      containers:
      - image: ghcr.io/dexidp/dex:v2.40.0
        name: dex
        command: ["/usr/local/bin/dex", "serve", "/etc/dex/cfg/config.yaml"]

        ports:
        - name: https
          containerPort: 5556

        volumeMounts:
        - name: config
          mountPath: /etc/dex/cfg
        - name: tls
          mountPath: /etc/dex/tls

        env:
        - name: GOOGLE_OAUTH2_CLIENT_ID
          valueFrom:
            secretKeyRef:
              name: google-client
              key: client-id
        - name: GOOGLE_OAUTH2_CLIENT_SECRET
          valueFrom:
            secretKeyRef:
              name: google-client
              key: client-secret

        readinessProbe:
          httpGet:
            path: /healthz
            port: 5556
            scheme: HTTPS
      volumes:
      - name: config
        configMap:
          name: dex
          items:
          - key: config.yaml
            path: config.yaml
      - name: tls
        secret:
          secretName: dex.dex.tls
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: dex
  namespace: dex
data:
  config.yaml: |
    logger:
      level: debug
    issuer: https://dex.example.com:32000
    storage:
      type: kubernetes
      config:
        inCluster: true
    web:
      https: 0.0.0.0:5556
      tlsCert: /etc/dex/tls/tls.crt
      tlsKey: /etc/dex/tls/tls.key
    connectors:
    - type: google
      id: google
      name: Google
      config:
        clientID: $GOOGLE_OAUTH2_CLIENT_ID
        clientSecret: $GOOGLE_OAUTH2_CLIENT_SECRET
        redirectURI: https://dex.example.com:32000/callback
    oauth2:
      skipApprovalScreen: true

    staticClients:
    - id: example-app
      redirectURIs:
      - 'http://127.0.0.1:5555/callback'
      name: 'Example App'
      secret: ZXhhbXBsZS1hcHAtc2VjcmV0
    - id: kubernetes
      name: kubernetes
      public: true

    enablePasswordDB: true
    staticPasswords:
    - email: "admin@example.com"
      # bcrypt hash of the string "password": $(echo password | htpasswd -BinC 10 admin | cut -d: -f2)
      hash: "$2a$10$2b2cU8CPhOTaGrs1HRQuAueS7JTT5ZHsHSzYiFPm1leZck7Mc8T4W"
      username: "admin"
      userID: "08a8684b-db88-4b73-90a9-3cd1661f5466"
---
apiVersion: v1
kind: Service
metadata:
  name: dex
  namespace: dex
spec:
  type: NodePort
  ports:
  - name: dex
    port: 5556
    protocol: TCP
    targetPort: 5556
    nodePort: 32000
  selector:
    app: dex
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: dex
  name: dex
  namespace: dex
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: dex
rules:
- apiGroups: ["dex.coreos.com"] # API group created by dex
  resources: ["*"]
  verbs: ["*"]
- apiGroups: ["apiextensions.k8s.io"]
  resources: ["customresourcedefinitions"]
  verbs: ["create"] # To manage its own resources, dex must be able to create customresourcedefinitions
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: dex
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: dex
subjects:
- kind: ServiceAccount
  name: dex           # Service account assigned to the dex pod, created above
  namespace: dex  # The namespace dex is running in
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment