Skip to content

Instantly share code, notes, and snippets.

@cmwylie19

cmwylie19/Kong-Gateway.md

Last active January 3, 2024 18:25
Show Gist options
  • Save cmwylie19/a5557ae3d598f6c57d5b8080568328a4 to your computer and use it in GitHub Desktop.
Save cmwylie19/a5557ae3d598f6c57d5b8080568328a4 to your computer and use it in GitHub Desktop.
Download ZIP
Kong Gateway on OpenShift
Raw
Kong-Gateway.md

Kong Builders LiveStream

Focus: Kong Gateway

Background: Deploy and Secure Istio's Bookinfo application through Kong Gateway.

Topics:

CP Prereqs

As usual, create the kong namespace and license secret.

k create ns kong
k create secret generic kong-enterprise-license \
--from-file=license=license.json -n kong

Generate Private Key and Digital Certificate for TLS

openssl req -new -x509 -nodes -newkey ec:<(openssl ecparam -name secp384r1) \
  -keyout ./cluster.key -out ./cluster.crt \
  -days 1095 -subj "/CN=kong_clustering"

k create secret tls kong-cluster-cert --cert=./cluster.crt --key=./cluster.key -n kong

Create Session Config for Kong Manager and Kong DevPortal

echo '{"cookie_name":"admin_session","cookie_samesite":"off","secret":"kong","cookie_secure":false,"storage":"kong"}' > admin_gui_session_conf
echo '{"cookie_name":"portal_session","cookie_samesite":"off","secret":"kong","cookie_secure":false,"storage":"kong"}' > portal_session_conf

k create secret generic kong-session-config -n kong --from-file=admin_gui_session_conf --from-file=portal_session_conf

Create Kong Manager Password

k create secret generic kong-enterprise-superuser-password -n kong --from-literal=password=kong

Add Kong charts repo

helm repo add kong https://charts.konghq.com
helm repo update

Deploy Control Plane

Deploy the Control Plane through Helm

helm install kong kong/kong -n kong \
--set env.database=postgres \
--set env.password.valueFrom.secretKeyRef.name=kong-enterprise-superuser-password \
--set env.password.valueFrom.secretKeyRef.key=password \
--set env.role=control_plane \
--set env.cluster_cert=/etc/secrets/kong-cluster-cert/tls.crt \
--set env.cluster_cert_key=/etc/secrets/kong-cluster-cert/tls.key \
--set cluster.enabled=true \
--set cluster.tls.enabled=true \
--set cluster.tls.servicePort=8005 \
--set cluster.tls.containerPort=8005 \
--set clustertelemetry.enabled=true \
--set clustertelemetry.tls.enabled=true \
--set clustertelemetry.tls.servicePort=8006 \
--set clustertelemetry.tls.containerPort=8006 \
--set image.repository=kong/kong-gateway \
--set image.tag=2.8.0.0-alpine \
--set admin.enabled=true \
--set admin.http.enabled=true \
--set admin.type=NodePort \
--set proxy.enabled=true \
--set ingressController.enabled=true \
--set ingressController.installCRDs=false \
--set ingressController.image.repository=kong/kubernetes-ingress-controller \
--set ingressController.image.tag=2.2.1 \
--set ingressController.env.kong_admin_token.valueFrom.secretKeyRef.name=kong-enterprise-superuser-password \
--set ingressController.env.kong_admin_token.valueFrom.secretKeyRef.key=password \
--set ingressController.env.enable_reverse_sync=true \
--set ingressController.env.sync_period="1m" \
--set postgresql.enabled=true \
--set postgresql.auth.username=kong \
--set postgresql.auth.database=kong \
--set postgresql.auth.password=kong \
--set postgresql.volumePermissions.enabled=false \
--set postgresql.volumePermissions.securityContext.runAsUser="auto" \
--set postgresql.primary.containerSecurityContext.enabled=false \
--set postgresql.primary.podSecurityContext.enabled=false \
--set enterprise.enabled=true \
--set enterprise.license_secret=kong-enterprise-license \
--set enterprise.rbac.enabled=true \
--set enterprise.rbac.session_conf_secret=kong-session-config \
--set enterprise.rbac.admin_gui_auth_conf_secret=admin-gui-session-conf \
--set enterprise.smtp.enabled=false \
--set enterprise.portal.enabled=true \
--set manager.enabled=true \
--set manager.type=NodePort \
--set portal.enabled=true \
--set portal.http.enabled=true \
--set env.portal_gui_protocol=http \
--set portal.type=NodePort \
--set portalapi.enabled=true \
--set portalapi.http.enabled=true \
--set portalapi.type=NodePort \
--set secretVolumes[0]=kong-cluster-cert

Expose the services through OpenShift routes

oc expose service kong-kong-admin -n kong
oc expose service kong-kong-manager -n kong
oc expose service kong-kong-portal -n kong
oc expose service kong-kong-portalapi -n kong

Wait for CP

k wait --for=condition=ready pod -n kong -l app.kubernetes.io/component=app --timeout=180s

k wait --for=condition=initialized pod -n kong -l app.kubernetes.io/component=init-migrations --timeout=180s

k wait --for=condition=ready pod -n kong -l app.kubernetes.io/name=postgresql --timeout=180s

Check the Admin API

http $(oc get route kong-kong-admin -o jsonpath='{.spec.host}' -n kong) kong-admin-token:kong | jq -r .version

Configure Kong Manager Service

oc patch deployment -n kong kong-kong -p "{\"spec\": { \"template\" : { \"spec\" : {\"containers\":[{\"name\":\"proxy\",\"env\": [{ \"name\" : \"KONG_ADMIN_API_URI\", \"value\": \"$(oc get route kong-kong-admin -o jsonpath='{.spec.host}' -n kong)\" }]}]}}}}"

Validate Patch

k get deploy -n kong kong-kong -ojsonpath='{ .spec.template.spec.containers }' | jq  | grep -A1 KONG_ADMIN_API_URI

DP Prereqs

As usual, create the namespace for the data plane and license secret. Also, create the secret for the TLS cert.

k create ns kong-dp

k create secret generic kong-enterprise-license \
--from-file=license=license.json -n kong-dp

k create secret tls kong-cluster-cert --cert=./cluster.crt --key=./cluster.key -n kong-dp

Deploy DP

Deploy the Data Plane in kong-dp through the helm

helm install kong-dp kong/kong -n kong-dp \
--set ingressController.enabled=false \
--set image.repository=kong/kong-gateway \
--set image.tag=2.8.0.0-alpine \
--set env.database=off \
--set env.role=data_plane \
--set env.cluster_cert=/etc/secrets/kong-cluster-cert/tls.crt \
--set env.cluster_cert_key=/etc/secrets/kong-cluster-cert/tls.key \
--set env.lua_ssl_trusted_certificate=/etc/secrets/kong-cluster-cert/tls.crt \
--set env.cluster_control_plane=kong-kong-cluster.kong.svc.cluster.local:8005 \
--set env.cluster_telemetry_endpoint=kong-kong-clustertelemetry.kong.svc.cluster.local:8006 \
--set proxy.enabled=true \
--set proxy.type=NodePort \
--set enterprise.enabled=true \
--set enterprise.license_secret=kong-enterprise-license \
--set enterprise.portal.enabled=false \
--set enterprise.rbac.enabled=false \
--set enterprise.smtp.enabled=false \
--set manager.enabled=false \
--set portal.enabled=false \
--set portalapi.enabled=false \
--set env.status_listen=0.0.0.0:8100 \
--set secretVolumes[0]=kong-cluster-cert

Expose the kong-dp proxy service as an OpenShift route

oc expose svc/kong-dp-kong-proxy -n kong-dp

Wait for kong-dp pod to be ready

k wait --for=condition=ready pod -n kong-dp -l app.kubernetes.io/component=app --timeout=180s

Check the DP Proxy

curl $(k get route -n kong-dp kong-dp-kong-proxy --template='{{ .spec.host }}') | jq

Check the clustering status to ensure cp and dp are properly configured.

http $(k get route -n kong kong-kong-admin --template='{{ .spec.host }}')/clustering/status kong-admin-token:kong

Deploy Demo App

Create namespace for the bookinfo and deploy through raw manifests

k create ns bookinfo

Allow pods to set securityContext.runAsUser in OpenShift admission controller

k create rolebinding bookinfo-anyuid --clusterrole=system:openshift:scc:anyuid --group=system:serviceaccounts:bookinfo -n bookinfo
k apply -n bookinfo -f https://raw.githubusercontent.com/istio/istio/release-1.14/samples/bookinfo/platform/kube/bookinfo.yaml

Delete Version 2 and 3 of reviews and ratings, we will not be using them

k delete deploy/reviews-v1 deploy/reviews-v2 -n bookinfo

Wait for bookinfo app to be ready

k wait --for=condition=ready pod -n bookinfo -l app=details
k wait --for=condition=ready pod -n bookinfo -l app=productpage
k wait --for=condition=ready pod -n bookinfo -l app=ratings
k wait --for=condition=ready pod -n bookinfo -l app=reviews

Check Product Page in the browser

k port-forward svc/productpage -n bookinfo 9080

Product Page

Expose Product Page through the Ingress

k create ing bookinfo-ingress --rule="*.co/=productpage:9080" -n bookinfo 

# annotate strip-patch

k annotate ing/bookinfo-ingress -n bookinfo konghq.com/strip-path="true"

# patch ingressClassname
k patch ing/bookinfo-ingress --type='json' -p='[{"op": "add", "path": "/spec/ingressClassName", "value":"kong"}]' -n bookinfo

# remove host
k patch ing/bookinfo-ingress --type='json' -p='[{"op": "remove", "path": "/spec/rules/0/host"}]' -n bookinfo

# change pathType to prefix
k patch ing/bookinfo-ingress --type='json' -p='[{"op": "replace", "path": "/spec/rules/0/http/paths/0/pathType", "value":"Prefix"}]' -n bookinfo

Test the ingress in the browser

k get route -n kong-dp kong-dp-kong-proxy --template='{{ .spec.host }}' | pbcopy

Configure Rate Limiting

k apply -f -<<EOF
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: rl-by-minute
  namespace: bookinfo
config: 
  minute: 3
  policy: local
  limit_by: ip
plugin: rate-limiting
EOF

Add the Rate Limiting Plugin to the Ingress

k annotate ing/bookinfo-ingress -n bookinfo konghq.com/plugins=rl-by-minute

Curl until you see a 429

curl --head http://kong-dp-kong-proxy-kong-dp.apps.kong-cp-cw.kni.syseng.devcluster.openshift.com/productpage?u=normal

Configure OIDC

Create namespace for keycloak and install operator

k create ns kong-keycloak

k apply -f -<<EOF
apiVersion: operators.coreos.com/v1alpha2
kind: OperatorGroup
metadata:
  name: kong-operator-group
  namespace: kong-keycloak
spec:
  targetNamespaces:
    - kong-keycloak
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: keycloak-operator
  namespace: kong-keycloak
spec:
  channel: alpha
  installPlanApproval: Automatic
  name: keycloak-operator
  source: community-operators
  sourceNamespace: openshift-marketplace
  startingCSV: keycloak-operator.v18.0.0
EOF

Wait for the keycloak operator to be read

k wait --for=condition=ready pod -l name=keycloak-operator -n kong-keycloak

Create an instance of the keycloak operator, a realm for kong, a customer role definition, a keycloak client, and a user

k create -f -<<EOF
apiVersion: keycloak.org/v1alpha1
kind: Keycloak
metadata:
  labels:
    app: kong-demo
  name: kong-keycloak
  namespace: kong-keycloak
spec:
  externalAccess:
    enabled: true
  instances: 1
---
apiVersion: keycloak.org/v1alpha1
kind: KeycloakRealm
metadata:
  name: kong-realm
  namespace: kong-keycloak
  labels:
    app: kong-demo
spec:
  instanceSelector:
    matchLabels:
      app: kong-demo
  realm:
    displayName: Kong Realm
    enabled: true
    id: kong
    realm: kong
    roles:
      realm:
        - name: customer
---
apiVersion: keycloak.org/v1alpha1
kind: KeycloakClient
metadata:
  name: kuma-demo-client
  namespace: kong-keycloak
  labels:
    app: kong-demo
spec:
  client:
    enabled: true
    clientAuthenticatorType: client-secret
    redirectUris:
      - 'http://kong-dp-kong-proxy-kong-dp.$(oc get ingresses.config/cluster -o jsonpath={.spec.domain})/*'
    clientId: kuma-demo-client
    name: kuma-demo-client
    rootUrl: 'http://kong-dp-kong-proxy-kong-dp.$(oc get ingresses.config/cluster -o jsonpath={.spec.domain})/'
    implicitFlowEnabled: false
    secret: client-secret
    publicClient: false
    standardFlowEnabled: true
    directAccessGrantsEnabled: true
    protocolMappers:
      - protocol: openid-connect
        protocolMapper: oidc-usermodel-realm-role-mapper
        name: realm roles
        config:
          access.token.claim: "true"
          multivalued: "true"
          jsonType.label: String
          claim.name: roles
  realmSelector:
    matchLabels:
      app: kong-demo
---
apiVersion: keycloak.org/v1alpha1
kind: KeycloakUser
metadata:
  name: kermit-user
  namespace: kong-keycloak
spec:
  realmSelector:
    matchLabels:
      app: kong-demo
  user:
    credentials:
      - type: password
        value: kong
    email: kthefrog@example.com
    enabled: true
    firstName: Kermit
    lastName: The Frog
    username: kermit
    realmRoles:
      - customer
EOF

Wait for the instance of Keycloak Operator and Postgres to be ready

k wait --for=condition=ready pod -l statefulset.kubernetes.io/pod-name=keycloak-0 -n kong-keycloak

k wait --for=condition=ready pod -l component=database -n kong-keycloak

Create the KongPlugin for OIDC in the bookinfo namespace

k create -f -<<EOF
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: keycloak-auth-plugin
  namespace: bookinfo
config: 
  auth_methods:
  - authorization_code
  - session
  hide_credentials: true
  issuer: https://keycloak-kong-keycloak.$(oc get ingresses.config/cluster -o jsonpath={.spec.domain})/auth/realms/kong
  client_id:
  - kuma-demo-client
  client_secret:
  - client-secret
  roles_required:
  - customer
plugin: openid-connect
EOF

Annotate the ingress to use the keycloak-auth-plugin

k annotate ing/bookinfo-ingress -n bookinfo konghq.com/plugins=keycloak-auth-plugin

Configure Network Policy

Architecture

First, create a deny all from the ingress

k apply -f -<<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
  namespace: bookinfo
spec:
  policyTypes: ["Ingress"]
  podSelector: {}
EOF

Check ProductPage

Allow Product Page to be accessed from anywhere

k apply -f -<<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: productpage-ingress
  namespace: bookinfo
spec:
  policyTypes: ["Ingress"]
  podSelector:
    matchLabels:
      app: productpage
  ingress:
  - from: []
EOF

Check ProductPage

Allow Product Page to talk to reviews

k apply -f -<<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: reviews-productpage
  namespace: bookinfo
spec:
  policyTypes: ["Ingress"]
  podSelector:
    matchLabels:
      app: reviews
  ingress:
  - from:
      - podSelector:
          matchLabels:
            app: productpage
EOF

Check ProductPage

Allow Reviews to talk to Ratings

k apply -f -<<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: ratings-reviews
  namespace: bookinfo
spec:
  policyTypes: ["Ingress"]
  podSelector:
    matchLabels:
      app: ratings
  ingress:
  - from:
      - podSelector:
          matchLabels:
            app: reviews
EOF

Check ProductPage

Allow Product Page to talk to Details

k apply -f -<<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: details-productpage
  namespace: bookinfo
spec:
  policyTypes: ["Ingress"]
  podSelector:
    matchLabels:
      app: details
  ingress:
  - from:
      - podSelector:
          matchLabels:
            app: productpage
EOF

Check ProductPage

Clean Up

helm uninstall kong -n kong; 

helm uninstall kong-dp -n kong-dp

k delete secret,cm,route,svc,deploy,po,role,rolebinding --all -n kong --force --grace-period=0

k delete secret,cm,route,svc,deploy,po,role,rolebinding --all -n kong-dp --force --grace-period=0

k delete secret,cm,route,svc,deploy,po,role,rolebinding,kongplugin,ingress,netpol --all -n bookinfo --force --grace-period=0

k delete csv -n kong-keycloak --all

k delete og,subs -n kong-keycloak --all 

k patch  keycloakclient/kuma-demo-client --type json -p '[ { "op": "remove", "path": "/metadata/finalizers" } ]' -n kong-keycloak 

k delete keycloakclient/kuma-demo-client -n kong-keycloak 

k patch  keycloakrealm/kong-realm --type json -p '[ { "op": "remove", "path": "/metadata/finalizers" } ]' -n kong-keycloak 

k delete keycloakrealm/kong-realm -n kong-keycloak 

k patch  keycloakuser/kermit-user --type json -p '[ { "op": "remove", "path": "/metadata/finalizers" } ]' -n kong-keycloak 

k delete keycloakuser/kermit-user -n kong-keycloak 

k delete keycloakclient,keycloakrealm,keycloakuser,keycloak -n kong-keycloak --force --all

k delete ns kong-keycloak

k delete -n bookinfo -f https://raw.githubusercontent.com/istio/istio/release-1.14/samples/bookinfo/platform/kube/bookinfo.yaml

k delete ns bookinfo kong kong-dp

rm cluster.crt cluster.key portal_session* admin_gui*

back to top

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment