pod security proof-of-concept

readme.md

Overview

Proof of concept demo testing out my comments on the "Bare Minimum Pod Security" proposal:

• Exemptions for users, namespaces, and runtimeclasses
• Support for restricted profile
• Policy versioning

Consider this a proof of concept of "Bare Medium" Pod Security

Three policy levels, mapping to Pod Security Standards levels:

• privileged
• baseline
• restricted

Three actions that can be taken, expressed as a namespace label of podsecurity.kubernetes.io/<action>=<level>:

• allow=<level> (reject pods that exceed <level>)
• warn=<level> (warn the user when creating a pod that exceeds <level>)
• audit=<level> (add info to the audit log when creating a pod that exceeds <level>)

Ability to indicate the version of the policy

• as it existed at a particular Kubernetes version should be used:
  • podsecurity.kubernetes.io/<action>.version=v1.7
• or the latest incarnation (default):
  • podsecurity.kubernetes.io/<action>.version=latest

Implications of the expressiveness of this structure:

• a namespace can have at most one level associated with a given action
• multiple actions can reference the same level (can warn and audit baseline)
• multiple actions can reference different levels (can warn restricted and audit baseline)
• multiple actions can reference different versions of the same level (can allow baseline v1.7 and warn+audit baseline latest)

Ability to query namespaces based on this structure:

• can find namespaces explicitly selecting to enforce
  • kubectl get ns --show-labels -l podsecurity.kubernetes.io/allow
• can find namespaces explicitly selecting a particular enforcement level
  • kubectl get ns --show-labels -l podsecurity.kubernetes.io/allow=baseline
• can find namespaces explicitly selecting a particular enforcement level that are using a policy version that is not latest
  • kubectl get ns --show-labels -l 'podsecurity.kubernetes.io/allow=restricted,podsecurity.kubernetes.io/allow.version,podsecurity.kubernetes.io/allow.version!=latest'
• can find namespaces explicitly selecting to enforce that haven't indicated what policy version they want
  • kubectl get ns --show-labels -l 'podsecurity.kubernetes.io/allow,!podsecurity.kubernetes.io/allow.version'

Demo

Branch at https://github.com/liggitt/kubernetes/tree/bare-medium-pod-security

# Create a bunch of namespaces
kubectl create ns demo-0-no-policy
kubectl label  ns demo-0-no-policy demo=true

kubectl create ns demo-1-privileged
kubectl label  ns demo-1-privileged demo=true podsecurity.kubernetes.io/allow=privileged

kubectl create ns demo-2-baseline
kubectl label  ns demo-2-baseline demo=true podsecurity.kubernetes.io/allow=baseline

kubectl create ns demo-2-baseline-warn
kubectl label  ns demo-2-baseline-warn demo=true podsecurity.kubernetes.io/warn=baseline

kubectl create ns demo-3-restricted
kubectl label  ns demo-3-restricted demo=true podsecurity.kubernetes.io/allow=restricted

kubectl create ns demo-3-restricted-warn
kubectl label  ns demo-3-restricted-warn demo=true podsecurity.kubernetes.io/warn=restricted

# namespace using the restricted policy as it existed in 1.7 (before the allowPrivilegeEscalation field existed)
kubectl create ns demo-3-restricted-legacy
kubectl label  ns demo-3-restricted-legacy demo=true podsecurity.kubernetes.io/allow=restricted podsecurity.kubernetes.io/allow.version=v1.7

clear

kubectl get ns -l demo --show-labels


# try to create a privileged pod in each demo namespace and show the results
kubectl get ns -l demo -o jsonpath={.items[*].metadata.name} | xargs -p -n 1 kubectl apply -f pod-privileged.yaml -n
kubectl get pods -A --field-selector metadata.name=privileged-pod

# try to create a baseline pod in each demo namespace and show the results
kubectl get ns -l demo -o jsonpath={.items[*].metadata.name} | xargs -p -n 1 kubectl apply -f pod-baseline.yaml -n
kubectl get pods -A --field-selector metadata.name=baseline-pod

# try to create a restricted pod in each demo namespace and show the results
kubectl get ns -l demo -o jsonpath={.items[*].metadata.name} | xargs -t -n 1 kubectl apply -f pod-restricted.yaml -n
kubectl get pods -A --field-selector metadata.name=restricted-pod

# auto-create
kubectl get ns -l demo -o jsonpath={.items[*].metadata.name} | xargs -t -n 1 kubectl apply -f pod-privileged.yaml -n
kubectl get ns -l demo -o jsonpath={.items[*].metadata.name} | xargs -t -n 1 kubectl apply -f pod-baseline.yaml -n
kubectl get ns -l demo -o jsonpath={.items[*].metadata.name} | xargs -t -n 1 kubectl apply -f pod-restricted.yaml -n
clear

# Now I have a mix of namespaces, pods, and policy levels

kubectl get pods -A

# What about cluster upgrades?
# Before an upgrade to v1.21, I could find enforcing namespaces that hadn't explicitly opted into the latest version and pin them to v1.20:
kubectl label ns --all -l 'podsecurity.kubernetes.io/allow,!podsecurity.kubernetes.io/audit.version' podsecurity.kubernetes.io/audit.version=v1.20

# Then upgrade the cluster to v1.21 and unpin at my leisure.
# To see what that would look like, zoom in on demo-3-restricted-legacy...
# This is a good example of a difficult to manage namespace.
# This namespace is pinned to an old version of the restricted policy,
# and has a pod that would be disallowed by the latest version of that policy

kubectl get ns demo-3-restricted-legacy --show-labels
kubectl get pods -n demo-3-restricted-legacy

# First, it is easy to find namespaces like this, explicitly indicating a level and version that is not "latest"
kubectl get ns -l 'podsecurity.kubernetes.io/allow,podsecurity.kubernetes.io/allow.version,podsecurity.kubernetes.io/allow.version!=latest' --show-labels

# When I'm ready to update the namespace, I have the following options:

# 1. Audit the new restricted policy and monitor audit logs for "a while":
kubectl label ns demo-3-restricted-legacy podsecurity.kubernetes.io/audit=restricted podsecurity.kubernetes.io/audit.version=latest

# 2. Surface warnings to the user if their stuff violated the latest restricted policy for "a while":
kubectl label ns demo-3-restricted-legacy podsecurity.kubernetes.io/warn=restricted podsecurity.kubernetes.io/warn.version=latest

# 3. Dry-run switching to the latest restricted policy and let the server warn me if it knows of pods that would be disallowed:
kubectl label --dry-run=server --overwrite ns demo-3-restricted-legacy podsecurity.kubernetes.io/allow=restricted podsecurity.kubernetes.io/allow.version=latest
# (No labels were changed in the checking of this policy)
kubectl get ns demo-3-restricted-legacy --show-labels

# Other interesting dry-run incantations:
# 1. check if it's safe to lock down all namespaces to latest baseline
kubectl label --dry-run=server --overwrite ns --all podsecurity.kubernetes.io/allow=baseline podsecurity.kubernetes.io/allow.version=latest

# 2. check if it's safe to change the cluster default to latest baseline
kubectl label --dry-run=server --overwrite ns -l '!podsecurity.kubernetes.io/allow' podsecurity.kubernetes.io/allow=baseline podsecurity.kubernetes.io/allow.version=latest

# 3. check if it's safe to change the cluster default to latest restricted
kubectl label --dry-run=server --overwrite ns -l '!podsecurity.kubernetes.io/allow' podsecurity.kubernetes.io/allow=restricted podsecurity.kubernetes.io/allow.version=latest

pod-baseline.yaml

kind: Pod
apiVersion: v1
metadata:
  name: baseline-pod
spec:
  containers:
  - name: default
    image: k8s.gcr.io/pause:3.2

pod-privileged.yaml

kind: Pod
apiVersion: v1
metadata:
  name: privileged-pod
spec:
  containers:
  - name: privileged
    image: k8s.gcr.io/pause:3.2
    securityContext:
      privileged: true

pod-restricted.yaml

kind: Pod
apiVersion: v1
metadata:
  name: restricted-pod
spec:
  containers:
  - name: default
    image: k8s.gcr.io/pause:3.2
    securityContext:
      allowPrivilegeEscalation: false