BrianSigafoos/0_k8s_security_checklist.md

## 0_k8s_security_checklist.md

      
    Raw
  

              0_k8s_security_checklist.md
            
          
    Kubernetes Security

NSA/CISA Kubernetes Hardening Guidance

A summary of the key recommendations from each section are:

Kubernetes Pod security

 Use containers built to run applications as non-root users
 Where possible, run containers with immutable file systems
 Scan container images for possible vulnerabilities or misconfigurations
 Use a Pod Security Policy (deprecated 1.21) -> Pod Security admission controller (new 1.22) to enforce a minimum level of security
including:

 Preventing privileged containers
 Denying container features frequently exploited to breakout, such
as hostPID, hostIPC, hostNetwork, allowedHostPath
 Rejecting containers that execute as the root user or allow
elevation to root
 Hardening applications against exploitation using security services
such as SELinux®, AppArmor®, and seccomp


Network separation and hardening

 Lock down access to control plane nodes using a firewall and role-based
access control (RBAC)
 Further limit access to the Kubernetes etcd server
 Configure control plane components to use authenticated, encrypted
communications using Transport Layer Security (TLS) certificates
 Set up network policies to isolate resources. Pods and services in different
namespaces can still communicate with each other unless additional
separation is enforced, such as network policies
 Place all credentials and sensitive information in Kubernetes Secrets
rather than in configuration files. Encrypt Secrets using a strong
encryption method


Authentication and authorization

 Disable anonymous login (enabled by default)
 Use strong user authentication
 Create RBAC policies to limit administrator, user, and service account
activity


Log auditing

 Enable audit logging (disabled by default)


References


https://www.nsa.gov/News-Features/Feature-Stories/Article-View/Article/2716980/nsa-cisa-release-kubernetes-hardening-guidance/
https://media.defense.gov/2021/Aug/03/2002820425/-1/-1/1/CTR_KUBERNETES%20HARDENING%20GUIDANCE.PDF


## deployment.yaml
apiVersion: apps/v1
kind: Deployment

metadata:
  name: company-webapp

  labels:
    app: webapp

# https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#deploymentspec-v1-apps
spec:
  replicas: 1

  selector:
    matchLabels:
      app: webapp

  template:
    metadata:
      labels:
        app: webapp

    spec:
      # https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#container-v1-core
      containers:
        - name: webapp-container
          image: ghcr.io/user/webapp
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsNonRoot: true

          # TODO: revisit these numbers based on actual production usage
          # and monitoring: kubectl resource-capacity --pods --util
          resources:
            requests:
              cpu: 100m
              memory: 600Mi
            limits:
              cpu: 1000m # 1 core
              memory: 1000Mi

          envFrom:
            - secretRef:
                name: webapp-secrets
            - configMapRef:
                name: webapp-config

          # Pass k8s metadata to containers.
          env:
            - name: K8S_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: K8S_POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            # For Rails web server, Puma expects this to be in tmp/pids/server.pid by default. But,
            # instead of mounting the writable /app/tmp directory, and creating
            # the tmp/pids folder, we can just configure Puma to use /app/tmp
            - name: PIDFILE
              value: tmp/server.pid

          # Naming port so that probes and service can reference it.
          ports:
            - name: pod-port
              containerPort: 3000 # http port, same as in config/puma.rb

# TODO:
#           readinessProbe:
#             httpGet:
#               path: /health_check/load_balancer
#               port: pod-port
#             periodSeconds: 10
#             failureThreshold: 3

          livenessProbe:
            httpGet:
              path: /health_check
              port: pod-port
            periodSeconds: 10
            failureThreshold: 3

          startupProbe:
            httpGet:
              path: /health_check
              port: pod-port
            initialDelaySeconds: 10
            periodSeconds: 10
            failureThreshold: 30

          volumeMounts:
            # Since the container file system is read-only,
            # we'll mount /app/tmp to allow writes to it by Rails cache
            # /app/tmp/cache/ and Puma /app/tmp/server.pid
            - name: app-tmp
              mountPath: /app/tmp

      imagePullSecrets:
        - name: ghcr-k8s-pull
        - name: dockerhub-k8s-pull

      volumes:
        - name: app-tmp
          emptyDir: {}

## Dockerfile
# Versions with defaults. Override with env var to build a different version.
ARG RUBY_VERSION=3.0.2
ARG POSTGRES_MAJOR_VERSION=13
ARG NODEJS_MAJOR_VERSION=16
ARG BUNDLER_VERSION=2.2.26
ARG YARN_VERSION=1.22.5

# More args
# For security, set a non-root user. Name is arbitrary.
ARG USER=nonroot
ARG USER_ID=1001
ARG REVISION=not_set

###
# Development system and dependencies
###
#
# This will have everything needed for local development, and is targeted
# in the docker-compose via tagert: development
FROM ruby:$RUBY_VERSION-slim-buster AS development

# Args needed for this container
ARG POSTGRES_MAJOR_VERSION
ARG NODEJS_MAJOR_VERSION
ARG BUNDLER_VERSION
ARG YARN_VERSION
ARG USER
ARG USER_ID

# Recommended by hadolint
# https://github.com/hadolint/hadolint/wiki/DL4006
SHELL ["/bin/bash", "-o", "pipefail", "-c"]

# Install dependencies
RUN apt-get update -qq \
  && apt-get install -yq --no-install-recommends \
   # ... \
   # ... \
   # ... \
  && apt-get clean \
  && rm -rf /var/cache/apt/archives/* \
  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
  && truncate -s 0 /var/log/*log

# Upgrade RubyGems and install required Bundler version
RUN gem update --system --no-document \
 && gem install bundler:$BUNDLER_VERSION --no-document

# Add non-root user
RUN groupadd --gid $USER_ID $USER \
  && useradd --uid $USER_ID --gid $USER --shell /bin/bash --create-home $USER

# Create a directory for the app code
RUN mkdir /app \
  && chown -R $USER:$USER /app

WORKDIR /app


###
# Install Ruby gems, copied into the prod_image
###
FROM development AS prod_gems

ENV BUNDLE_PATH=/usr/local/bundle
ENV GEM_HOME=/usr/local/bundle

# COPY Gemfile and Gemfile.lock for bundler to use
COPY --chown=$USER:$USER Gemfile* ./

# Tell bundler we're building for production
RUN bundle config set deployment true \
  && bundle config set without development test \
  && bundle install


###
# Final production image
###
FROM ruby:$RUBY_VERSION-slim-buster AS production

# Args needed for this container
ARG POSTGRES_MAJOR_VERSION
ARG USER
ARG USER_ID
ARG REVISION

ENV RAILS_ENV=production
ENV BUNDLE_PATH=/usr/local/bundle
ENV GEM_HOME=$BUNDLE_PATH

# Add non-root user
RUN groupadd --gid $USER_ID $USER \
  && useradd --uid $USER_ID --gid $USER --shell /bin/bash --create-home $USER

# Recommended by hadolint
# https://github.com/hadolint/hadolint/wiki/DL4006
SHELL ["/bin/bash", "-o", "pipefail", "-c"]

# Install dependencies
RUN apt-get update -qq \
  && apt-get install -yq --no-install-recommends \
   # ... \
   # ... \
   # ... \
  && apt-get clean \
  && rm -rf /var/cache/apt/archives/* \
  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
  && truncate -s 0 /var/log/*log

# Create a directory for the app code
RUN mkdir /app \
  && chown -R $USER:$USER /app

WORKDIR /app

# COPY in files built during earlier stages
COPY --chown=$USER:$USER --from=prod_gems $BUNDLE_PATH $BUNDLE_PATH

# Add the app code
COPY --chown=$USER:$USER . ./

# Set user to non-root $USER
# This needs to be the numeric uid, not the username, for the k8s
# securityContext: runAsNonRoot check to work.
USER $USER_ID

# Env vars for production
ENV RAILS_ENV=production
ENV NODE_ENV=production
ENV REVISION=$REVISION

# Start web server
# bundle exec puma -C config/puma.rb
CMD ["bundle", "exec", "puma", "-C", "config/puma.rb"]
	apiVersion: apps/v1
	kind: Deployment

	metadata:
	name: company-webapp

	labels:
	app: webapp

	# https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#deploymentspec-v1-apps
	spec:
	replicas: 1

	selector:
	matchLabels:
	app: webapp

	template:
	metadata:
	labels:
	app: webapp

	spec:
	# https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#container-v1-core
	containers:
	- name: webapp-container
	image: ghcr.io/user/webapp
	securityContext:
	allowPrivilegeEscalation: false
	readOnlyRootFilesystem: true
	runAsNonRoot: true

	# TODO: revisit these numbers based on actual production usage
	# and monitoring: kubectl resource-capacity --pods --util
	resources:
	requests:
	cpu: 100m
	memory: 600Mi
	limits:
	cpu: 1000m # 1 core
	memory: 1000Mi

	envFrom:
	- secretRef:
	name: webapp-secrets
	- configMapRef:
	name: webapp-config

	# Pass k8s metadata to containers.
	env:
	- name: K8S_NODE_NAME
	valueFrom:
	fieldRef:
	fieldPath: spec.nodeName
	- name: K8S_POD_NAME
	valueFrom:
	fieldRef:
	fieldPath: metadata.name
	# For Rails web server, Puma expects this to be in tmp/pids/server.pid by default. But,
	# instead of mounting the writable /app/tmp directory, and creating
	# the tmp/pids folder, we can just configure Puma to use /app/tmp
	- name: PIDFILE
	value: tmp/server.pid

	# Naming port so that probes and service can reference it.
	ports:
	- name: pod-port
	containerPort: 3000 # http port, same as in config/puma.rb

	# TODO:
	# readinessProbe:
	# httpGet:
	# path: /health_check/load_balancer
	# port: pod-port
	# periodSeconds: 10
	# failureThreshold: 3

	livenessProbe:
	httpGet:
	path: /health_check
	port: pod-port
	periodSeconds: 10
	failureThreshold: 3

	startupProbe:
	httpGet:
	path: /health_check
	port: pod-port
	initialDelaySeconds: 10
	periodSeconds: 10
	failureThreshold: 30

	volumeMounts:
	# Since the container file system is read-only,
	# we'll mount /app/tmp to allow writes to it by Rails cache
	# /app/tmp/cache/ and Puma /app/tmp/server.pid
	- name: app-tmp
	mountPath: /app/tmp

	imagePullSecrets:
	- name: ghcr-k8s-pull
	- name: dockerhub-k8s-pull

	volumes:
	- name: app-tmp
	emptyDir: {}
	# Versions with defaults. Override with env var to build a different version.
	ARG RUBY_VERSION=3.0.2
	ARG POSTGRES_MAJOR_VERSION=13
	ARG NODEJS_MAJOR_VERSION=16
	ARG BUNDLER_VERSION=2.2.26
	ARG YARN_VERSION=1.22.5

	# More args
	# For security, set a non-root user. Name is arbitrary.
	ARG USER=nonroot
	ARG USER_ID=1001
	ARG REVISION=not_set

	###
	# Development system and dependencies
	###
	#
	# This will have everything needed for local development, and is targeted
	# in the docker-compose via tagert: development
	FROM ruby:$RUBY_VERSION-slim-buster AS development

	# Args needed for this container
	ARG POSTGRES_MAJOR_VERSION
	ARG NODEJS_MAJOR_VERSION
	ARG BUNDLER_VERSION
	ARG YARN_VERSION
	ARG USER
	ARG USER_ID

	# Recommended by hadolint
	# https://github.com/hadolint/hadolint/wiki/DL4006
	SHELL ["/bin/bash", "-o", "pipefail", "-c"]

	# Install dependencies
	RUN apt-get update -qq \
	&& apt-get install -yq --no-install-recommends \
	# ... \
	# ... \
	# ... \
	&& apt-get clean \
	&& rm -rf /var/cache/apt/archives/* \
	&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
	&& truncate -s 0 /var/log/*log

	# Upgrade RubyGems and install required Bundler version
	RUN gem update --system --no-document \
	&& gem install bundler:$BUNDLER_VERSION --no-document

	# Add non-root user
	RUN groupadd --gid $USER_ID $USER \
	&& useradd --uid $USER_ID --gid $USER --shell /bin/bash --create-home $USER

	# Create a directory for the app code
	RUN mkdir /app \
	&& chown -R $USER:$USER /app

	WORKDIR /app


	###
	# Install Ruby gems, copied into the prod_image
	###
	FROM development AS prod_gems

	ENV BUNDLE_PATH=/usr/local/bundle
	ENV GEM_HOME=/usr/local/bundle

	# COPY Gemfile and Gemfile.lock for bundler to use
	COPY --chown=$USER:$USER Gemfile* ./

	# Tell bundler we're building for production
	RUN bundle config set deployment true \
	&& bundle config set without development test \
	&& bundle install


	###
	# Final production image
	###
	FROM ruby:$RUBY_VERSION-slim-buster AS production

	# Args needed for this container
	ARG POSTGRES_MAJOR_VERSION
	ARG USER
	ARG USER_ID
	ARG REVISION

	ENV RAILS_ENV=production
	ENV BUNDLE_PATH=/usr/local/bundle
	ENV GEM_HOME=$BUNDLE_PATH

	# Add non-root user
	RUN groupadd --gid $USER_ID $USER \
	&& useradd --uid $USER_ID --gid $USER --shell /bin/bash --create-home $USER

	# Recommended by hadolint
	# https://github.com/hadolint/hadolint/wiki/DL4006
	SHELL ["/bin/bash", "-o", "pipefail", "-c"]

	# Install dependencies
	RUN apt-get update -qq \
	&& apt-get install -yq --no-install-recommends \
	# ... \
	# ... \
	# ... \
	&& apt-get clean \
	&& rm -rf /var/cache/apt/archives/* \
	&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
	&& truncate -s 0 /var/log/*log

	# Create a directory for the app code
	RUN mkdir /app \
	&& chown -R $USER:$USER /app

	WORKDIR /app

	# COPY in files built during earlier stages
	COPY --chown=$USER:$USER --from=prod_gems $BUNDLE_PATH $BUNDLE_PATH

	# Add the app code
	COPY --chown=$USER:$USER . ./

	# Set user to non-root $USER
	# This needs to be the numeric uid, not the username, for the k8s
	# securityContext: runAsNonRoot check to work.
	USER $USER_ID

	# Env vars for production
	ENV RAILS_ENV=production
	ENV NODE_ENV=production
	ENV REVISION=$REVISION

	# Start web server
	# bundle exec puma -C config/puma.rb
	CMD ["bundle", "exec", "puma", "-C", "config/puma.rb"]