Privileged Netobs Agent WIP

README.md

Problem: the privileged pod can be correctly deployed in OpenShift but the netobserv-agent fails in Kubernetes when it tries to remove the memory lock, internally executing the prlimit syscall. The received error is "operation not permitted".

time="2022-04-06T11:04:27Z" level=fatal msg="can't start netobserv-agent"
error="removing mem lock: failed to set memlock rlimit: operation not permitted"

We have tested it in Kind and Minikube. Maybe we should try it in another Vanilla K8s distribution.

k8s_nonworking_ns_permissions.yml

# This approach doesn't work. As PodSecurityPolicy are deprecated, the documentation
# recommends just tagging the namespace. But also fails with the same permission error as in the previous case
apiVersion: v1
kind: Namespace
metadata:
  name: network-observability
  labels:
    pod-security.kubernetes.io/enforce: privileged
    pod-security.kubernetes.io/audit: privileged

k8s_nonworking_permissions.yml

# This doesn't work in K8s to deploy the netobserv-agent (tried only in Kind and Minikube)
# The pod is deployed but crashes due to lack of privileges, as if the "privileged: true" option of the container
# didn't take effect. Also tried adding manual capabilities
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: netobserv-agent
  namespace: network-observability
---
kind: PodSecurityPolicy
apiVersion: policy/v1beta1
metadata:
  name: netobserv-agent
  namespace: network-observability
spec:
  privileged: true
  hostNetwork: true
  allowPrivilegeEscalation: true
  defaultAllowPrivilegeEscalation: true
  fsGroup:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  defaultAddCapabilities:
    - IPC_LOCK
    - SYS_ADMIN
    - NET_BIND_SYSTEM
    - SYS_RESOURCE
  allowedCapabilities:
    - IPC_LOCK
    - SYS_ADMIN
    - NET_BIND_SYSTEM
    - SYS_RESOURCE
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: netobserv-agent
  namespace: network-observability
rules:
  - apiGroups: ['policy']
    resources: ['podsecuritypolicies']
    verbs:     ['use']
    resourceNames:
      - netobserv-agent
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: netobserv-agent
  namespace: network-observability
roleRef:
  kind: Role
  name: netobserv-agent
  apiGroup: rbac.authorization.k8s.io
subjects:
  # Authorize all service accounts in a namespace (recommended):
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: system:serviceaccounts:network-obseravbility
  - kind: ServiceAccount
    name: netobserv-agent
    namespace: network-observability

openshift_permissions.yml

# This works perfectly in Openshift to deploy the netobserv-agent
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: netobserv-agent
---
apiVersion: security.openshift.io/v1
kind: SecurityContextConstraints
metadata:
  name: example
allowPrivilegedContainer: true
allowHostDirVolumePlugin: true
allowHostNetwork: true
allowHostPorts: true
runAsUser:
  type: RunAsAny
seLinuxContext:
  type: RunAsAny
users:
  - system:serviceaccount:network-observability:netobserv-agent