A personal note on how to install Datadog on OpenShift

datadog_on_openshift.md

Install Datadog on OpenShift

1. First we want to create a namespace for the datadog agent:

oc new-project datadog

2. Once you create the namespace, lets create secrets to hold the appkey and apikey:

oc create secret generic datadog-app-key -n datadog --from-literal=app-key=<ADD_YOUR_APP_KEY_HERE>
oc create secret generic datadog-api-key -n datadog --from-literal=api-key=<ADD_YOUR_API_KEY_HERE>

3. Now create a values file, an example below with a few features enabled:

datadog:
  clusterName: <YOUR_CLUSTER_NAME>
  apiKeyExistingSecret: datadog-api-key
  appKeyExistingSecret: datadog-app-key
  site: datadoghq.com

  apm:
    portEnabled: true
  logs:
    enabled: true
    containerCollectAll: true
  processAgent:
    enabled: true
    processCollection: true
  dogstatsd:
    useHostPort: true
  kubeStateMetricsCore:
    enabled: true
  kubeStateMetricsEnabled: false
  kubelet:
    tlsVerify: false
  networkMonitoring:
    enabled: true
  securityAgent:
    compliance:
      enabled: true
    runtime:
      enabled: true

agents:
  tolerations:
  - key: node-role.kubernetes.io/master
    operator: Exists
    effect: NoSchedule
  podSecurity:
    securityContextConstraints:
      create: true

clusterAgent:
  enabled: true
  replicas: 2
  createPodDisruptionBudget: true
  confd:
    kube_controller_manager.yaml: |-
      cluster_check: true
      init_config:
      instances:
        - prometheus_url: https://kube-controller-manager.openshift-kube-controller-manager/metrics
          ssl_verify: false
          bearer_token_auth: true
          leader_election: false
    kube_scheduler.yaml: |-
      cluster_check: true
      init_config:
      instances:
        - prometheus_url: https://scheduler.openshift-kube-scheduler/metrics
          ssl_verify: false
          bearer_token_auth: true
    kube_apiserver_metrics.yaml: |-
      cluster_check: true
      init_config:
      instances:
        - prometheus_url: https://apiserver.openshift-kube-apiserver/metrics
          ssl_verify: false
          bearer_token_auth: true
    coredns.yaml: |-
      cluster_check: true
      init_config:
      instances:
        - prometheus_url: https://dns-default.openshift-dns:9154/metrics
          ssl_verify: false
          bearer_token_auth: true

The values above will enable a few things like security compliance, Logs, and APM. My personal preference for APM is to use via TCP port, therefore the portEnabled: true setting.

Also, adjust the number of replicas for the cluster agent. The values above we set 2 replicas.

4. Using HELM, lets install the datadog agent now the values.yaml file we created above:

helm repo add datadog https://helm.datadoghq.com
helm upgrade --install datadog datadog/datadog --namespace datadog -f values.yaml

5. After few minutes, you should see an output like the one below:

Get all pods (my example cluster has one node)

oc get pods

Output:

NAME                                     READY   STATUS    RESTARTS   AGE
datadog-9pnbg                            5/5     Running   0          12h
datadog-cluster-agent-674b887d95-6mx5q   1/1     Running   0          12h
datadog-cluster-agent-674b887d95-6tt7b   1/1     Running   0          12h

See the deployment:

oc get all -n datadog

Output:

NAME                                         READY   STATUS    RESTARTS   AGE
pod/datadog-9pnbg                            5/5     Running   0          12h
pod/datadog-cluster-agent-674b887d95-6mx5q   1/1     Running   0          12h
pod/datadog-cluster-agent-674b887d95-6tt7b   1/1     Running   0          12h

NAME                                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)             AGE
service/datadog                                      ClusterIP   10.217.5.63    <none>        8125/UDP,8126/TCP   12h
service/datadog-cluster-agent                        ClusterIP   10.217.5.4     <none>        5005/TCP            12h
service/datadog-cluster-agent-admission-controller   ClusterIP   10.217.5.195   <none>        443/TCP             12h

NAME                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/datadog   1         1         1       1            1           kubernetes.io/os=linux   12h

NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/datadog-cluster-agent   2/2     2            2           12h

NAME                                               DESIRED   CURRENT   READY   AGE
replicaset.apps/datadog-cluster-agent-674b887d95   2         2         2       12h
replicaset.apps/datadog-cluster-agent-84c5f4b96b   0         0         0       12h

The example above was created on a OpenShift CodeRedyContainer on my local laptop. If you are using a fully managed cluster such ROSA or ARO, then the settings may vary. I will be updating this note once I test on managed instances.