Getting Started with Elastic Cloud on K8S

README.md

Quickstart Elastic Cloud on Kubernetes

Starting up a minikube for playing

minikube start -p eck --memory 8192 --cpus 4 --disk-size 50g

The Original Quickstart docs

here

Quick start everything

Install the Official CRDs and the ECK Operator, and spawn the quickstart ES with a Kibana:

kubectl apply -f https://download.elastic.co/downloads/eck/1.1.0/all-in-one.yaml
kubectl apply -f pv-standard.yaml
kubectl apply -f quickstart-es+kibana.yaml

Check if ES and Kibana and up and ready

$ kubectl get es
NAME         HEALTH   NODES   VERSION   PHASE   AGE
quickstart   green    1       7.6.2     Ready   72m
$ kubectl get kibana
NAME         HEALTH   NODES   VERSION   AGE
quickstart   green    1       7.6.2     73m

Check that you can reach ES locally

Fetch the password from the minikube:

PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')

Start a local kube proxy to reach the ES locally:

kubectl port-forward service/quickstart-es-http 9200

Issue a check GET request to the ES cluster via the local kube proxy:

curl -u "elastic:$PASSWORD" -k "http://localhost:9200"
{
  "name" : "quickstart-es-default-0",
  "cluster_name" : "quickstart",
  "cluster_uuid" : "7QUV_Hn1RnmOsLAHjxknRg",
  "version" : {
    "number" : "7.6.2",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-03-26T06:34:37.794943Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Port forward the Kibana locally and you can reach the Dashboard

kubectl port-forward service/quickstart-kb-http 5601
Forwarding from 127.0.0.1:5601 -> 5601
Forwarding from [::1]:5601 -> 5601

And now open Kibana on https://localhost:5601

Rumblings with fileBeat: Setup in k8s

Let's use the ES cluster we just created with the name quickstart. Setup fluentBit to use this ES cluster as an output.

es cluster url	elastic user	password secret name	password secret key
quickstart-es-http.default.svc.cluster.local	elastic	quickstart-es-elastic-user	elastic

Setup as DaemonSet to forward all container outputs to ES

See the official docs

First fetch the official fileBeat manifest:

curl -L -O https://raw.githubusercontent.com/elastic/beats/7.6/deploy/kubernetes/filebeat-kubernetes.yaml

Now you need to modify the manifest to configure the ES credentials. See the diff below:

--- filebeat-kubernetes.yaml    2020-05-04 00:28:40.000000000 +0200
+++ filebeat-eck.yaml   2020-05-05 22:15:32.000000000 +0200
@@ -71,13 +71,17 @@
         ]
         env:
         - name: ELASTICSEARCH_HOST
-          value: elasticsearch
+          value: quickstart-es-http.default.svc.cluster.local
         - name: ELASTICSEARCH_PORT
           value: "9200"
         - name: ELASTICSEARCH_USERNAME
           value: elastic
         - name: ELASTICSEARCH_PASSWORD
-          value: changeme
+          valueFrom:
+            secretKeyRef:
+              optional: false
+              name: quickstart-es-elastic-user
+              key: elastic
         - name: ELASTIC_CLOUD_ID
           value:
         - name: ELASTIC_CLOUD_AUTH

Apply the modified manifest, and filebeat should startup and ship logs to ES.

kubectl apply -f filebeat-eck.yaml

Setup as a sidecar into your POD to ship logs from a file

So I have a k8s manifest prepared which which runs an ubuntu POD which just echoes a Hello message into it's stdout and into a file on a volume (emptyDir) which is configured to fileBeat as an input for log files.

kubectl apply -f filebeat-sidecar.yaml

filebeat-sidecar.yaml

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
data:
  filebeat.yml: |-
    filebeat.inputs:
      - type: log
        enabled: true
        paths:
          - /var/log/sharedlog/*.log

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
      # index: "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}"
      username: ${ELASTICSEARCH_USERNAME:elastic}
      password: ${ELASTICSEARCH_PASSWORD}

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: logger-config
data:
  LOGFILE: "/var/log/sharedlog/testbeat.log"
  SLEEPTIME: "5"
  MESSAGE: "HELLO fileBeat!"
---
apiVersion: v1
kind: Pod
metadata:
  name: testbeat
spec:
  volumes:
    - name: sharedlog
      emptyDir: {}
    - name: config
      configMap:
        defaultMode: 0600
        name: filebeat-config
  containers:
    - name: logger
      image: ubuntu
      command: 
        - bash
        - '-c'
        - 'while true; do echo ${MESSAGE:-"Hello World"} | tee -a ${LOGFILE:-"/var/log/sharedlog/testbeat.log"}; sleep ${SLEEPTIME:-10}; done' 
      envFrom:
        - configMapRef:
            name: logger-config
      volumeMounts:
        - name: sharedlog
          mountPath: /var/log/sharedlog
    - name: filebeat
      image: docker.elastic.co/beats/filebeat:7.6.2
      args:
        - "-c"
        - "/etc/filebeat.yml"
        - "-e"
      env:
        - name: ELASTICSEARCH_HOST
          value: quickstart-es-http.default.svc.cluster.local
        - name: ELASTICSEARCH_USERNAME
          value: elastic
        - name: ELASTICSEARCH_PASSWORD
          valueFrom:
            secretKeyRef:
              optional: false
              name: quickstart-es-elastic-user
              key: elastic
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
      securityContext:
        runAsUser: 0
      resources:
        limits:
          memory: 200Mi
        requests:
          cpu: 100m
          memory: 100Mi
      volumeMounts:
        - name: config
          mountPath: /etc/filebeat.yml
          readOnly: true
          subPath: filebeat.yml
        - name: sharedlog
          mountPath: /var/log/sharedlog

pv-standard.yaml

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: task-pv-volume-0
  labels:
    type: local
spec:
  storageClassName: standard
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"

quickstart-es+kibana.yaml

---
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 7.6.2
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  nodeSets:
  - name: default
    count: 1
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.store.allow_mmap: false
---
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: quickstart
spec:
  version: 7.6.2
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  count: 1
  elasticsearchRef:
    name: quickstart