Skip to content

Instantly share code, notes, and snippets.

@abohmeed
Last active October 10, 2020 14:21
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save abohmeed/fb384aa2194aed551aca7aef66114148 to your computer and use it in GitHub Desktop.
Save abohmeed/fb384aa2194aed551aca7aef66114148 to your computer and use it in GitHub Desktop.
fluentbit configmap for rewrite_tag
apiVersion: v1
kind: ConfigMap
metadata:
name: fluent-bit-config
labels:
k8s-app: fluent-bit
data:
# Configuration files: server, input, filters and output
# ======================================================
fluent-bit.conf: |
[SERVICE]
Flush 1
Log_Level info
Daemon off
Parsers_File parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 2020
@INCLUDE input-myapp.conf
@INCLUDE es-output.conf
input-myapp.conf: |
[INPUT]
Name tail
Tag myapp.*
Path /var/log/containers/*myapp*.log
DB /var/log/myapp_flb_kube.db
Buffer_Chunk_Size 5MB
Buffer_Max_Size 5MB
Mem_Buf_Limit 25MB
Skip_Long_Lines On
Refresh_Interval 10
filter-kubernetes.conf: |
[FILTER]
Name kubernetes
Match myapp.web.*
Kube_URL https://kubernetes.default.svc:443
Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
Merge_Log On
Merge_Log_Key log_processed
K8S-Logging.Exclude On
Labels On
Annotations On
K8S-Logging.Parser Off
Regex_Parser myapp-k8s-custom-tag
[FILTER]
Name kubernetes
Match myapp.app.*
Kube_URL https://kubernetes.default.svc:443
Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
Kube_Tag_Prefix myapp.var.log.containers.
Merge_Log On
Merge_Log_Key log_processed
K8S-Logging.Exclude On
Labels On
Annotations On
K8S-Logging.Parser Off
Regex_Parser myapp-k8s-custom-tag
parsers.conf: |
[PARSER]
Name myapp-k8s-custom-tag
Format regex
Regex (?<tag>[^.]+)?\.?(?<pod_name>[a-z0-9](?:[-a-z0-9]*[a-z0-9])?(?:\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<namespace_name>[^_]+)_(?<container_name>.+)-(?<docker_id>[a-z0-9]{64})\.log$
es-output.conf: |
[OUTPUT]
Name es
Match myapp.app.*
Host localhost
Port 9200
Logstash_Format On
Logstash_Prefix myapp-app-logs
Retry_Limit 15
[OUTPUT]
Name es
Match myapp.web.*
Host localhost
Port 9200
Logstash_Format On
Logstash_Prefix myapp-web-logs
Retry_Limit 15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment