Skip to content

Instantly share code, notes, and snippets.

@dobesv
Created July 24, 2019 23:22
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dobesv/ffb27537fcc2df74ce8049fcfb937c4d to your computer and use it in GitHub Desktop.
Save dobesv/ffb27537fcc2df74ce8049fcfb937c4d to your computer and use it in GitHub Desktop.
Example configuration for fluentd-es namespace per index using record_modifier and elasticsearch
<filter kubernetes.**>
@type kubernetes_metadata
</filter>
# Use tag as index name prefix by default
<filter **>
@type record_modifier
<record>
_es_index ${tag_parts[0]}.${Time.at(time).strftime('%F')}
</record>
</filter>
# If kubernetes namespace available, use that as index name prefix
<filter kubernetes.**>
@type record_modifier
<record>
_es_index ${record['kubernetes']['namespace_name'] or 'kubernetes'}.${Time.at(time).strftime('%F')}
</record>
</filter>
<match **>
@id elasticsearch
@type elasticsearch
@log_level info
include_tag_key true
host elasticsearch
port 9200
logstash_format true
target_index_key _es_index
<buffer>
@type file
path /var/log/fluentd-buffers/kubernetes.system.buffer
flush_mode interval
retry_type exponential_backoff
flush_thread_count 2
flush_interval 5s
retry_forever
retry_max_interval 30
chunk_limit_size 2M
queue_limit_length 8
overflow_action block
</buffer>
</match>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment