Last active
July 13, 2018 12:23
Star
You must be signed in to star a gist
EFK
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sed -e "s/?ELASTICSEARCH?/$ELASTICSEARCH/g" -e "s/?ELASTICINDEXPREFIX?/$ELASTICINDEXPREFIX/g" < /fluentd/etc/elastic.conf.template > /fluentd/etc/elastic.conf |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
version: "2" | |
services: | |
fluentd: | |
build: ../fluentd | |
ports: | |
- "24224:24224" | |
environment: | |
- ELASTICSEARCH | |
- ELASTICINDEXPREFIX | |
command: /bin/sh -c "/fluentd/config.sh && fluentd -c /fluentd/etc/fluent.conf" | |
service1: | |
depends_on: | |
- fluentd | |
image: someimage:tag | |
environment: | |
- ELASTICSEARCH | |
- ELASTICINDEXPREFIX | |
logging: | |
driver: fluentd | |
options: | |
fluentd-address: localhost:24224 | |
tag: 'docker.{{.ImageName}}.{{.Name}}.{{.ID}}' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FROM fluent/fluentd:v0.12.34 | |
USER root | |
COPY ./etc/* /fluentd/etc/ | |
COPY config.sh /fluentd/ | |
RUN apk add --update --virtual .build-deps \ | |
sudo build-base ruby-dev \ | |
# cutomize following instruction as you wish | |
&& sudo -u fluent gem install \ | |
fluent-plugin-elasticsearch \ | |
&& sudo -u fluent gem sources --clear-all \ | |
&& apk del .build-deps \ | |
&& rm -rf /var/cache/apk/* \ | |
/home/fluent/.gem/ruby/2.3.0/cache/*.gem | |
USER fluent | |
EXPOSE 24224 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@type elasticsearch | |
logstash_format true | |
hosts ?ELASTICSEARCH? #(optional; default="localhost") | |
#port #(optional; default=9200) | |
logstash_prefix ?ELASTICINDEXPREFIX? #(optional; default=logstash) | |
include_tag_key true | |
flush_interval 5 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<source> | |
@type forward | |
port 24224 | |
</source> | |
<filter fluent.**> | |
@type record_transformer | |
<record> | |
fields.module fluentd | |
</record> | |
</filter> | |
<match fluent.**> | |
@include elastic.conf | |
</match> | |
# JSON-Parse | |
<filter **> | |
@type parser | |
@log_level trace | |
format json | |
key_name log | |
hash_value_field fields | |
</filter> | |
#Don't override anything if no further parts of the tag are given or they say 'no'! | |
#docker.<UID> is the default tag. No override! | |
<match logging logging.no logging.no.no docker.** *> | |
@include elastic.conf | |
</match> | |
<filter logging.no.*> | |
@include overrideuuid.conf | |
</filter> | |
<match logging.no.*> | |
@include elastic.conf | |
</match> | |
<filter logging.* logging.*.no> | |
@include overridemodule.conf | |
</filter> | |
<match logging.* logging.*.no> | |
@include elastic.conf | |
</match> | |
<filter logging.**> | |
@include overridemodule.conf | |
@include overrideuuid.conf | |
</filter> | |
<match logging.**> | |
@include elastic.conf | |
</match> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@type record_transformer | |
<record> | |
fields.module ${tag_parts[1]} | |
</record> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@type record_transformer | |
<record> | |
fields.uuid ${tag_parts[2]} | |
</record> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Folder structure: | |
- fluentd | |
- etc | |
- elastic.conf.template | |
- fluent.conf | |
- overridemodule | |
- overrideuuid.conf | |
- Dockerfile | |
- config.sh | |
- deploy | |
- docker-compose.yml |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment