Instantly share code, notes, and snippets.

Embed
What would you like to do?
A manifest that describes "platform" agents that we run on every instance. See http://tech.grammarly.com/blog/posts/How-We-Deploy-Containers-at-Grammarly.html
# This is a manifest that describes "platform" agents that we run on every instance.
# It includes auxilary services that work alongside with the application containers
# to provide logging and monitoring.
#
# This file looks a bit tough, but that's what happens when you meet the reality.
# We have also pruned it a bit to remove our company specifics.
#
# You may notice "env", "role" and "track" variables. We specify them as EC2 instance
# tags and pass them to rocker-compose on every run. This allows us to distinguish
# instances and do basic service discovery and A/B testing.
#
# Examples of env: prod, qa, staging
# Examples of role: db, web, logs
# Examples of track: default, a, b, stable, canary
#
namespace: platform
containers:
# StatsD agent container. It uses `uts:host` to automatically get the
# hostname of the machine and use it for metrics prefixing.
statsd:
image: registry.company.com/platform/statsd:master
uts: host
# Passing environment variables to StatsD container
env:
GRAPHITE_HOST: graphite1.internal
GRAPHITE_PORT: "2013"
# StatsD uses these vars to construct metric names prefix.
ENV: {{ or .env "noenv" }}
ROLE: {{ or .role "norole" }}
TRACK: {{ or .track "default" }}
# StatD listens on 8125/udp port on the bridge interface,
# so by knowing `bridgeIp` other containers can write metrics to it.
ports:
- "8125:8125/udp"
expose:
- "8125/udp"
# Logstash-forwarder collects logs of platform and application services of local
# containers and forwards them to the remote logstash server.
logstash_fwd:
image: registry.company.com/platform/logstash-forwarder:master
uts: host
volumes:
- /mnt/log:/mnt/log
volumes_from:
- logstash_fwd_data
- logstash_fwd_conf
# We're persisting logstash-forwarder state, so after a restart it can resume from where it was stopped
logstash_fwd_data:
image: registry.company.com/scratch:latest
state: created
volumes:
- /etc/logstash-forwarder/data
# Logstash-forwarder config generator which listens for containers changes and for those with
# the "logs" label generates a corresponding configuration entry.
# So, once a container appears, we start collecting logs from it.
logstash_fwd_conf:
# This image is based on docker-gen, https://github.com/jwilder/docker-gen
image: registry.company.com/platform/logstash-forwarder-gen:master
cmd: ["-watch", "-restart-container", "platform.logstash_fwd", "/templates/logstash-forwarder.json.tmpl", "/etc/logstash-forwarder/conf/logstash-forwarder.json"]
env:
LOGSTASH_HOSTS: "logstash1.internal:5959,logstash2.internal:5959"
ENV: {{ or .env "noenv" }}
ROLE: {{ or .role "norole" }}
TRACK: {{ or .track "default" }}
volumes:
- /var/run/docker.sock:/tmp/docker.sock
# Here we use sensu to gather system metrics and send them to the local StatsD
sensu_client:
image: busybox:latest
cmd: ["/usr/sbin/chroot", "/host", "/etc/sensu/run.sh"]
volumes:
- /dev:/host/dev
- /run:/host/run
- /proc:/host/proc
- /var/run/:/host/var/run
- /:/host
env:
CLIENT_SUBSCRIPTIONS: system
SENSU_TYPE: client
# As mentioned above, StatsD is listening on the bridge interface so we can use bridgeIp helper
# provided by rocker-compose. This allows to decouple sensu client from StatsD
STATSD_HOST: {{ bridgeIp }}
volumes_from:
- sensu_rootfs
expose:
- "3030"
ports:
- "3030:3030"
uts: host
net: host
# Black magic: Image with sensu client executables and all required libraries, that is not overriden
# in sensu_client container by `volumes: /:/host`
sensu_rootfs:
image: registry.company.com/sensu_rootfs:v0.20.0-2
state: created
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment