afolarin/docker-log-gist.md

## docker-log-gist.md

      
    Raw
  

              docker-log-gist.md
            
          
    #Some discussions on logging from docker:
Using logstash
Using Papertrail
Issue Loging Drivers
Issue Enhanced Logging
A lot of this boils down to whether you want a single or multi-process (systemd, supervisord etc.) container...
#Getting at the container's merged logs
Getting logs is still not something perfected in docker, and remains a little clunky.
Let's create a container named "foo"
me@host~$ docker run --name foo ubuntu /bin/bash -c 'while true; do echo "I am up.."; sleep 2; done' 

docker-logs CLI command merges both stdout and stderr into one stream:
me@host~$ docker logs foo

##Split the docker-logs
You can then simply split the stdout & stderr of the container by piping the separate streams:
and send them to files:
me@host~$ docker logs foo > stdout.log 2>stderr.log
me@host~$ cat stdout.log
me@host~$ cat stderr.log

or split stdout and error to host stdout:
me@host~$ docker logs foo > -
me@host~$ docker logs foo 2> -

#Pass a contained application's log
There are a few ways to get at a given application's log files, they either involve either getting these specific log files to the containers stderr|stdout stream or to mount the log file's dir as a volume.
If we were running something in like a webserver
me@host~ docker run --name foo -d -p 80:80 dockerfile/nginx

##Method 1
while this is perhaps possible, I don't recommend it. This just tails the log file to stdout or stderr:
me@foo~# tail --pid $$ -F /var/log/nginx/access.log &

##Method 2
BETTER WAY
This way involves creating links  /dev/stdout|stderr as /app/log.file. The ngix can write to its logfile, as a file, but as a result the lines will go to stdout & stderr (but can be split see above).
e.g. ngix dockerfile https://github.com/nginxinc/docker-nginx/blob/master/Dockerfile
RUN ln -sf /dev/stdout /var/log/nginx/access.log 
RUN ln -sf /dev/stderr /var/log/nginx/error.log

##Access the logs on a host mounted directory
The alternative would be to mount a host directory as the log folder and then access the log files directly on the host.
me@host~$ docker run -d -p 80:80 -v <sites-enabled-dir>:/etc/nginx/sites-enabled -v <certs-dir>:/etc/nginx/certs -v <log-dir>:/var/log/nginx dockerfile/nginx

me@host~$ ls <log-dir> 

#Inject a log monitor process with docker-exec
While it is not generally considered a good idea to run multi-process containers, from of docker version 1.3, it is possible to insert a secondary process into a container, using docker-exec. So it is then possible to attach a process, to access arbitrary log files. Once a second process is inserted, you can see this with <ps aux> in the container.
interactively:
me@host~$ docker exec -it foo /bin/bash
me@foo~ # cat /some/log/file


or singularly:
me@host~$ docker exec -t foo cat /some/log/file

You can pretty much take your pick of how you now access the container. Though it might be best to exercise a bit of process hygiene.
#Dedicated syslog container
One from Jerome. http://jpetazzo.github.io/2014/08/24/syslog-docker/  https://github.com/jpetazzo/syslogdocker
He uses 3 containers, but you can dispense with the inspection container if you are using docker v1.3 as you can simply inject a shell in to inspect the /var/log file.

Build the syslog container:

FROM ubuntu:14.04
RUN apt-get update -q
RUN apt-get install rsyslog
CMD rsyslogd -n
VOLUME /dev
VOLUME /var/log

`docker build -t syslog .`


syslog Container-- Run the syslog container, here all the logs will be gathered, when syslog starts it creates /dev/log which will now be in the mounted host dir /tmp/syslogdev:
docker run  --name syslog -d -v /tmp/syslogdev:/dev syslog


Log generating container -- Start another container to send logs:
docker run --name foo -v /tmp/syslogdev/log:/dev/log ubuntu logger -t "foo says" hello


(a) Inspection Container -- Tail them logs:
docker run --name seelog --volumes-from syslog ubuntu tail -f /var/log/syslog


(b) Alternatively, as of docker v1.3 use the docker-exec command to inspect syslog container directly
docker exec -t syslog tail -f /var/log/syslog


See in the log message from "foo", show up in the "seelog" container.


#TODO...

 basic log tutorial.
 syslog
[] logrotate
[] aggregation services e.g. Fluentd, Papertrail, Logstash
[] discuss further|better ways to use docker-exec