Shipping Docker Desktop logs to Elasticsearch using Filebeat

docker-desktop-filebeat.md

It's sometimes really useful to ship the logs of Docker Desktop to Elasticsearch. This gist describes the required steps. They need to be performed in your favorite WSL2 distro.

1. Mount the UNC path of Docker Desktop:

sudo mkdir /mnt/docker-desktop-data
sudo mount -t drvfs '\\wsl$\docker-desktop-data' /mnt/docker-desktop-data/

2. Create a folder for Filebeat. Here it's created under /tmp:

cd /tmp
mkdir -p filebeat/data filebeat/config
cd filebeat

3. In the filebeatfolder, create the file docker-compose.yml with the following content:

version: "3.3"
services:
  filebeat:
    image: "elastic/filebeat:8.1.3"
    container_name: filebeat
    volumes:
      - "/tmp/filebeat/config/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro"
      - "/tmp/filebeat/data:/usr/share/filebeat/data"
      - "/mnt/docker-desktop-data/version-pack-data/community/docker/containers:/var/lib/docker/containers:ro"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    user: root
    environment:
      - "ELASTICSEARCH_HOSTS=yourelasticsearchhost:9200"
      - "ELASTICSEARCH_USERNAME=yourusername"

A few notes:

• The container runs as root, to access the Docker socket
• Enter your Elasticsearch hostname
• Enter the name of the Elasticsearch user you created in Elasticsearch

4. In the filebeat/config folder, create the file filebeat.yml with the following content:

filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

processors:
- add_cloud_metadata: ~

output.elasticsearch:
  hosts: '${ELASTICSEARCH_HOSTS:elasticsearch:9200}'
  username: '${ELASTICSEARCH_USERNAME:}'
  password: '${ELASTICSEARCH_PASSWORD:}'

5. Make the directory filebeat with everything in it owned by root:

sudo chown -R root /tmp/filebeat

6. Create the keystore with the password of the Elasticsearch user:

docker-compose run --rm filebeat keystore create
docker-compose run --rm filebeat keystore add ELASTICSEARCH_PASSWORD

7. Start Filebeat:

docker-compose up -d