Obsidian Header Linking Bug

__Obsidian Header Linking Bug

-

docker and docker-compose.md

tags	uid
docker main moc	58ee12f7-54c8-4db0-9418-0b1e532355bb

[!Warning] The docker-compose command is deprecated As of V2, docker-compose is now part of docker proper. Any commands should work simply by removing the dash (-).

However, as of Sep 2023, Docker on Synology still does not support the newer command format.

[!success] See here for help switching from snap to apt version Docker containers running, but not showing up in docker ps

Links

• Compose
• Compose file - default: compose.yaml
• Versions and upgrading
• Use secrets in Docker Compose

[!tip] Filename for Docker compose YAML config file The default path for a Compose file is compose.yaml (preferred) or compose.yml that is placed in the working directory. Compose also supports docker-compose.yaml and docker-compose.yml for backwards compatibility of earlier versions. If both files exist, Compose prefers the canonical compose.yaml.

Real-world Uses of Docker Compose

• [[Run periodic speedtests and graph historic data|speedtest-tracker]]
• [[BookStack]]
• [[Docker on Ubuntu20 - Gatus|Gatus]]

Concepts

• Data/changes are persistent by default, but will be lost when the container is rebuilt or updated
• For storing data that should persist across rebuilds, use volumes
• Images are built with Dockerfiles

Environment Variables

• Environment variables precedence in Docker Compose
• Set environment variables in Compose (env files)
• Using .env files
• How to setup environment variables in configuration
• [[this note will link to a heading in the long note#Define Environment Variables]]

[!bug] Don't try to edit vars after containers are built You can only set up variables during container creation (via -e FOO=BAR) It is not really feasible or recommended to change them after.

Generating Compose Files (compose.yml)

To paste a compose file in from Terminal

cat <<'EOF' >compose.yml

(...YAML...)

EOF

Dockerfiles

• Dockerfile reference
• Difference between COPY and ADD in a Dockerfile (TL;DR: use COPY)

Troubleshooting

• Docker containers running, but not showing up in docker ps
• I lose my data when the container exits

Volumes

• Volumes
• Bind mounts
• Volumes top-level element
• docker volume ls
• docker volume inspect

[!tip] Default location for Volumes (aka Root Dir) On default Docker installs (Linux) Named Volumes are typically stored at /var/lib/docker/volumes. On Synology however, you'll find them at /volume1/@docker/volumes. Bind mounts on the other hand can be placed anywhere.

To programmatically determine the root dir, you can execute docker info -f "{{.DockerRootDir}}/volumes"

[!example] Auto-creation of BIND mounts Recent versions of docker-compose support automatic creation of missing paths, by specifying the following directives in the compose file:

volumes:
  - type: bind
    source: /volume1/docker/foo
    target: /data
    bind:
      create_host_path: true

N.B.—this doesn't currently seem to work on Synology!

docker inspect -f '{{ json .Mounts }}' name | jq
docker inspect -f '{{ .Mounts }}' name
docker inspect <name> | jq -r '.[].Mounts[].Source'
docker inspect foo | jq -r 'map(.Mounts[] | [.Type, .Source] )[] | @tsv'

docker container ls -q | while read id; do docker inspect $id | jq -r '.[].Mounts[].Source'; done
docker container ls -q | while read id; do docker inspect $id -f '{{ .Mounts }}'; done

docker volume ls ...
docker volume ls --format "{{.Name}} {{.Mountpoint}}"

docker ps -aq
docker ps -a --format '{{ .ID }}' 

Networking

• Networking in Compose
• Networks top-level element
• Services top-level element
• Services → Networks

Status of Single Container

docker inspect -f '{{.State.Status}}' foo1

Logs - Tailing

• Read the daemon logs

journalctl -xu docker.service
docker logs -f container-name
docker events --filter name=container

Debugging a Container that won't start

docker run -it pairdrop sh
/usr/local/bin/docker-entrypoint.sh

Synology

[!question] on Synology, Docker Compose configs are called "Projects"

[!warning] Updating docker-compose binary Synology comes with an outdated version of docker-compose. You can download the latest from here and then copy it to /usr/local/bin (find your architecture with uname -m)

# /usr/local/bin/docker-compose version
Docker Compose version v2.9.0-6413-g38f6acd
# docker-compose version
Docker Compose version v2.9.0-6413-g38f6acd
# ./docker-compose-linux-x86_64 version
Docker Compose version v2.21.0

Screenshots of setting up a Project

![[IMG-20230831100452447.png|500]]

![[IMG-20230831100452601.png|500]]

![[IMG-20230831100452648.png|500]]

![[IMG-20230831100452734.png|500]]

![[IMG-20230831100452823.png|500]]

![[IMG-20230831100452894.png|500]]

![[IMG-20230831100435117.png|300]]

![[IMG-20230831100452948.png|500]]

![[IMG-20230831100453007.png|500]]

this note will link to a heading in the long note.md

#selfhosted #docker #synology #speedtest

Links

• local instance, running on nas
• alexjustesen/speedtest-tracker
• Docker pulls from here
• Docs
• Installation
• Synology-specific
• FAQ
• [[Docker - container network troubleshooting]]
• [[docker and docker-compose#Environment Variables]]
• Install video - DBTech

Abstract

Speedtest Tracker is a self-hosted internet performance tracking application that runs speedtest checks against Ookla's Speedtest service.

Installation

[!warning] Starting with v0.11.18, SQLite is deprecated. Use MariaDB. See #669. This section will be updated as the docs are updated.

Set up Networking

[!danger] Required for proper inter-container DNS resolution Setting up a dedicated network and not using the default bridge is also a best-practice.

docker network create --driver bridge prod

Create low-privilege user

• username: container
• pw: Startling6$Enamel

![[IMG-20230830101312452.png|550]]

Make sure to grant r/w access to the docker dir ![[IMG-20230831110629567.png|550]]

Docker Compose

[!question] On Synology, Docker Compose apps are known as Projects

[!bug] You should be using secrets in your .env files! Instead of storing passwords directly in the .env file, you should use secrets. #tbd

Define Environment Variables

see [[docker and docker-compose#Environment Variables]] (==this link should take you to "Environment Variables" but often will just jump you to the top of the note==)

Setup

[!bug] You CAN set up the containers via the GUI, but... Synology doesn't process .env files during creation, so you need to generate the compose.yml file manually using docker-compose config or other custom build scripts.

1. create docker compose dir to hold the configs

   ssh nas 'mkdir -p /volume1/docker/compose/speedtest-tracker'

2. scp the files—pop open this folder then open a Terminal

   scp -O template.yml generate-config nas:/volume1/docker/compose/speedtest-tracker/aa

3. Generate .env files and verify:

   ./generate-config
   docker-compose config

4. Continue in GUI...

Manually edit env and compose configs

1. mount nas with ShellFish.app
2. edit files as needed

Up! 🚀

1. Click Container Manager → Project → Create
   ![[IMG-20230831111443697.png|450]]
2. Name it speedtest
3. For Path, navigate to /docker/compose/speedtest-tracker
4. click Use existing an docker-compose.yml to create the project when prompted
5. Confirm settings look good
   ![[IMG-20230831121730193.png|450]]
6. Don't enable web portal
7. Start the project
   ![[IMG-20230831112000053.png|450]]

Or via the shell... (add -d to daemonize)

docker-compose up [-d]

Check that it's running

docker-compose ps

Down it (this will delete the containers, use stop/start to just stop them)

docker-compose down
docker-compose down --rmi local -v

---

Manual (no longer working)

MariaDB container

Get Started with MariaDB using Docker

1. download MariaDB image (Synology Container Mgr)
2. create /volume1/docker/mariadb dir
3. ssh to the nas, and execute the code below to set up the database:

docker run -d \
-p 3306:3306 \
-e PUID=${PUID} \
-e PGID=${PGID} \
-e TZ=${TZ} \
-e MYSQL_ROOT_PASSWORD=${DB_ROOT_PASS} \
-v /volume1/docker/mariadb:/var/lib/mysql \
--net prod \
--name mariadb \
--hostname mariadb \
--restart unless-stopped \
mariadb

Testing, accessing mdb shell

docker exec -it mariadb sh

docker exec -it mariadb \
mariadb --user root -p$DB_ROOT_PASS --auto-rehash

Set up speedtest db

docker exec -i mariadb mariadb -u root -p$DB_ROOT_PASS <<EOS
CREATE DATABASE $MARIADB_DATABASE;
GRANT ALL ON $MARIADB_DATABASE.* TO '$MARIADB_USER'@'localhost' IDENTIFIED BY '$MARIADB_PASSWORD';
FLUSH PRIVILEGES;
EOS

speedtest container

docker run -d \
-p 8080:80 \
-p 8443:443 \
-e PUID=${PUID} \
-e PGID=${PGID} \
-e TZ=${TZ} \
-e DB_CONNECTION=mysql \
-e DB_HOST=mariadb \
-e DB_PORT=3306 \
-e DB_DATABASE=${MARIADB_DATABASE} \
-e DB_USERNAME=${MARIADB_USER} \
-e DB_PASSWORD=${MARIADB_PASSWORD} \
-v /etc/localtime:/etc/localtime:ro \
-v /volume1/docker/speedtest-tracker:/config \
--net prod \
--name speedtest-tracker \
--hostname speedtest-tracker \
--restart unless-stopped \
ghcr.io/alexjustesen/speedtest-tracker:latest

---

Old docs... may not work anymore

1. Install Container Manager package
2. Open it
3. Click Registry
4. Search for ajustesen/speedtest-tracker
   ![[IMG-20230806142942566.png|500]]
5. Download it
6. Create a folder to store data e.g. /volume1/docker/speedtest-tracker
   ![[IMG-20230806143131536.png|500]]
7. Create (Run) new container
   ![[IMG-20230806144843305.png|500]]
   ![[IMG-20230806144756562.png|500]]
8. Map ports 8443 → 443, 8080 → 80, 9000 → 9000

[!tip] You can check for open ports with this 1-liner

for p in 8443 8080 9000; do porttest -p $p -t nas; done

9. Map volume: /volume1/docker/speedtest-tracker → /config:
   ![[IMG-20230806144711289.png|500]]
10. DON'T set Network to host:
    ![[IMG-20230807125201334.png|500]]
11. Confirm settings ![[IMG-20230806144444656.png|500]]
12. Up and Running!
    ![[IMG-20230806144340070.png|500]]
13. Log on to https://xxx.redacted:8443/
14. Default credentials are admin@example.com \ password

SMTP for alerts

see mail.php for config options, mailgun is recommended.

Edit the .env file at /volume1/docker/speedtest-tracker/.env and then restart the container:

MAIL_MAILER=smtp
MAIL_HOST=smtp.mailgun.org
MAIL_PORT=587
MAIL_USERNAME="xxx@redacted"
MAIL_PASSWORD=xxx.redacted
MAIL_ENCRYPTION=tls
MAIL_FROM_ADDRESS="xxx@redacted"
MAIL_FROM_NAME="${APP_NAME}"

Also set APP_URL and TZ

APP_URL=https://xxx.redacted
TZ=America/New_York

![[IMG-20230806151937319.png|500]]

![[IMG-20230806152003454.png|500]]

SSL

Use cloudflare tunnel...

![[IMG-20230806160459149.png]]

copy token...

xxx.redacted

![[IMG-20230806161236227.png|350]]

![[IMG-20230806161233530.png|500]]

![[IMG-20230806161433303.png|500]]

General settings, schedule

![[IMG-20230830094213853.png|500]]

• Cron: */30 * * * *
• Servers: (multiple)

Thresholds

300-300-100 seems about right...

![[IMG-20230831115250029.png|500]]

Note: iperf3 on Mac

iperf_start.sh
ssh r1 fwrule_toggle.php n6,27 enable

ssh r1 fwrule_toggle.php n6,27 disable

Video

header_linking.mp4