Ayon docker, install, setup and management notes

ayon_docker_notes.md

Note these are not any official notes - just random scribbles

Bootstrap

Running bootstrap wont give you the latest dev versions, only whats been released on the bootstrap server by the ynput team.

Updating it from time to time is a good way to see what progress has been made in terms of addons etc. It shouldnt remove old addon version, only add the new ones. That way you can still use older addons should you need. You would need to create a new bundle to accommodate that.

If you want to run the latest addons, you would need to grab that from the openpype github repository (as that is the current location of the ayon addons) then manually build those (instructions should be on the repository in the Server Addon README) then manually upload those newer addons to the ayon server. You can do this via the ayon web interface or command line should you need.

Extending session timeouts

Set e.g. this in the server section of docker-compose.yml:

environment:
  - "AYON_SESSION_TTL=604800"

Updating

Update ayon-docker, also see AYON Docs: Server Deployment - updates and comment: AYON Updating docker + picking the right tag

Latest:

docker pull ynput/ayon:latest
docker compose up -d --build

Dev:

docker pull ynput/ayon:dev
docker compose up -d --build

Before someone helps to debug, it would be good to make sure you are on the latest build. Run make update

Updating server addons

https://github.com/ynput/OpenPype/tree/19f0a77966dbc84898e48425b06015416c48eeef/server_addon#how-to-start

./.poetry/bin/poetry run python ./server_addon/create_ayon_addons.py

Admin

Cleanup / pruning / clearing docker caches

Generally its along the lines of docker buildx prune. Where buildx is the container, image, etc. Just be sure not to blow your volumes away!

Run docker system prune -a' on development machine as part of digital hygiene procedure. It tells you what it is going to delete (unused things). Make sure to keep stuff you want to keep running.

Dump entire database

docker compose exec postgres pg_dump -U ayon > backup.sql

Backing up + restoring

Source: https://discord.com/channels/517362899170230292/611571347058982927/1187432211067183174

• run docker compose exec postgres pg_dump -U ayon > backup.sql to create database dump
• copy storage/ docker-compose.yml and backup.sql to the new machine
• run docker compose up postgres on the new machine to start only the database (if you run the entire stack, it will be populated with defaults, we want to keep it empty for now)
• run docker compose exec -T postgres psql -U ayon ayon < backup.sql to restore the backup
• start everything using docker compose up

By the way, this method also allows upgrading the database server. If you dump the db storage and restore it on the target machine, you need to use the same postgres version, while using pg_dump converts everything in the database to a huge list of sql queries, which you can run on a newer postgres database.

Note: The exact commands may differ on Windows see here for a Powershell example:

If you are struggling with redirections like docker compose exec postgres pg_dump -U ayon > backup.sql.

In Powershell on windows it is docker compose exec postgres pg_dump -U ayon | Out-File backup.sql.

And the other way around like docker compose exec -T postgres psql -U ayon ayon < backup.sql.

In Powershell on windows is powershell cat backup.sql | docker compose exec -T postgres psql -U ayon ayon.

And bonus points - you may want to use best practice naming conventions for your backup files

Ftrack setup notes: https://community.ynput.io/t/ayon-beta-setup-with-ftrack-linux-rocky-8/739

Running AYON from network location for e.g. renderfarm

• AYON Community Forum - Render node pull Ayon launcher from network locations
• AYON Discord - Renders node pull Ayon launcher from network locations instead of local install

Updating AYON docker + picking the right tag (e.g. dev versus latest)

More details:

• Available AYON docker tags on Docker Hub: https://hub.docker.com/r/ynput/ayon/tags
• ayon-docker github repository
• ayon server deployment documentation

To update to a certain tag:

• set the docker docker-compose.yml to ynput/ayon:dev instead of ynput/ayon:latest
• docker pull ynput/ayon:dev - the available tags are here: https://hub.docker.com/r/ynput/ayon/tags
• docker compose up -d --build

Note that it's "not safe" to downgrade the server and thus usually going from dev to latest is not recommended because it may break things because the server can perform database "upgrades" going forward but cannot "downgrade" safely.

AYON getting started / onboarding discussions:

• https://discord.com/channels/517362899170230292/563751989075378201/1201502099188494336

Code snippet to generate an AYON API Key token for a normal user:

import ayon_api
connection = ayon_api.ServerAPI('<your_ayon_url>')
connection.login('<your_username>','<your_password>')
print(connection.access_token)

Source

Is there any way through the ayon API to copy settings between bundles? I want to automate uploading new addons, creating a new bundle using the new uploaded addons, copy settings from last production bundle, set the new bundle as production.
I sometimes release updates multiple times a day 😅 and it's a bit cumbersome with all the steps involved.

assuming you are updating addon foo from version 1.0.0 to 1.0.1:

• [GET] /api/addons/foo/1.0.0/settings?as=1.0.1 - loads settings of 1.0.0, but interpret them using settings model of 1.0.1 (do the best match and conversions if there are any). that returns a json
• which you then post here [POST] /api/addons/foo/1.0.1/settings.

Both endpoints also accept variant query argument (which defaults to production).

For project settings, it's the same thing, just use /api/addons/{addon_name}/{addon_version}/settings/{project_name}

Source on discord

The launcher logic caches where the current machine can find executables - and it will take from that cache if a matching version is found within it. This may be a local build which is not a deployed install - even when not running in dev mode.

This is the logic of ayon launcher:

• look what bundle should be used
• look which version of ayon-launcher should be used
• find in cached paths of ayon-launcher if the version is there
  • it is? just use it
  • it is not? try to download it, install, and use it

source on discord

So it may be that you're seeing an executable path listing a build instead of an installed path:

When finding it in the cache there should be a 'validation' occurring by means of checking the version.py file whether the version listed in the cache actually matches the current version at that path.

---

On Windows that local 'cache' is stored here: %LOCALAPPDATA%\Ynput\AYON\executables.json

Setting up AYON docker environment variables, e.g. for e-mail server

For more details on setting up Ayon server environment variables in the docker container see the discussion here

It is in the docker-compose.yml under services for the server add an environment section, e.g:

Special note:

btw: in order to restart the container with new env vars, you need to run docker compose up server --detach --build , not just make reload or docker compose restart server

---

Send e-mail from AYON addon

if you want to send mail from an addon, you can:

from ayon_server.helpers.email import send_mail
await send_mail(["someone@somewhere.com"], subject="meh", text="text version", html="html <b>version</b>")

or

user = await UserEntity.load("someone")
await user.send_mail("subject", "text_version", "html <b>version</b>")

Configuring the login failed retry attempts and timeouts

See discussion here.

Testing ayon-frontend PRs; how?

You need to have Node environment setup.

• Install Node (somehow) and then Yarn (npm install -g yarn)
• Go to the frontend directory and checkout the branch you need
• run yarn install to install dependencies
• run yarn dev to start the development server

By default development server listens on localhost:3000 and proxies all API request to localhost:5000 (so it works if you run ayon locally). But it is also possible to use .env file in the frontend directory to set .... SERVER_URL environment variable to something else: SERVER_URL=https://ayon.dev
so when you go to http://localhost:5000 you will access the frontend bundled with the image, but when you go to http://localhost:3000 you will access the version you have checked-out

Note: Development server is MUCH more verbose and with react devtools browser extension it allows you to trace a lot of potential issues.
You can also run yarn build that builds the production version to dist directory, that you can then mount as a volume to docker container's /frontend dir but i don't even remember doing that. never needed it

How to deploy with SSL?

• I got tipped this: https://medium.com/@manishgupta.ait/deploy-your-app-with-https-enabled-using-lets-encrypt-docker-docker-compose-bf2fdc981406

source

Fix "Always allow" prompt not showing for web actions in AYON in Chrome.

• See Discord thread.

This ended up working here:

Go to:

chrome://flags/#unsafely-treat-insecure-origin-as-secure

Enable it and add your server's URL

• Source

How to delete products? See on discord

How do I test an AYON server/backend Postgres SQL query like?

project_name = ""
path = ""
res = await Postgres.fetch(f"select id from project_{project_name}.hierarchy where path = $1", path.strip("/"))
if not res:
  raise NotFoundException()
folder = await FolderEntity.load(project_name, res[0]["id"])

sql schemas are in ayon-backend/schemas/schema.public.sql and ayon-backend/schemas/schema.project.sql.... you can access postgres shell using make dbshell command or expose a postgres port (5432) in docker-compose.yml and then access the db using a GUI tool of your choice 🙂

source