running sytest against live dendrite

post.md

First run a dendrite instance in vscode debugger (or other debugger of choice)

docker run --rm -it \
    -v "$PWD:/sytest" \
    -v "$PWD/../dendrite:/dendrite" \
    -e TIMEOUT_FACTOR=1000000 \
    matrixdotorg/sytest:local \
    /bin/bash -c "cd /sytest && \
        ./install-deps.pl && \
        ./run-tests.pl \
            -I Manual \
            --location http://host.docker.internal:8008 \
            --server-name hs1.local \
            -W /dendrite/sytest-whitelist \
            -B /dendrite/sytest-blacklist \
            --exclude-deprecated \
            -s \
            -C \
            --room-version org.matrix.msc4014 \
            tests/30rooms/06invite.pl"

• docker run --rm -it \
  • run this container and delete it afterwards etc.
• -v "$PWD:/sytest" \
  • mounts sytest to /sytest in the container (assuming this command is run in sytest repo, change if not)
• -v "$PWD/../dendrite:/dendrite" \
  • mounts dendrite to /dendrite in the container (assuming ../dendrite is a dendrite repo, change if not)
• -e TIMEOUT_FACTOR=1000000 \
  • by default sytest times out after ~10 seconds, but stepping through debuggers takes a while, so this makes it very long
• matrixdotorg/sytest:local \
  • sytest container, this can be pulled from docker hub by changing :local, but you can also build sytest locally (with base.Dockerfile)
• /bin/bash -c "cd /sytest && \
• ./install-deps.pl && \
  • I think this needs to be run, not entirely sure what it does beyond probably installing dependencies
• ./run-tests.pl \
• -I Manual \
  • using a manual homeserver, as this is connecting to a separate homeserver instance
• --location http://host.docker.internal:8008 \
  • to access localhost of the host machine, you use host.docker.internal, change the port as needed
• --server-name hs1.local \
  • server name of your local homeserver
• -W /dendrite/sytest-whitelist \
• -B /dendrite/sytest-blacklist \
• --exclude-deprecated \
  • these three options are to mimic how ./run-sytest.sh runs tests
• -s \
  • this option makes sytest stop on first fail, depending on the tests you're running, this can be handy for trimming log output when also using -C, you may want to remove this
• -C \
  • this option prints out all http req/res, which can be handy in debugging, you might want to remove this
• --room-version org.matrix.msc4014 \
  • run with pseudo IDs room version, you may want to change this to 10 if you're comparing differences
• tests/30rooms/06invite.pl"
  • the specific tests you want to run

problems

• some tests create users or aliases with hardcoded values, and so you might want to consider clearing your local dendrite's database before running (in vscode I did this with a preLaunchTask).