Skip to content

Instantly share code, notes, and snippets.

@KMouratidis

KMouratidis/Other benchmarks.txt

Created October 22, 2025 18:13
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save KMouratidis/88456bea439ea8d38f452bb6df289b58 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save KMouratidis/88456bea439ea8d38f452bb6df289b58 to your computer and use it in GitHub Desktop.
Download ZIP
Beelink GTR9Pro (AI Max+ 395) benchmarks
Raw
llama-bench and power.txt
# Balanced mode (160-170W max)
~$ /opt/llmserver/llama.cpp/build/bin/llama-bench -fa 0,1 -m /opt/llmserver/models/unsloth/GLM-4.5-Air-GGUF/UD-Q4_K_XL/GLM-4.5-Air-UD-Q4_K_XL-00001-of-00002.gguf
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| glm4moe 106B.A12B Q4_K - Medium | 68.01 GiB | 110.47 B | ROCm | 99 | 0 | pp512 | 142.90 ± 1.39 |
| glm4moe 106B.A12B Q4_K - Medium | 68.01 GiB | 110.47 B | ROCm | 99 | 0 | tg128 | 20.31 ± 0.07 |
| glm4moe 106B.A12B Q4_K - Medium | 68.01 GiB | 110.47 B | ROCm | 99 | 1 | pp512 | 152.65 ± 1.49 |
| glm4moe 106B.A12B Q4_K - Medium | 68.01 GiB | 110.47 B | ROCm | 99 | 1 | tg128 | 20.83 ± 0.12 |
# Balanced mode (160-170W max)
~$ /opt/llmserver/llama.cpp/build/bin/llama-bench -fa 0,1 -m /opt/llmserver/models/unsloth/Qwen3-30B-A3B-GGUF/Qwen3-30B-A3B-UD-Q4_K_XL.gguf
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 16.49 GiB | 30.53 B | ROCm | 99 | 0 | pp512 | 496.63 ± 11.29 |
| qwen3moe 30B.A3B Q4_K - Medium | 16.49 GiB | 30.53 B | ROCm | 99 | 0 | tg128 | 63.26 ± 0.28 |
| qwen3moe 30B.A3B Q4_K - Medium | 16.49 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 503.25 ± 6.42 |
| qwen3moe 30B.A3B Q4_K - Medium | 16.49 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 64.43 ± 0.71 |
# Balanced mode (160-170W max)
~$ /opt/llmserver/llama.cpp/build/bin/llama-bench -fa 0,1 -m /opt/llmserver/models/unsloth/gpt-oss-120b-GGUF/gpt-oss-120b-F16.gguf
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 0 | pp512 | 636.25 ± 5.49 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 0 | tg128 | 34.44 ± 0.01 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1 | pp512 | 732.70 ± 5.99 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1 | tg128 | 34.60 ± 0.07 |
# Balanced mode (160-170W), amd_iommu=off
~$ /opt/llmserver/llama.cpp/build/bin/llama-bench -m /opt/llmserver/models/unsloth/gpt-oss-120b-GGUF/gpt-oss-120b-F16.gguf -fa 0,1
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 0 | pp512 | 637.67 ± 5.11 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 0 | tg128 | 34.42 ± 0.02 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1 | pp512 | 730.39 ± 5.21 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1 | tg128 | 34.57 ± 0.07 |
# Performance mode (140-150W max (!?))
~$ /opt/llmserver/llama.cpp/build/bin/llama-bench -m /opt/llmserver/models/unsloth/gpt-oss-120b-GGUF/gpt-oss-120b-F16.gguf -fa 0,1
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 0 | pp512 | 639.66 ± 2.87 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 0 | tg128 | 34.43 ± 0.02 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1 | pp512 | 729.91 ± 3.80 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1 | tg128 | 34.58 ± 0.08 |
# Eco mode (~100W max)
~$ /opt/llmserver/llama.cpp/build/bin/llama-bench -m /opt/llmserver/models/unsloth/gpt-oss-120b-GGUF/gpt-oss-120b-F16.gguf -fa 0,1
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 0 | pp512 | 585.08 ± 6.18 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 0 | tg128 | 33.58 ± 0.06 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1 | pp512 | 642.01 ± 3.92 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1 | tg128 | 33.72 ± 0.11 |
# Balanced mode, PBO GPU +200 MHz
~$ /opt/llmserver/llama.cpp/build/bin/llama-bench -fa 0,1 -m /opt/llmserver/models/unsloth/gpt-oss-120b-GGUF/gpt-oss-120b-F16.gguf
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 0 | pp512 | 653.32 ± 4.41 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 0 | tg128 | 34.74 ± 0.02 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1 | pp512 | 751.01 ± 2.69 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1 | tg128 | 34.88 ± 0.06 |
# Balanced mode, PBO GPU +200 MHz, more options, bigger length
~$ /opt/llmserver/llama.cpp/build/bin/llama-bench -m /opt/llmserver/models/unsloth/gpt-oss-120b-GGUF/gpt-oss-120b-F16.gguf -fa 0,1 -b 1024,2
048,4096 -pg 4096,512
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_batch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | -: | --------------: | -------------------: |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1024 | 0 | pp512 | 657.06 ± 5.18 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1024 | 0 | tg128 | 34.72 ± 0.03 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1024 | 0 | pp4096+tg512 | 190.00 ± 0.73 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1024 | 1 | pp512 | 771.05 ± 6.96 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1024 | 1 | tg128 | 35.44 ± 0.17 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 1024 | 1 | pp4096+tg512 | 208.45 ± 0.48 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 2048 | 0 | pp512 | 698.40 ± 8.47 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 2048 | 0 | tg128 | 35.46 ± 0.19 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 2048 | 0 | pp4096+tg512 | 190.40 ± 0.37 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 2048 | 1 | pp512 | 775.86 ± 6.39 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 2048 | 1 | tg128 | 35.54 ± 0.18 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 2048 | 1 | pp4096+tg512 | 208.34 ± 1.34 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 4096 | 0 | pp512 | 689.05 ± 31.77 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 4096 | 0 | tg128 | 35.44 ± 0.00 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 4096 | 0 | pp4096+tg512 | 190.36 ± 0.45 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 4096 | 1 | pp512 | 778.31 ± 3.59 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 4096 | 1 | tg128 | 35.61 ± 0.01 |
| gpt-oss 120B F16 | 60.87 GiB | 116.83 B | ROCm | 99 | 4096 | 1 | pp4096+tg512 | 209.15 ± 0.29 |
Raw
Other benchmarks.txt
https://www.reddit.com/r/LocalLLaMA/comments/1mqtnz7/comment/n8uhbp3/
https://www.reddit.com/r/LocalLLaMA/comments/1mqtnz7/comment/n8wnxzc/
https://forum.level1techs.com/t/strix-halo-ryzen-ai-max-395-llm-benchmark-results/233796
https://github.com/lhl/strix-halo-testing/tree/main/llm-bench/gpt-oss-120b-F16
If you have your own, I'll be happy to include them here! If you want me to run stuff so you can add to your own dashboards,
happy to do that too.
Raw
Personal notes & setup.txt
I bought this primarily as a Linux server, and secondarily out of curiosity. Running LLMs was just a really nice extra. I ordered
it on 2025/09/25 and it arrived yesterday (2025/10/21). It is much larger than I expected, it needs more than 2U of space (close
to 2.33-2.5U) and weights >3.2kg (>7lbs).
First, I tried the pre-installed Windows. It made create a new account even though I had the network cable connected. Beelink warns
you about that, but I ignored it since I would be installing Linux anyway. The "mini" PC came preconfigured with 96GB allocated
to VRAM and would often reach >170W while doing heavy stuff, without me touching the BIOS. I installed a few games, and tried
Horizon Zero Dawn (4K@60 Ultra, no upscaling), it was at around 30 FPS. I turned on upscaling (I think Balanced), and it reached
a stable 60 FPS, for about half a minute and then the OS crashed with some threading exception. This happened repeatedly. I tried
running LMStudio, it worked for a bit, and then the computer crashed again. I tried all the updates but they failed, and Windows
ended up being corrupted. After it appplied a "repair" / update, the 10 GbE stopped working, and so did LMStudio (would try to
load everything in "system RAM"). I tried rolling back graphics/internet drivers, applying BIOS updates, no luck. The only way to
reset the network card is to unplug the machine, reboots don't work. It is a known issue, and supposedly Beelink and Intel are
working on a fix, but it's been a couple weeks already. Some Redditors pointed out that it's possible it's a hardware fault, not
a driver/BIOS/software issue. Some have reported issues on Linux too. Some have found old driver versions that fix their problems,
but it didn't work for anyone. Not a great first day.
Today (2025/10/22), I installed DietPi and used Ansible to set up user accounts, personal configs, install some packages, etc.
Nothing related to LLMs or this device. The network card didn't work, but thankfully WiFi did... at less than 1 Gbps. Not cool.
I surf the internet waves and stubmle upon the intel driver, I download it, install it, and it works! `speedtest-cli` reports
3-4 Gbps, but I also tested my speeds with `iperf3` using both my old server and my ISP (init7's nerd mode is really cool, see:
https://www.init7.net/en/support/faq/speedtest), and I'm happy to report I'm getting my full 10GbE speeds. The commands are
roughly like this:
apt install -y build-essential linux-headers-amd64 # you might need more, or less
wget https://github.com/intel/ethernet-linux-ixgbe/releases/download/v6.1.3/ixgbe-6.1.3.tar.gz
tar -xzvf ixgbe-6.1.3.tar.gz
mkdir /usr/local/src/ixgbe
mv /usr/local/src/ixgbe1 /usr/local/src/ixgbe/ixgbe-6.1.3
cd /usr/local/src/ixgbe/ixgbe-6.1.3/src
make install
rmmod ixgbe; modprobe ixgbe
Then back to "dietpi-config" to ensure it's enabled, and we're good to go! I didn't keep the logs, but it's easy to re-run:
~$ speedtest
[...]
Download: 8252.93 Mbit/s
Upload: 1073.59 Mbit/s
~$ iperf3 -c speedtest.init7.net
Connecting to host speedtest.init7.net, port 5201
[...]
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 1.09 GBytes 9.35 Gbits/sec 0 2.87 MBytes
[ 5] 1.00-2.00 sec 1.10 GBytes 9.42 Gbits/sec 0 2.87 MBytes
[ 5] 2.00-3.00 sec 1.10 GBytes 9.41 Gbits/sec 0 2.87 MBytes
[ 5] 3.00-4.00 sec 1.10 GBytes 9.42 Gbits/sec 0 2.87 MBytes
[ 5] 5.00-6.00 sec 1.10 GBytes 9.41 Gbits/sec 0 2.87 MBytes
Time to get llama.cpp working, because vLLM can be a pain on non-GPU systems (and sometimes on them too). It seems there are a few
options, like Vulkan, ROCm/HIP(?), CPU. I tried CPU-only in Windows and it seemed slower, and some dude on Reddit also had the same
experience, so iGPU it is. Vulkan seemed like the more lengthy route, so ROCm it is. Previously I had read of people using nightly
builds and hunting unofficial repositories, but the llama.cpp docs just pointed me to the official website and I went there like a
trusting noob. It was easy and it seems to have worked. Here are the commands I used (more or less):
apt install python3-setuptools python3-wheel
/opt/amddeb/
wget -P /opt/amddeb https://repo.radeon.com/amdgpu-install/7.0.2/ubuntu/noble/amdgpu-install_7.0.2.70002-1_all.deb
apt install /opt/amddeb/amdgpu-install_7.0.2.70002-1_all.deb
apt update
usermod -a -G render,video root # not sure if necessary
usermod -a -G render,video <my_user_name>
apt install rocm amdgpu-dkms
tee --append /etc/ld.so.conf.d/rocm.conf <<EOF
/opt/rocm/lib
/opt/rocm/lib64
EOF
ldconfig
update-alternatives --display rocm
cat /etc/ld.so.conf.d/rocm.conf
source /etc/profile.d/modules.sh
echo 'export LD_LIBRARY_PATH=/opt/rocm-7.0.2/lib' >> ~/.bashrc
source ~/.bashrc
For good measure, do a reboot. There were probably easier ways to do the above, like using AMD's scripts, but why bother reading
the documentation when you can just blindly copy-paste commands? I also have no idea what any of the commands below do, or if they
were necessary, but I ran them anyway:
wget -P /opt/amddeb https://repo.radeon.com/amdgpu/30.10.2/ubuntu/pool/main/a/amdgpu-insecure-instinct-udev-rules/amdgpu-insecure-instinct-udev-rules_30.10.2.0-2226257.24.04_all.deb
apt install /opt/amddeb/amdgpu-insecure-instinct-udev-rules_30.10.2.0-2226257.24.04_all.deb
groupadd devteam
tee /etc/udev/rules.d/70-amdgpu.rules <<EOF
KERNEL=="kfd", GROUP="render", MODE="0666"
SUBSYSTEM=="drm", KERNEL=="renderD*", GROUP="render", MODE="0666"
EOF
udevadm control --reload-rules && udevadm trigger
Now for llama.cpp, you can probably grub the prebuilt project, but I didn't think of that at the time and went with the manual route.
It's probably wise to switch to a non-root user at this point, or even earlier, but Linux fellas know that You Only Lib Once. Or was
it Link? Anyway...
apt install curl libcurl4-openssl-dev
/opt/llmserver
cd /opt/llmserver
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" cmake -S . -B build -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 \
-DGGML_HIP_ROCWMMA_FATTN=ON -DBUILD_SHARED_LIBS=ON -DGGML_CUDA_ENABLE_UNIFIED_MEMORY=1 -DCMAKE_PREFIX_PATH=/opt/rocm-7.0.2
cmake --build build --config Release -j 16
For you rocm might be in a different folder, look it up in the docs, use "find", or something -- I believe in you! Now just download
your favorite LLM and run like the wind. On a calm day. Inside the house. This thing isn't a replacement for the DGX GB300 in your
basement, but it will run your daily n8n workflows without you going deaf if it's in the same room as you.
Raw
Speculative decoding.txt
I only tried Llama-3.3-70B (UD-Q4_K_XL) with Llama-3.2-1B (UD-Q6_K_XL & UD-Q8_K_XL). I tried a few prompts but they were too slow
for me to bother and I deleted them before I thought to run llama-bench, but here are some of the logs in case you're curious:
~$ /opt/llmserver/llama.cpp/build/bin/llama-server --gpu-layers 999 --batch-size 4096 --flash-attn on \
--threads 24 --port 11435 --host 0.0.0.0 --alias "homelab/bllm" -ctk q8_0 -ctv q8_0 --jinja -c 32768 \
-m /opt/llmserver/models/unsloth/Llama-3.3-70B-Instruct-GGUF/Llama-3.3-70B-Instruct-UD-Q4_K_XL.gguf \
--draft-max 24 --draft-min 1 --draft-p-min 0.8 -ngld 99 \
-md /opt/llmserver/models/unsloth/Llama-3.2-1B-Instruct-GGUF/Llama-3.2-1B-Instruct-UD-Q6_K_XL.gguf
prompt eval time = 36092.89 ms / 2740 tokens ( 13.17 ms per token, 75.92 tokens per second)
eval time = 105826.52 ms / 681 tokens ( 155.40 ms per token, 6.44 tokens per second)
total time = 141919.41 ms / 3421 tokens
draft acceptance rate = 0.69869 ( 320 accepted / 458 generated)
-----------------
~$ /opt/llmserver/llama.cpp/build/bin/llama-server --gpu-layers 999 --batch-size 4096 --flash-attn on \
--threads 24 --port 11435 --host 0.0.0.0 --alias "homelab/bllm" -ctk q8_0 -ctv q8_0 --jinja -c 32768 \
-m /opt/llmserver/models/unsloth/Llama-3.3-70B-Instruct-GGUF/Llama-3.3-70B-Instruct-UD-Q4_K_XL.gguf \
--draft-max 24 --draft-min 1 --draft-p-min 0.8 -ngld 99 \
-md /opt/llmserver/models/unsloth/Llama-3.2-1B-Instruct-GGUF/Llama-3.2-1B-Instruct-UD-Q8_K_XL.gguf
prompt eval time = 35764.21 ms / 2740 tokens ( 13.05 ms per token, 76.61 tokens per second)
eval time = 85524.73 ms / 593 tokens ( 144.22 ms per token, 6.93 tokens per second)
total time = 121288.94 ms / 3333 tokens
draft acceptance rate = 0.74224 ( 311 accepted / 419 generated)
Maybe some smaller model (20-30B) or a large MoE model (Qwen3 235B, GLM-4.5(-Air)) could be good alternatives if you are willing
to find (or train) draft models and experiment to see what works.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment