Created
November 5, 2025 22:00
-
-
Save Hermann-SW/1d0b7f84610e1aee657ab1cad8e09460 to your computer and use it in GitHub Desktop.
Simple stacking model allowing AMD Instinct MI50 GPUs to be operated by Riser cards from side
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| "use strict" | |
| const jscad = require('@jscad/modeling') | |
| const { intersect, subtract, union } = jscad.booleans | |
| const { translate } = jscad.transforms | |
| const { cuboid } = jscad.primitives | |
| const L=112 | |
| const H=41+12 | |
| const W=10 | |
| const T=5 | |
| const D=20 | |
| const w=3 | |
| const t=2.5 | |
| const d=8 | |
| function main() { | |
| return [ | |
| subtract( | |
| union( | |
| translate([0,0,10/2], cuboid({size: [L+2*W,D,T+10]})), | |
| translate([L/2+W/2,0,(T+H)/2-T/2], cuboid({size: [W,2*D,T+H]})), | |
| translate([-(L/2+W/2),0,(T+H)/2-T/2], cuboid({size: [W,2*D,T+H]})), | |
| translate([(L+W)/2,0,T+H-t/2], cuboid({size: [w,d,t]})), | |
| translate([-(L+W)/2,0,T+H-t/2], cuboid({size: [w,d,t]})), | |
| ), | |
| translate([(L+W)/2,0,-T/2+t/2], cuboid({size: [w+0.3,d+0.3,t+0.3]})), | |
| translate([-(L+W)/2,0,-T/2+t/2], cuboid({size: [w+0.3,d+0.3,t+0.3]})), | |
| ) | |
| ] | |
| } | |
| module.exports = { main } |
Author
Author
4 stacked Instinct MI50 GPUs, connected to PC with 4 Riser cards:
Author
AMD 7600X CPU PC with 5× AMD Instinct MI50 GPUs.
The 32GB VRAM GPU is inside PCs x16 PCIE slot, the 4 stacked GPUs have 16GB VRAM.
PC running in photo, with 5x 6.705 TFLOPs FP64 or 33.525 TFLOPs FP64 theoretical max in total.
Power cables from PC inside to power the GPUs as well as the 4 BBQ blower fans.
Author
5 MI50s and internal GPU detected:
hermann@7600x:~$ rocminfo | grep Uuid
Uuid: CPU-XX
Uuid: GPU-4124412172e62126
Uuid: GPU-304c70e172dc768c
Uuid: GPU-d64a58a17330f0ed
Uuid: GPU-6e56508172dc76b6
Uuid: GPU-f890794172e62691
Uuid: GPU-XX
hermann@7600x:~$
Author
Top GPU is the 32GB MI50 inside PC:
hermann@7600x:~$ clinfo | egrep "(Board|compute)"
Board name: AMD Radeon Graphics
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Radeon Graphics
Max compute units: 1
hermann@7600x:~$
Author
Measurents:
Idle power consumption 250W.
gpuowl benchmark (*)
| GPU UUID | us/iter (*) | ID | power[W] | VRAM[GB] |
|---|---|---|---|---|
| 4124412172e62126 | 535 | 0 | 398 | 32 |
| 304c70e172dc768c | 515 | 1 | 411 | 16 |
| d64a58a17330f0ed | 530 | 2 | 399 | 16 |
| 6e56508172dc76b6 | 514 | 3 | 410 | 16 |
| f890794172e62691 | 529 | 4 | 380 | 16 |
Author
I added one more Instinct MI50, now 5 outside and 1 inside PC case.
With theoretical 6.705 TFLOPS FP64 per MI50, this PC now has 40.23 TFLOPS FP64 theoretical max.
All powered from 1850W Zumax PSU inside PC — tests needed how many can work in parallel.
Plan is to have 4 MI50s connected to this (7600X CPU) PC.
And the other 4 MI50s connected to AMD 7950X CPU PC, which has Zumax 1850W PSU inside as well.
hermann@7600x:~$ rocminfo | grep Uuid
Uuid: CPU-XX
Uuid: GPU-4124412172e62126
Uuid: GPU-304c70e172dc768c
Uuid: GPU-d64a58a17330f0ed
Uuid: GPU-6e56508172dc76b6
Uuid: GPU-f890794172e62691
Uuid: GPU-13c24061732c730c
Uuid: GPU-XX
hermann@7600x:~$
hermann@7600x:~$ clinfo | egrep "(Board|compute)"
Board name: AMD Radeon Graphics
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Radeon Graphics
Max compute units: 1
hermann@7600x:~$
Top two GPU holders 2-colored because beige filament ran empty during 3Dprint.
Author
Not sure why the 32GB MI50 inside PC (device 0) has much lower FP64 GFLOPS numbers than the other 16GB MI50s.
It is located on main board PCIE x16 slot, the others on Riser cards.
Riser cards connected to one PCIE x1 slot and to one x1 to 4 port x16 Riser adapter card inside PC.
hermann@7600x:~$ for((d=0;d<6;++d)); do clpeak --compute-dp -d $d | grep "double "; done
double : 5850.00
double : 6224.23
double : 6202.26
double : 6228.68
double : 6265.88
double : 6228.16
hermann@7600x:~$
Author
Sequential gpuowl benchmark for the 6 MI50s.
There is nothing wrong with not having internal GPU disabled.
But last night I had HDMI cable plugged in to see boot sequence on HDMI display.
That did make displayed GPU UUIDs displayed shifted by one, and therefore wrong.
Now I just unplugged the HDMI cable and executed the same command again.
As can be seen below the gpuowl reported GPU UUIDs for devices 0-5 correspond with UUIDs reported by rocminfo:
hermann@7600x:~/gpuowl-v7.2-131-gca22dce-ubuntu$ rocminfo | grep Uuid
Uuid: CPU-XX
Uuid: GPU-4124412172e62126
Uuid: GPU-304c70e172dc768c
Uuid: GPU-d64a58a17330f0ed
Uuid: GPU-6e56508172dc76b6
Uuid: GPU-f890794172e62691
Uuid: GPU-13c24061732c730c
Uuid: GPU-XX
hermann@7600x:~/gpuowl-v7.2-131-gca22dce-ubuntu$
hermann@7600x:~/gpuowl-v7.2-131-gca22dce-ubuntu$ for((d=0;d<6;++d))
> do
> rm -rf 77* && ./gpuowl -prp 77936867 -iters 30000 -log 10000 -fft 4M -d $d
> echo "================================================"
> done
20251108 23:06:02 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:06:02 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:06:02 Note: not found 'config.txt'
20251108 23:06:02 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 0
20251108 23:06:02 device 0, unique id '4124412172e62126'
20251108 23:06:02 4124412172e62126 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251108 23:06:05 4124412172e62126 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251108 23:06:06 4124412172e62126 77936867 OpenCL compilation in 0.76 s
20251108 23:06:06 4124412172e62126 77936867 PRP starting from beginning
20251108 23:06:06 4124412172e62126 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251108 23:06:06 4124412172e62126 77936867 validating proof residues for power 8
20251108 23:06:06 4124412172e62126 77936867 Proof using power 8
20251108 23:06:12 4124412172e62126 77936867 OK 10000 0.01% fc4f135f7cf4ad29 535 us/it + check 0.25s + save 0.09s; ETA 11:34
20251108 23:06:18 4124412172e62126 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 535 us/it + check 0.25s + save 0.09s; ETA 11:35
20251108 23:06:23 4124412172e62126 77936867 Stopping, please wait..
20251108 23:06:23 4124412172e62126 77936867 OK 30000 0.04% c4e0ff35e3290d98 536 us/it + check 0.25s + save 0.09s; ETA 11:36
20251108 23:06:23 4124412172e62126 Exiting because "stop requested"
20251108 23:06:23 4124412172e62126 Bye
================================================
20251108 23:06:23 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:06:23 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:06:23 Note: not found 'config.txt'
20251108 23:06:23 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 1
20251108 23:06:23 device 1, unique id '304c70e172dc768c'
20251108 23:06:23 304c70e172dc768c 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251108 23:06:23 304c70e172dc768c 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251108 23:06:24 304c70e172dc768c 77936867 OpenCL compilation in 0.79 s
20251108 23:06:24 304c70e172dc768c 77936867 PRP starting from beginning
20251108 23:06:25 304c70e172dc768c 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251108 23:06:25 304c70e172dc768c 77936867 validating proof residues for power 8
20251108 23:06:25 304c70e172dc768c 77936867 Proof using power 8
20251108 23:06:30 304c70e172dc768c 77936867 OK 10000 0.01% fc4f135f7cf4ad29 514 us/it + check 0.28s + save 0.08s; ETA 11:08
20251108 23:06:36 304c70e172dc768c 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 515 us/it + check 0.28s + save 0.10s; ETA 11:08
20251108 23:06:41 304c70e172dc768c 77936867 Stopping, please wait..
20251108 23:06:41 304c70e172dc768c 77936867 OK 30000 0.04% c4e0ff35e3290d98 515 us/it + check 0.28s + save 0.09s; ETA 11:08
20251108 23:06:41 304c70e172dc768c Exiting because "stop requested"
20251108 23:06:41 304c70e172dc768c Bye
================================================
20251108 23:06:41 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:06:41 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:06:41 Note: not found 'config.txt'
20251108 23:06:41 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 2
20251108 23:06:41 device 2, unique id 'd64a58a17330f0ed'
20251108 23:06:41 d64a58a17330f0ed 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251108 23:06:41 d64a58a17330f0ed 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251108 23:06:42 d64a58a17330f0ed 77936867 OpenCL compilation in 0.78 s
20251108 23:06:42 d64a58a17330f0ed 77936867 PRP starting from beginning
20251108 23:06:43 d64a58a17330f0ed 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251108 23:06:43 d64a58a17330f0ed 77936867 validating proof residues for power 8
20251108 23:06:43 d64a58a17330f0ed 77936867 Proof using power 8
20251108 23:06:48 d64a58a17330f0ed 77936867 OK 10000 0.01% fc4f135f7cf4ad29 530 us/it + check 0.28s + save 0.09s; ETA 11:28
20251108 23:06:54 d64a58a17330f0ed 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 530 us/it + check 0.28s + save 0.09s; ETA 11:28
20251108 23:06:59 d64a58a17330f0ed 77936867 Stopping, please wait..
20251108 23:07:00 d64a58a17330f0ed 77936867 OK 30000 0.04% c4e0ff35e3290d98 530 us/it + check 0.28s + save 0.09s; ETA 11:28
20251108 23:07:00 d64a58a17330f0ed Exiting because "stop requested"
20251108 23:07:00 d64a58a17330f0ed Bye
================================================
20251108 23:07:00 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:07:00 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:07:00 Note: not found 'config.txt'
20251108 23:07:00 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 3
20251108 23:07:00 device 3, unique id '6e56508172dc76b6'
20251108 23:07:00 6e56508172dc76b6 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251108 23:07:00 6e56508172dc76b6 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251108 23:07:00 6e56508172dc76b6 77936867 OpenCL compilation in 0.78 s
20251108 23:07:01 6e56508172dc76b6 77936867 PRP starting from beginning
20251108 23:07:01 6e56508172dc76b6 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251108 23:07:01 6e56508172dc76b6 77936867 validating proof residues for power 8
20251108 23:07:01 6e56508172dc76b6 77936867 Proof using power 8
20251108 23:07:06 6e56508172dc76b6 77936867 OK 10000 0.01% fc4f135f7cf4ad29 514 us/it + check 0.28s + save 0.09s; ETA 11:07
20251108 23:07:12 6e56508172dc76b6 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 514 us/it + check 0.28s + save 0.08s; ETA 11:08
20251108 23:07:17 6e56508172dc76b6 77936867 Stopping, please wait..
20251108 23:07:17 6e56508172dc76b6 77936867 OK 30000 0.04% c4e0ff35e3290d98 514 us/it + check 0.28s + save 0.09s; ETA 11:08
20251108 23:07:17 6e56508172dc76b6 Exiting because "stop requested"
20251108 23:07:17 6e56508172dc76b6 Bye
================================================
20251108 23:07:18 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:07:18 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:07:18 Note: not found 'config.txt'
20251108 23:07:18 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 4
20251108 23:07:18 device 4, unique id 'f890794172e62691'
20251108 23:07:18 f890794172e62691 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251108 23:07:18 f890794172e62691 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251108 23:07:18 f890794172e62691 77936867 OpenCL compilation in 0.79 s
20251108 23:07:19 f890794172e62691 77936867 PRP starting from beginning
20251108 23:07:19 f890794172e62691 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251108 23:07:19 f890794172e62691 77936867 validating proof residues for power 8
20251108 23:07:19 f890794172e62691 77936867 Proof using power 8
20251108 23:07:25 f890794172e62691 77936867 OK 10000 0.01% fc4f135f7cf4ad29 528 us/it + check 0.28s + save 0.09s; ETA 11:26
20251108 23:07:30 f890794172e62691 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 530 us/it + check 0.28s + save 0.08s; ETA 11:28
20251108 23:07:35 f890794172e62691 77936867 Stopping, please wait..
20251108 23:07:36 f890794172e62691 77936867 OK 30000 0.04% c4e0ff35e3290d98 530 us/it + check 0.28s + save 0.09s; ETA 11:28
20251108 23:07:36 f890794172e62691 Exiting because "stop requested"
20251108 23:07:36 f890794172e62691 Bye
================================================
20251108 23:07:36 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:07:36 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251108 23:07:36 Note: not found 'config.txt'
20251108 23:07:36 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 5
20251108 23:07:36 device 5, unique id '13c24061732c730c'
20251108 23:07:36 13c24061732c730c 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251108 23:07:36 13c24061732c730c 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251108 23:07:37 13c24061732c730c 77936867 OpenCL compilation in 0.78 s
20251108 23:07:37 13c24061732c730c 77936867 PRP starting from beginning
20251108 23:07:37 13c24061732c730c 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251108 23:07:37 13c24061732c730c 77936867 validating proof residues for power 8
20251108 23:07:37 13c24061732c730c 77936867 Proof using power 8
20251108 23:07:43 13c24061732c730c 77936867 OK 10000 0.01% fc4f135f7cf4ad29 525 us/it + check 0.26s + save 0.09s; ETA 11:22
20251108 23:07:48 13c24061732c730c 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 526 us/it + check 0.26s + save 0.11s; ETA 11:23
20251108 23:07:54 13c24061732c730c 77936867 Stopping, please wait..
20251108 23:07:54 13c24061732c730c 77936867 OK 30000 0.04% c4e0ff35e3290d98 526 us/it + check 0.26s + save 0.09s; ETA 11:23
20251108 23:07:54 13c24061732c730c Exiting because "stop requested"
20251108 23:07:54 13c24061732c730c Bye
================================================
hermann@7600x:~/gpuowl-v7.2-131-gca22dce-ubuntu$
Author
hermann@7600x:~/genefer22/bin$ ./geneferg -b 572186 -n 17 -q -d 0
geneferg version 24.04.1 (linux x64, gcc-13.3.0)
Copyright (c) 2022, Yves Gallot
genefer is free source code, under the MIT license.
Command line: '-b 572186 -n 17 -q -d 0'
Running on device 'gfx906:sramecc+:xnack-', vendor 'Advanced Micro Devices, Inc.', version 'OpenCL 2.0 ', driver '3635.0 (HSA1.1,LC)', data size: 3 MB.
572186^{2^17} + 1 is a probable prime, time = 00:02:24.
hermann@7600x:~/genefer22/bin$ ./geneferg -b 572186 -n 17 -q -d 1
geneferg version 24.04.1 (linux x64, gcc-13.3.0)
Copyright (c) 2022, Yves Gallot
genefer is free source code, under the MIT license.
Command line: '-b 572186 -n 17 -q -d 1'
Running on device 'gfx906:sramecc+:xnack-', vendor 'Advanced Micro Devices, Inc.', version 'OpenCL 2.0 ', driver '3635.0 (HSA1.1,LC)', data size: 3 MB.
572186^{2^17} + 1 is a probable prime, time = 00:03:03.
hermann@7600x:~/genefer22/bin$ ./geneferg -b 572186 -n 17 -q -d 2
geneferg version 24.04.1 (linux x64, gcc-13.3.0)
Copyright (c) 2022, Yves Gallot
genefer is free source code, under the MIT license.
Command line: '-b 572186 -n 17 -q -d 2'
Running on device 'gfx906:sramecc+:xnack-', vendor 'Advanced Micro Devices, Inc.', version 'OpenCL 2.0 ', driver '3635.0 (HSA1.1,LC)', data size: 3 MB.
572186^{2^17} + 1 is a probable prime, time = 00:03:03.
hermann@7600x:~/genefer22/bin$ ./geneferg -b 572186 -n 17 -q -d 3
geneferg version 24.04.1 (linux x64, gcc-13.3.0)
Copyright (c) 2022, Yves Gallot
genefer is free source code, under the MIT license.
Command line: '-b 572186 -n 17 -q -d 3'
Running on device 'gfx906:sramecc+:xnack-', vendor 'Advanced Micro Devices, Inc.', version 'OpenCL 2.0 ', driver '3635.0 (HSA1.1,LC)', data size: 3 MB.
572186^{2^17} + 1 is a probable prime, time = 00:02:55.
hermann@7600x:~/genefer22/bin$ ./geneferg -b 572186 -n 17 -q -d 4
geneferg version 24.04.1 (linux x64, gcc-13.3.0)
Copyright (c) 2022, Yves Gallot
genefer is free source code, under the MIT license.
Command line: '-b 572186 -n 17 -q -d 4'
Running on device 'gfx906:sramecc+:xnack-', vendor 'Advanced Micro Devices, Inc.', version 'OpenCL 2.0 ', driver '3635.0 (HSA1.1,LC)', data size: 3 MB.
572186^{2^17} + 1 is a probable prime, time = 00:02:57.
hermann@7600x:~/genefer22/bin$ ./geneferg -b 572186 -n 17 -q -d 5
geneferg version 24.04.1 (linux x64, gcc-13.3.0)
Copyright (c) 2022, Yves Gallot
genefer is free source code, under the MIT license.
Command line: '-b 572186 -n 17 -q -d 5'
Running on device 'gfx906:sramecc+:xnack-', vendor 'Advanced Micro Devices, Inc.', version 'OpenCL 2.0 ', driver '3635.0 (HSA1.1,LC)', data size: 3 MB.
572186^{2^17} + 1 is a probable prime, time = 00:02:58.
hermann@7600x:~/genefer22/bin$
Author
PSU used in my AMD 7600X as well as 7950X PC is:
"Mining Zumax ZU 1850W ATX nonModular"
Cables and power supply connectors:
- 8 cables with 6 + 2pin connectors (on each braid of 2 connectors, all-16 PCs)
(4 long, 4 short) - 3 cables with SATA connectors, Molex
- 1 cables for connecting to the motherboard.
- 1 cables for connecting the processor.
I did simply cut a hole into both PC cases where the PSU cables leave PSU inside PC.
4 long and 1 short 8pin cables leave the PCs this way, allowing to power 5 GPUs outside for now.
One GPU is plugged into 7600X PC x16 PCIE slot and powered with one of the remaining 3 short 8pin cables.
 | 
|
Author
I added two Instinct MI50s from basement, and unplugged my (only) 32GB Instinct MI50 GPU from x16 PCIE slot inside PC and connected outside as well.
This is system working.
All 4 long and 4 short GPU power cables of Zumax 1850W PSU are now in use.
4 long, 1 short and 2 cables with 2× SATA and 2× Molex each leave PC through side hole in PC case.
SATA splitter cable leaves PC back together with remaining 3× GPU power cables.
The 4 Molex connectors are used to power the 8 BBQ fans.
The Riser modules are powered with 8 SATA connectors.
I had to replace the 4× Riser card with a 6× Riser card, one x1 adapter plugs into x16 slot now:
|
|
After some recabling, now the visible GPU UUID order corresponds to OS view:
hermann@7600x:~$ rocminfo | grep Uuid
Uuid: CPU-XX
Uuid: GPU-4124412172e62126
Uuid: GPU-c49e19417337ece3
Uuid: GPU-6a0e7961732c730d
Uuid: GPU-13c24061732c730c
Uuid: GPU-304c70e172dc768c
Uuid: GPU-6e56508172dc76b6
Uuid: GPU-d64a58a17330f0ed
Uuid: GPU-f890794172e62691
Uuid: GPU-XX
hermann@7600x:~$
Leftmost characters are hidden by 3Dprinted holder in photo, rightmost characters can be compared:
Total 3Dprint time for all stckable MI50 holder parts was 48 hours.
Author
Nice output:
hermann@7600x:~$ clinfo | egrep "(Board|compute)"
Board name: AMD Radeon Graphics
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Instinct MI50/MI60
Max compute units: 60
Board name: AMD Radeon Graphics
Max compute units: 1
hermann@7600x:~$
Measured peak performances sum up to 49.658 TFLOPs (4.9658×10¹³ double precision floating point operations per second):
hermann@7600x:~$ for((d=0;d<8;++d)); do clpeak --compute-dp -d $d | grep "double "; done
double : 5985.70
double : 6245.95
double : 6220.22
double : 6219.18
double : 6254.97
double : 6239.06
double : 6252.34
double : 6240.47
hermann@7600x:~$
The 32GB Instinct MI50 keeps outside being slower than the other MI50s.
Author
Function test with gpuowl benchmark:
| GPU UUID | us/iter (*) | ID | VRAM[GB] |
|---|---|---|---|
| 4124412172e62126 | 537 | 0 | 32 |
| c49e19417337ece3 | 529 | 1 | 16 |
| 6a0e7961732c730d | 530 | 2 | 16 |
| 13c24061732c730c | 530 | 3 | 16 |
| 304c70e172dc768c | 514 | 4 | 16 |
| 6e56508172dc76b6 | 514 | 5 | 16 |
| d64a58a17330f0ed | 529 | 6 | 16 |
| f890794172e62691 | 531 | 7 | 16 |
hermann@7600x:~$ cd gpuowl-v7.2-131-gca22dce-ubuntu/
hermann@7600x:~/gpuowl-v7.2-131-gca22dce-ubuntu$ for((d=0;d<8;++d)); do rm -rf 77* && ./gpuowl -prp 77936867 -iters 30000 -log 10000 -fft 4M -d $d; echo "================================================"; done
20251112 20:58:39 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 20:58:39 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 20:58:39 Note: not found 'config.txt'
20251112 20:58:39 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 0
20251112 20:58:39 device 0, unique id '4124412172e62126'
20251112 20:58:39 4124412172e62126 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251112 20:58:39 4124412172e62126 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251112 20:58:40 4124412172e62126 77936867 OpenCL compilation in 0.79 s
20251112 20:58:40 4124412172e62126 77936867 PRP starting from beginning
20251112 20:58:40 4124412172e62126 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251112 20:58:40 4124412172e62126 77936867 validating proof residues for power 8
20251112 20:58:40 4124412172e62126 77936867 Proof using power 8
20251112 20:58:46 4124412172e62126 77936867 OK 10000 0.01% fc4f135f7cf4ad29 535 us/it + check 0.27s + save 0.09s; ETA 11:35
20251112 20:58:52 4124412172e62126 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 537 us/it + check 0.27s + save 0.09s; ETA 11:37
20251112 20:58:57 4124412172e62126 77936867 Stopping, please wait..
20251112 20:58:58 4124412172e62126 77936867 OK 30000 0.04% c4e0ff35e3290d98 537 us/it + check 0.27s + save 0.09s; ETA 11:37
20251112 20:58:58 4124412172e62126 Exiting because "stop requested"
20251112 20:58:58 4124412172e62126 Bye
================================================
20251112 20:58:58 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 20:58:58 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 20:58:58 Note: not found 'config.txt'
20251112 20:58:58 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 1
20251112 20:58:58 device 1, unique id 'c49e19417337ece3'
20251112 20:58:58 c49e19417337ece3 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251112 20:58:58 c49e19417337ece3 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251112 20:58:58 c49e19417337ece3 77936867 OpenCL compilation in 0.78 s
20251112 20:58:59 c49e19417337ece3 77936867 PRP starting from beginning
20251112 20:58:59 c49e19417337ece3 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251112 20:58:59 c49e19417337ece3 77936867 validating proof residues for power 8
20251112 20:58:59 c49e19417337ece3 77936867 Proof using power 8
20251112 20:59:05 c49e19417337ece3 77936867 OK 10000 0.01% fc4f135f7cf4ad29 529 us/it + check 0.28s + save 0.09s; ETA 11:27
20251112 20:59:10 c49e19417337ece3 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 529 us/it + check 0.28s + save 0.08s; ETA 11:28
20251112 20:59:16 c49e19417337ece3 77936867 Stopping, please wait..
20251112 20:59:16 c49e19417337ece3 77936867 OK 30000 0.04% c4e0ff35e3290d98 529 us/it + check 0.28s + save 0.09s; ETA 11:27
20251112 20:59:16 c49e19417337ece3 Exiting because "stop requested"
20251112 20:59:16 c49e19417337ece3 Bye
================================================
20251112 20:59:16 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 20:59:16 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 20:59:16 Note: not found 'config.txt'
20251112 20:59:16 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 2
20251112 20:59:16 device 2, unique id '6a0e7961732c730d'
20251112 20:59:16 6a0e7961732c730d 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251112 20:59:16 6a0e7961732c730d 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251112 20:59:17 6a0e7961732c730d 77936867 OpenCL compilation in 0.78 s
20251112 20:59:17 6a0e7961732c730d 77936867 PRP starting from beginning
20251112 20:59:17 6a0e7961732c730d 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251112 20:59:17 6a0e7961732c730d 77936867 validating proof residues for power 8
20251112 20:59:17 6a0e7961732c730d 77936867 Proof using power 8
20251112 20:59:23 6a0e7961732c730d 77936867 OK 10000 0.01% fc4f135f7cf4ad29 528 us/it + check 0.28s + save 0.09s; ETA 11:26
20251112 20:59:29 6a0e7961732c730d 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 530 us/it + check 0.28s + save 0.08s; ETA 11:28
20251112 20:59:34 6a0e7961732c730d 77936867 Stopping, please wait..
20251112 20:59:34 6a0e7961732c730d 77936867 OK 30000 0.04% c4e0ff35e3290d98 530 us/it + check 0.28s + save 0.08s; ETA 11:28
20251112 20:59:34 6a0e7961732c730d Exiting because "stop requested"
20251112 20:59:34 6a0e7961732c730d Bye
================================================
20251112 20:59:34 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 20:59:34 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 20:59:34 Note: not found 'config.txt'
20251112 20:59:34 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 3
20251112 20:59:34 device 3, unique id '13c24061732c730c'
20251112 20:59:34 13c24061732c730c 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251112 20:59:34 13c24061732c730c 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251112 20:59:35 13c24061732c730c 77936867 OpenCL compilation in 0.78 s
20251112 20:59:35 13c24061732c730c 77936867 PRP starting from beginning
20251112 20:59:36 13c24061732c730c 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251112 20:59:36 13c24061732c730c 77936867 validating proof residues for power 8
20251112 20:59:36 13c24061732c730c 77936867 Proof using power 8
20251112 20:59:41 13c24061732c730c 77936867 OK 10000 0.01% fc4f135f7cf4ad29 529 us/it + check 0.28s + save 0.09s; ETA 11:28
20251112 20:59:47 13c24061732c730c 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 530 us/it + check 0.28s + save 0.09s; ETA 11:29
20251112 20:59:52 13c24061732c730c 77936867 Stopping, please wait..
20251112 20:59:53 13c24061732c730c 77936867 OK 30000 0.04% c4e0ff35e3290d98 530 us/it + check 0.29s + save 0.09s; ETA 11:29
20251112 20:59:53 13c24061732c730c Exiting because "stop requested"
20251112 20:59:53 13c24061732c730c Bye
================================================
20251112 20:59:53 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 20:59:53 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 20:59:53 Note: not found 'config.txt'
20251112 20:59:53 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 4
20251112 20:59:53 device 4, unique id '304c70e172dc768c'
20251112 20:59:53 304c70e172dc768c 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251112 20:59:53 304c70e172dc768c 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251112 20:59:54 304c70e172dc768c 77936867 OpenCL compilation in 0.78 s
20251112 20:59:54 304c70e172dc768c 77936867 PRP starting from beginning
20251112 20:59:54 304c70e172dc768c 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251112 20:59:54 304c70e172dc768c 77936867 validating proof residues for power 8
20251112 20:59:54 304c70e172dc768c 77936867 Proof using power 8
20251112 21:00:00 304c70e172dc768c 77936867 OK 10000 0.01% fc4f135f7cf4ad29 514 us/it + check 0.28s + save 0.09s; ETA 11:08
20251112 21:00:05 304c70e172dc768c 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 515 us/it + check 0.28s + save 0.09s; ETA 11:08
20251112 21:00:10 304c70e172dc768c 77936867 Stopping, please wait..
20251112 21:00:11 304c70e172dc768c 77936867 OK 30000 0.04% c4e0ff35e3290d98 514 us/it + check 0.28s + save 0.08s; ETA 11:08
20251112 21:00:11 304c70e172dc768c Exiting because "stop requested"
20251112 21:00:11 304c70e172dc768c Bye
================================================
20251112 21:00:11 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 21:00:11 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 21:00:11 Note: not found 'config.txt'
20251112 21:00:11 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 5
20251112 21:00:11 device 5, unique id '6e56508172dc76b6'
20251112 21:00:11 6e56508172dc76b6 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251112 21:00:11 6e56508172dc76b6 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251112 21:00:12 6e56508172dc76b6 77936867 OpenCL compilation in 0.78 s
20251112 21:00:12 6e56508172dc76b6 77936867 PRP starting from beginning
20251112 21:00:12 6e56508172dc76b6 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251112 21:00:12 6e56508172dc76b6 77936867 validating proof residues for power 8
20251112 21:00:12 6e56508172dc76b6 77936867 Proof using power 8
20251112 21:00:18 6e56508172dc76b6 77936867 OK 10000 0.01% fc4f135f7cf4ad29 514 us/it + check 0.28s + save 0.08s; ETA 11:08
20251112 21:00:23 6e56508172dc76b6 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 514 us/it + check 0.28s + save 0.08s; ETA 11:08
20251112 21:00:28 6e56508172dc76b6 77936867 Stopping, please wait..
20251112 21:00:29 6e56508172dc76b6 77936867 OK 30000 0.04% c4e0ff35e3290d98 514 us/it + check 0.28s + save 0.08s; ETA 11:08
20251112 21:00:29 6e56508172dc76b6 Exiting because "stop requested"
20251112 21:00:29 6e56508172dc76b6 Bye
================================================
20251112 21:00:29 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 21:00:29 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 21:00:29 Note: not found 'config.txt'
20251112 21:00:29 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 6
20251112 21:00:29 device 6, unique id 'd64a58a17330f0ed'
20251112 21:00:29 d64a58a17330f0ed 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251112 21:00:29 d64a58a17330f0ed 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251112 21:00:29 d64a58a17330f0ed 77936867 OpenCL compilation in 0.78 s
20251112 21:00:30 d64a58a17330f0ed 77936867 PRP starting from beginning
20251112 21:00:30 d64a58a17330f0ed 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251112 21:00:30 d64a58a17330f0ed 77936867 validating proof residues for power 8
20251112 21:00:30 d64a58a17330f0ed 77936867 Proof using power 8
20251112 21:00:36 d64a58a17330f0ed 77936867 OK 10000 0.01% fc4f135f7cf4ad29 530 us/it + check 0.28s + save 0.09s; ETA 11:28
20251112 21:00:41 d64a58a17330f0ed 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 530 us/it + check 0.28s + save 0.11s; ETA 11:28
20251112 21:00:47 d64a58a17330f0ed 77936867 Stopping, please wait..
20251112 21:00:47 d64a58a17330f0ed 77936867 OK 30000 0.04% c4e0ff35e3290d98 529 us/it + check 0.28s + save 0.09s; ETA 11:27
20251112 21:00:47 d64a58a17330f0ed Exiting because "stop requested"
20251112 21:00:47 d64a58a17330f0ed Bye
================================================
20251112 21:00:47 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 21:00:47 GpuOwl VERSION v7.2-131-gca22dce-dirty
20251112 21:00:47 Note: not found 'config.txt'
20251112 21:00:47 config: -prp 77936867 -iters 30000 -log 10000 -fft 4M -d 7
20251112 21:00:47 device 7, unique id 'f890794172e62691'
20251112 21:00:47 f890794172e62691 77936867 FFT: 4M 1K:8:256 (18.58 bpw)
20251112 21:00:47 f890794172e62691 77936867 OpenCL args "-DEXP=77936867u -DWIDTH=1024u -DSMALL_HEIGHT=256u -DMIDDLE=8u -DAMDGPU=1 -DMM_CHAIN=1u -DMM2_CHAIN=2u -DWEIGHT_STEP=0.33644726404543274 -DIWEIGHT_STEP=-0.25174750481886216 -DIWEIGHTS={0,-0.44011820345520131,-0.37306474779553728,-0.29798072935699788,-0.21390437908665341,-0.11975874301407295,-0.014337887291734644,-0.44814572555075455,} -DFWEIGHTS={0,0.78609128957452257,0.5950610473469905,0.42446232150303748,0.2721098723818392,0.1360521812214803,0.014546452690911484,0.81207258201996746,} -cl-std=CL2.0 -cl-finite-math-only "
20251112 21:00:48 f890794172e62691 77936867 OpenCL compilation in 0.73 s
20251112 21:00:48 f890794172e62691 77936867 PRP starting from beginning
20251112 21:00:48 f890794172e62691 77936867 OK 0 on-load: blockSize 400, 0000000000000003
20251112 21:00:48 f890794172e62691 77936867 validating proof residues for power 8
20251112 21:00:48 f890794172e62691 77936867 Proof using power 8
20251112 21:00:54 f890794172e62691 77936867 OK 10000 0.01% fc4f135f7cf4ad29 525 us/it + check 0.26s + save 0.09s; ETA 11:21
20251112 21:01:00 f890794172e62691 77936867 OK 20000 0.03% 3cd1bd9d5e09cbc5 526 us/it + check 0.26s + save 0.09s; ETA 11:23
20251112 21:01:05 f890794172e62691 77936867 Stopping, please wait..
20251112 21:01:05 f890794172e62691 77936867 OK 30000 0.04% c4e0ff35e3290d98 531 us/it + check 0.26s + save 0.09s; ETA 11:30
20251112 21:01:05 f890794172e62691 Exiting because "stop requested"
20251112 21:01:05 f890794172e62691 Bye
================================================
hermann@7600x:~/gpuowl-v7.2-131-gca22dce-ubuntu$
Author
1×16GB RAM 7600X
1×32GB VRAM Instinct MI50
7×16GB VRAM Instinct MI50
1×8GB VRAM iGPU
hermann@7600x:~$ rocminfo| egrep "( Size:[ ]*[0-9][0-9][0-9]|pute|Uuid:|^ Name:)"
Name: AMD Ryzen 5 7600X 6-Core Processor
Uuid: CPU-XX
Compute Unit: 12
Size: 15461884(0xebedfc) KB
Size: 15461884(0xebedfc) KB
Size: 15461884(0xebedfc) KB
Size: 15461884(0xebedfc) KB
Name: gfx906
Uuid: GPU-4124412172e62126
Compute Unit: 60
Size: 33538048(0x1ffc000) KB
Size: 33538048(0x1ffc000) KB
Name: gfx906
Uuid: GPU-c49e19417337ece3
Compute Unit: 60
Size: 16760832(0xffc000) KB
Size: 16760832(0xffc000) KB
Name: gfx906
Uuid: GPU-6a0e7961732c730d
Compute Unit: 60
Size: 16760832(0xffc000) KB
Size: 16760832(0xffc000) KB
Name: gfx906
Uuid: GPU-13c24061732c730c
Compute Unit: 60
Size: 16760832(0xffc000) KB
Size: 16760832(0xffc000) KB
Name: gfx906
Uuid: GPU-304c70e172dc768c
Compute Unit: 60
Size: 16760832(0xffc000) KB
Size: 16760832(0xffc000) KB
Name: gfx906
Uuid: GPU-6e56508172dc76b6
Compute Unit: 60
Size: 16760832(0xffc000) KB
Size: 16760832(0xffc000) KB
Name: gfx906
Uuid: GPU-d64a58a17330f0ed
Compute Unit: 60
Size: 16760832(0xffc000) KB
Size: 16760832(0xffc000) KB
Name: gfx906
Uuid: GPU-f890794172e62691
Compute Unit: 60
Size: 16760832(0xffc000) KB
Size: 16760832(0xffc000) KB
Name: gfx1036
Uuid: GPU-XX
Compute Unit: 2
Size: 7730940(0x75f6fc) KB
Size: 7730940(0x75f6fc) KB
hermann@7600x:~$
Author
Today system booted first time since last November.
Previous comment output shows iGPU output a bottom.
So internal graphics was enabled.
But that does not allow to work with all 8 Instinct MI50s for above clpeak test.
I disabled iGPU in Asrock bios "Advanced > AMD CBS > NBIO Common Options > GFX Configuration" and rebooted.
No display as expected, but no boot either, and after powercycling wakeonlan did not work.
Powered off, disconnected power cable, opened case, did short Jumper 7 (clear cmos) for more than 3 seconds.
Powering then started with iGPU.
After disabling iGPU in Asrock bios again, this time shudown and wakeonlan later worked.
I had to relearn that clinfo sometimes produces no output. In that case, just execute a 2nd time to get output.
With iGPU disabled now all eight Instinct MI50 GPUs work with clpeak:
hermann@7600x:~$ for((d=0;d<8;++d)); do echo -n "$d"; clpeak --compute-dp -d $d | grep "double "; done
0 double : 5956.04
1 double : 6207.20
2 double : 6241.84
3 double : 6244.02
4 double : 6265.59
5 double : 6250.40
6 double : 6224.31
7 double : 6245.23
hermann@7600x:~$
Author
System idle shows 410W draw.
Running 1xgpuowl shows 488W.
Running 2xgpuowl in parallel shows 733W.
Running 3xgpuowl in parallel shows 991W.
Running 4xgpuowl in parallel shows 1251W.
Todo:
Use SATA dual PSU power sync to add 1850W Zumax PSU to internal 2000W Zumax PSU, allowing to run all 8 MI50s in parallel.
Top beige parts are this gist model turned upside down in PrusaSlicer and then moved into the ground so that the two higher parts get cut by PrusaSlicer. The reason that the top in photo is very smooth is that it had direct contact with 3D printer plate while printing:
Author
Added 2nd PSU with SATA dual PSU power sync, connected devices 0-3 (from top) and corresponding riser cards and BBQ blower fans to new Zumax 1850W PSU. Devices 4-7 plus are powered by Zumax 2000W PSU inside PC.
Sequential clpeak works:
hermann@7600x:~/gpuowl-v7.2-131-gca22dce-ubuntu$ for((d=0;d<8;++d)); do echo -n "$d"; clpeak --compute-dp -d $d | grep "double "; done
0 double : 5938.33
1 double : 6243.94
2 double : 6169.87
3 double : 6220.46
4 double : 6245.27
5 double : 6228.60
6 double : 6229.20
7 double : 6241.56
hermann@7600x:~/gpuowl-v7.2-131-gca22dce-ubuntu$
Parallel gpuowl works as well, and system draws 2285W with all 8 GPUs working:
hermann@7600x:~/gpuowl-v7.2-131-gca22dce-ubuntu$ cat doit
#!/bin/bash
for((d=0; d<8; ++d))
do
cd /home/hermann/gpuowl-v7.2-131-gca22dce-ubuntu/$d
rm -rf 77*
/home/hermann/gpuowl-v7.2-131-gca22dce-ubuntu/gpuowl -prp 77936867 -iters 50000 -log 10000 -fft 4M -d $d &
echo $d
done
hermann@7600x:~/gpuowl-v7.2-131-gca22dce-ubuntu$
Reason for "only" 2285W with MI50 TDP of 300W is that the 32GB MI50 has a cap at 225W, and the other seven 16GB MI50s have a 250W cap.
Part of output created (column right of time is GPU Uuid):
...
20260402 14:00:06 c49e19417337ece3 77936867 Proof using power 8
20260402 14:00:12 4124412172e62126 77936867 OK 10000 0.01% fc4f135f7cf4ad29 535 us/it + check 0.27s + save 0.09s; ETA 11:35
20260402 14:00:12 f890794172e62691 77936867 OK 10000 0.01% fc4f135f7cf4ad29 527 us/it + check 0.26s + save 0.09s; ETA 11:25
20260402 14:00:12 304c70e172dc768c 77936867 OK 10000 0.01% fc4f135f7cf4ad29 514 us/it + check 0.39s + save 0.12s; ETA 11:07
20260402 14:00:12 6a0e7961732c730d 77936867 OK 10000 0.01% fc4f135f7cf4ad29 529 us/it + check 0.38s + save 0.10s; ETA 11:27
20260402 14:00:12 6e56508172dc76b6 77936867 OK 10000 0.01% fc4f135f7cf4ad29 514 us/it + check 0.39s + save 0.12s; ETA 11:07
20260402 14:00:12 13c24061732c730c 77936867 OK 10000 0.01% fc4f135f7cf4ad29 530 us/it + check 0.40s + save 0.12s; ETA 11:28
20260402 14:00:12 d64a58a17330f0ed 77936867 OK 10000 0.01% fc4f135f7cf4ad29 530 us/it + check 0.29s + save 0.09s; ETA 11:28
20260402 14:00:12 c49e19417337ece3 77936867 OK 10000 0.01% fc4f135f7cf4ad29 529 us/it + check 0.28s + save 0.09s; ETA 11:27
...
2 PSUs bringup:
- PC case front closed again because the cables leaving front are not needed anymore
- when installed in basement unused cables from 2nd PSU will be secured in plastic and moved to back
- current system for bringup only
- sometimes only 7 GPUs run (clpeak as well as gpuowl);
it seems device 0 is the device not starting in such situation - only (safe) connection between both PSUs is SATA from PC and power connecter from 2nd PSU
in SATA dual PSU power sync (between 2nd PSU left bottom and white desk light base) - green BBQ blower fan connector attached to only Instinct MI50 32GB, the other 7 are Instinct MI50 16GB versions
Author
Today I made the move to basement.
There was enough space for my AMD 7950X CPU PC on left side in addition, on 80x50cm Ikea Ivar shelf board.
Since no external GPUs need to be connected to 7950X PC, the power cords leaving right side of it case are secured in plastic on top.
Author
"Audio demo of BBQ blower fans attached to 8× AMD Instinct MI50 GPUs of 50 TFLOPs FP64 PC"
https://youtu.be/czedHB-gsNA
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The model in JSCAD:
https://jscad.app/#https://gist.github.com/Hermann-SW/1d0b7f84610e1aee657ab1cad8e09460/raw/108ea6a689a6cc840f11716871df464a3c67d85b/stacking.Instinct_MI50s.jscad