Multiplying a 60000x60000 matrix with itself in torch

gistfile1.txt

float32: 3.71 s ± 2.95 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
float16: 2.29 s ± 8.15 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
bfloat16: 2.29 s ± 9.57 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

torch.backends.cuda.matmul.allow_tf32 = False
float32: 24.4 s ± 41.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)