Skip to content

Instantly share code, notes, and snippets.

@nathanchance

nathanchance/llvm-11.0.0-results-1.md Secret

Last active August 27, 2020 14:31
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nathanchance/3ac507d20850a02172a083148809f63b to your computer and use it in GitHub Desktop.
Save nathanchance/3ac507d20850a02172a083148809f63b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
llvm-11.0.0-results-1.md

LLVM 11.0.0 benchmarks

Testing scripts: https://github.com/nathanchance/tc-build/tree/14611ae3d6280907f18bbb0b07eccca854aba285/benchmark

m2.xlarge.x86 results

Total runtime: 4d 20h 38m 10s

LLVM build times (7 runs)

All ./build-llvm.py invocations were run with --check-targets clang lld llvm --no-ccache --no-update.

Command Mean [s] Min [s] Max [s] Relative
./build-llvm.py --build-stage1-only 542.569 ± 0.933 541.578 544.256 1.00
./build-llvm.py 811.556 ± 1.178 809.899 812.746 1.50 ± 0.00
./build-llvm.py --lto=thin 2687.647 ± 3.812 2683.183 2693.010 4.95 ± 0.01
./build-llvm.py --lto=full 7148.708 ± 75.290 7045.807 7223.985 13.18 ± 0.14
./build-llvm.py --pgo 3005.483 ± 7.151 2999.534 3019.566 5.54 ± 0.02
./build-llvm.py --lto=thin --pgo 4967.305 ± 10.972 4948.391 4982.013 9.16 ± 0.03
./build-llvm.py --lto=full --pgo 6484.316 ± 24.973 6448.183 6523.901 11.95 ± 0.05

arm32 defconfig (56 runs)

Command Mean [s] Min [s] Max [s] Relative
GCC 10.2.0 96.802 ± 0.078 96.627 97.013 1.08 ± 0.00
LLVM (1 stage) 115.212 ± 0.080 115.079 115.378 1.29 ± 0.00
LLVM (2 stage) 114.679 ± 0.100 114.469 115.013 1.28 ± 0.00
LLVM (ThinLTO) 110.278 ± 0.086 110.115 110.448 1.23 ± 0.00
LLVM (LTO) 109.705 ± 0.090 109.521 109.932 1.23 ± 0.00
LLVM (PGO) 93.988 ± 0.098 93.733 94.168 1.05 ± 0.00
LLVM (PGO + ThinLTO) 89.496 ± 0.079 89.325 89.688 1.00
LLVM (PGO + LTO) 90.288 ± 0.082 90.112 90.442 1.01 ± 0.00

arm64 defconfig (56 runs)

Command Mean [s] Min [s] Max [s] Relative
GCC 10.2.0 138.002 ± 0.117 137.697 138.248 1.00
LLVM (1 stage) 183.506 ± 0.190 183.209 184.096 1.33 ± 0.00
LLVM (2 stage) 183.226 ± 0.172 182.871 183.707 1.33 ± 0.00
LLVM (ThinLTO) 174.654 ± 0.145 174.429 174.989 1.27 ± 0.00
LLVM (LTO) 174.250 ± 0.138 174.031 174.609 1.26 ± 0.00
LLVM (PGO) 152.818 ± 0.164 152.498 153.273 1.11 ± 0.00
LLVM (PGO + ThinLTO) 145.098 ± 3.463 143.652 161.880 1.05 ± 0.03
LLVM (PGO + LTO) 146.300 ± 3.237 144.262 157.799 1.06 ± 0.02

powerpc ppc44x_defconfig (56 runs)

Command Mean [s] Min [s] Max [s] Relative
GCC 10.2.0 29.974 ± 0.395 29.582 31.943 1.09 ± 0.02
LLVM (1 stage) 34.156 ± 0.370 33.335 34.710 1.24 ± 0.02
LLVM (2 stage) 34.137 ± 0.333 33.244 34.618 1.24 ± 0.02
LLVM (ThinLTO) 32.875 ± 0.325 32.039 33.332 1.19 ± 0.02
LLVM (LTO) 32.515 ± 0.340 31.781 32.963 1.18 ± 0.02
LLVM (PGO) 28.827 ± 0.273 28.060 29.235 1.05 ± 0.01
LLVM (PGO + ThinLTO) 27.530 ± 0.278 26.811 27.970 1.00
LLVM (PGO + LTO) 27.697 ± 0.284 27.023 28.151 1.01 ± 0.01

powerpc powernv_defconfig (56 runs)

Command Mean [s] Min [s] Max [s] Relative
GCC 10.2.0 87.717 ± 0.235 87.014 88.053 1.02 ± 0.01
LLVM (1 stage) 106.545 ± 0.565 105.047 107.567 1.24 ± 0.01
LLVM (2 stage) 106.184 ± 0.522 104.563 106.829 1.24 ± 0.01
LLVM (ThinLTO) 101.995 ± 0.556 100.544 103.097 1.19 ± 0.01
LLVM (LTO) 101.899 ± 0.448 100.200 102.553 1.19 ± 0.01
LLVM (PGO) 90.041 ± 0.419 88.888 90.765 1.05 ± 0.01
LLVM (PGO + ThinLTO) 85.667 ± 0.375 84.417 86.191 1.00
LLVM (PGO + LTO) 86.823 ± 0.583 84.185 87.490 1.01 ± 0.01

x86_64 defconfig (56 runs)

Command Mean [s] Min [s] Max [s] Relative
GCC 10.2.0 59.964 ± 0.096 59.772 60.172 1.00
LLVM (1 stage) 81.672 ± 0.157 81.235 82.122 1.36 ± 0.00
LLVM (2 stage) 79.080 ± 0.118 78.842 79.342 1.32 ± 0.00
LLVM (ThinLTO) 78.675 ± 0.137 78.345 78.939 1.31 ± 0.00
LLVM (LTO) 78.241 ± 0.104 78.012 78.434 1.30 ± 0.00
LLVM (PGO) 69.520 ± 0.122 69.028 69.859 1.16 ± 0.00
LLVM (PGO + ThinLTO) 66.697 ± 0.107 66.417 66.934 1.11 ± 0.00
LLVM (PGO + LTO) 66.812 ± 0.119 66.562 67.090 1.11 ± 0.00

x2.xlarge.x86 results

Total runtime: 4d 20h 55m 39s

LLVM build times (7 runs)

All ./build-llvm.py invocations were run with --check-targets clang lld llvm --no-ccache --no-update.

Command Mean [s] Min [s] Max [s] Relative
./build-llvm.py --build-stage1-only 543.944 ± 1.284 542.578 546.155 1.00
./build-llvm.py 811.632 ± 1.069 810.241 813.303 1.49 ± 0.00
./build-llvm.py --lto=thin 2697.264 ± 6.473 2691.280 2710.341 4.96 ± 0.02
./build-llvm.py --lto=full 7149.724 ± 70.788 7032.389 7238.756 13.14 ± 0.13
./build-llvm.py --pgo 3013.860 ± 8.327 3002.389 3024.829 5.54 ± 0.02
./build-llvm.py --lto=thin --pgo 4966.329 ± 8.547 4951.493 4975.028 9.13 ± 0.03
./build-llvm.py --lto=full --pgo 6493.604 ± 48.256 6418.230 6556.303 11.94 ± 0.09

arm32 defconfig (56 runs)

Command Mean [s] Min [s] Max [s] Relative
GCC 10.2.0 96.886 ± 0.092 96.690 97.192 1.08 ± 0.00
LLVM (1 stage) 115.009 ± 0.069 114.860 115.217 1.29 ± 0.00
LLVM (2 stage) 114.704 ± 0.076 114.593 114.877 1.28 ± 0.00
LLVM (ThinLTO) 110.251 ± 0.083 110.048 110.431 1.23 ± 0.00
LLVM (LTO) 109.660 ± 0.088 109.467 109.854 1.23 ± 0.00
LLVM (PGO) 93.913 ± 0.106 93.685 94.135 1.05 ± 0.00
LLVM (PGO + ThinLTO) 89.463 ± 0.096 89.286 89.684 1.00
LLVM (PGO + LTO) 90.241 ± 0.096 90.075 90.510 1.01 ± 0.00

arm64 defconfig (56 runs)

Command Mean [s] Min [s] Max [s] Relative
GCC 10.2.0 138.027 ± 0.146 137.758 138.822 1.00
LLVM (1 stage) 185.463 ± 1.223 184.418 190.713 1.34 ± 0.01
LLVM (2 stage) 185.103 ± 0.672 184.418 188.927 1.34 ± 0.01
LLVM (ThinLTO) 177.006 ± 0.876 175.965 180.042 1.28 ± 0.01
LLVM (LTO) 176.554 ± 0.540 175.870 178.127 1.28 ± 0.00
LLVM (PGO) 155.981 ± 1.228 154.678 159.984 1.13 ± 0.01
LLVM (PGO + ThinLTO) 148.719 ± 1.591 146.683 154.314 1.08 ± 0.01
LLVM (PGO + LTO) 150.446 ± 1.573 148.065 154.260 1.09 ± 0.01

powerpc ppc44x_defconfig (56 runs)

Command Mean [s] Min [s] Max [s] Relative
GCC 10.2.0 29.967 ± 0.286 29.592 31.157 1.09 ± 0.01
LLVM (1 stage) 34.102 ± 0.359 33.230 34.795 1.24 ± 0.02
LLVM (2 stage) 34.074 ± 0.356 33.304 34.541 1.24 ± 0.02
LLVM (ThinLTO) 32.776 ± 0.382 31.910 33.356 1.19 ± 0.02
LLVM (LTO) 32.451 ± 0.370 31.582 33.021 1.18 ± 0.02
LLVM (PGO) 28.737 ± 0.293 28.078 29.109 1.04 ± 0.01
LLVM (PGO + ThinLTO) 27.565 ± 0.218 26.921 27.898 1.00
LLVM (PGO + LTO) 27.632 ± 0.239 26.987 28.067 1.00 ± 0.01

powerpc powernv_defconfig (56 runs)

Command Mean [s] Min [s] Max [s] Relative
GCC 10.2.0 87.667 ± 0.268 86.914 88.063 1.02 ± 0.01
LLVM (1 stage) 106.222 ± 0.752 103.445 107.379 1.23 ± 0.01
LLVM (2 stage) 106.156 ± 0.510 104.187 107.021 1.23 ± 0.01
LLVM (ThinLTO) 101.956 ± 0.473 100.367 102.902 1.18 ± 0.01
LLVM (LTO) 101.660 ± 0.578 99.741 102.486 1.18 ± 0.01
LLVM (PGO) 90.003 ± 0.347 89.018 90.471 1.04 ± 0.01
LLVM (PGO + ThinLTO) 86.251 ± 0.370 84.899 86.874 1.00 ± 0.01
LLVM (PGO + LTO) 86.190 ± 0.364 84.707 86.725 1.00

x86_64 defconfig (56 runs)

Command Mean [s] Min [s] Max [s] Relative
GCC 10.2.0 59.981 ± 0.101 59.615 60.241 1.00
LLVM (1 stage) 81.578 ± 0.122 81.326 81.812 1.36 ± 0.00
LLVM (2 stage) 79.037 ± 0.186 78.116 79.394 1.32 ± 0.00
LLVM (ThinLTO) 78.608 ± 0.096 78.416 78.830 1.31 ± 0.00
LLVM (LTO) 78.231 ± 0.127 77.958 78.478 1.30 ± 0.00
LLVM (PGO) 69.543 ± 0.078 69.334 69.685 1.16 ± 0.00
LLVM (PGO + ThinLTO) 64.461 ± 0.092 64.181 64.637 1.07 ± 0.00
LLVM (PGO + LTO) 67.042 ± 0.116 66.761 67.398 1.12 ± 0.00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment