Last active
December 17, 2020 16:53
-
-
Save varungandhi-apple/8d5d811d12b1403c4d203ee5c14be28c to your computer and use it in GitHub Desktop.
Swiftc assertions overhead
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # 2020/12/16 Swift compiler performance measurements with/without assertions | |
| Configuration: | |
| - 2017 iMac Pro with 10-core Intel Xeon processor | |
| - Build-matrix of {Alamofire, swift-nio, swift-syntax} x {debug, release} x {5 compiler configurations} | |
| - Note: hyperfine's "Nx times faster" really refers to time ratios, not speed ratios. | |
| Benchmark #1: ./compile_package.sh Alamofire debug SwiftNoAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 11.235 s ± 0.183 s [User: 86.212 s, System: 16.691 s] | |
| Range (min … max): 11.019 s … 11.814 s 15 runs | |
| Benchmark #2: ./compile_package.sh Alamofire debug SwiftAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 12.393 s ± 0.152 s [User: 99.257 s, System: 16.744 s] | |
| Range (min … max): 12.192 s … 12.773 s 15 runs | |
| Benchmark #3: ./compile_package.sh Alamofire debug SwiftAssert_LLVMNoAssert | |
| Time (mean ± σ): 12.404 s ± 0.070 s [User: 101.230 s, System: 16.199 s] | |
| Range (min … max): 12.284 s … 12.555 s 15 runs | |
| Benchmark #4: ./compile_package.sh Alamofire debug SwiftAssert_LLVMAssert_NoSILVerify | |
| Time (mean ± σ): 12.884 s ± 0.092 s [User: 102.343 s, System: 16.541 s] | |
| Range (min … max): 12.702 s … 13.001 s 15 runs | |
| Benchmark #5: ./compile_package.sh Alamofire debug SwiftAssert_LLVMAssert | |
| Time (mean ± σ): 12.958 s ± 0.124 s [User: 104.457 s, System: 16.244 s] | |
| Range (min … max): 12.805 s … 13.300 s 15 runs | |
| Summary | |
| './compile_package.sh Alamofire debug SwiftNoAssert_LLVMNoAssert_NoSILVerify' ran | |
| 1.10 ± 0.02 times faster than './compile_package.sh Alamofire debug SwiftAssert_LLVMNoAssert_NoSILVerify' | |
| 1.10 ± 0.02 times faster than './compile_package.sh Alamofire debug SwiftAssert_LLVMNoAssert' | |
| 1.15 ± 0.02 times faster than './compile_package.sh Alamofire debug SwiftAssert_LLVMAssert_NoSILVerify' | |
| 1.15 ± 0.02 times faster than './compile_package.sh Alamofire debug SwiftAssert_LLVMAssert' | |
| Benchmark #1: ./compile_package.sh Alamofire release SwiftNoAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 18.022 s ± 0.093 s [User: 30.719 s, System: 1.473 s] | |
| Range (min … max): 17.860 s … 18.175 s 15 runs | |
| Benchmark #2: ./compile_package.sh Alamofire release SwiftAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 20.415 s ± 0.102 s [User: 33.237 s, System: 1.555 s] | |
| Range (min … max): 20.203 s … 20.603 s 15 runs | |
| Benchmark #3: ./compile_package.sh Alamofire release SwiftAssert_LLVMNoAssert | |
| Time (mean ± σ): 22.576 s ± 0.062 s [User: 35.466 s, System: 1.540 s] | |
| Range (min … max): 22.483 s … 22.675 s 15 runs | |
| Benchmark #4: ./compile_package.sh Alamofire release SwiftAssert_LLVMAssert_NoSILVerify | |
| Time (mean ± σ): 21.908 s ± 0.076 s [User: 39.104 s, System: 1.543 s] | |
| Range (min … max): 21.774 s … 22.044 s 15 runs | |
| Benchmark #5: ./compile_package.sh Alamofire release SwiftAssert_LLVMAssert | |
| Time (mean ± σ): 23.863 s ± 0.187 s [User: 40.732 s, System: 1.542 s] | |
| Range (min … max): 23.619 s … 24.119 s 15 runs | |
| Summary | |
| './compile_package.sh Alamofire release SwiftNoAssert_LLVMNoAssert_NoSILVerify' ran | |
| 1.13 ± 0.01 times faster than './compile_package.sh Alamofire release SwiftAssert_LLVMNoAssert_NoSILVerify' | |
| 1.22 ± 0.01 times faster than './compile_package.sh Alamofire release SwiftAssert_LLVMAssert_NoSILVerify' | |
| 1.25 ± 0.01 times faster than './compile_package.sh Alamofire release SwiftAssert_LLVMNoAssert' | |
| 1.32 ± 0.01 times faster than './compile_package.sh Alamofire release SwiftAssert_LLVMAssert' | |
| Benchmark #1: ./compile_package.sh swift-nio debug SwiftNoAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 16.441 s ± 0.241 s [User: 66.832 s, System: 17.130 s] | |
| Range (min … max): 15.966 s … 16.725 s 15 runs | |
| Benchmark #2: ./compile_package.sh swift-nio debug SwiftAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 18.415 s ± 0.233 s [User: 80.411 s, System: 17.562 s] | |
| Range (min … max): 18.144 s … 19.103 s 15 runs | |
| Benchmark #3: ./compile_package.sh swift-nio debug SwiftAssert_LLVMNoAssert | |
| Time (mean ± σ): 19.334 s ± 0.286 s [User: 87.007 s, System: 17.657 s] | |
| Range (min … max): 18.809 s … 19.922 s 15 runs | |
| Benchmark #4: ./compile_package.sh swift-nio debug SwiftAssert_LLVMAssert_NoSILVerify | |
| Time (mean ± σ): 19.066 s ± 0.237 s [User: 83.968 s, System: 17.728 s] | |
| Range (min … max): 18.674 s … 19.557 s 15 runs | |
| Benchmark #5: ./compile_package.sh swift-nio debug SwiftAssert_LLVMAssert | |
| Time (mean ± σ): 20.145 s ± 0.200 s [User: 90.898 s, System: 17.664 s] | |
| Range (min … max): 19.817 s … 20.353 s 15 runs | |
| Summary | |
| './compile_package.sh swift-nio debug SwiftNoAssert_LLVMNoAssert_NoSILVerify' ran | |
| 1.12 ± 0.02 times faster than './compile_package.sh swift-nio debug SwiftAssert_LLVMNoAssert_NoSILVerify' | |
| 1.16 ± 0.02 times faster than './compile_package.sh swift-nio debug SwiftAssert_LLVMAssert_NoSILVerify' | |
| 1.18 ± 0.02 times faster than './compile_package.sh swift-nio debug SwiftAssert_LLVMNoAssert' | |
| 1.23 ± 0.02 times faster than './compile_package.sh swift-nio debug SwiftAssert_LLVMAssert' | |
| Benchmark #1: ./compile_package.sh swift-nio release SwiftNoAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 34.260 s ± 0.156 s [User: 90.878 s, System: 6.948 s] | |
| Range (min … max): 34.037 s … 34.703 s 15 runs | |
| Benchmark #2: ./compile_package.sh swift-nio release SwiftAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 39.861 s ± 0.192 s [User: 101.213 s, System: 7.231 s] | |
| Range (min … max): 39.461 s … 40.173 s 15 runs | |
| Benchmark #3: ./compile_package.sh swift-nio release SwiftAssert_LLVMNoAssert | |
| Time (mean ± σ): 44.832 s ± 0.129 s [User: 108.782 s, System: 7.221 s] | |
| Range (min … max): 44.549 s … 44.997 s 15 runs | |
| Benchmark #4: ./compile_package.sh swift-nio release SwiftAssert_LLVMAssert_NoSILVerify | |
| Time (mean ± σ): 42.090 s ± 0.150 s [User: 116.091 s, System: 7.223 s] | |
| Range (min … max): 41.859 s … 42.459 s 15 runs | |
| Benchmark #5: ./compile_package.sh swift-nio release SwiftAssert_LLVMAssert | |
| Time (mean ± σ): 47.232 s ± 0.286 s [User: 123.783 s, System: 7.320 s] | |
| Range (min … max): 46.888 s … 47.854 s 15 runs | |
| Summary | |
| './compile_package.sh swift-nio release SwiftNoAssert_LLVMNoAssert_NoSILVerify' ran | |
| 1.16 ± 0.01 times faster than './compile_package.sh swift-nio release SwiftAssert_LLVMNoAssert_NoSILVerify' | |
| 1.23 ± 0.01 times faster than './compile_package.sh swift-nio release SwiftAssert_LLVMAssert_NoSILVerify' | |
| 1.31 ± 0.01 times faster than './compile_package.sh swift-nio release SwiftAssert_LLVMNoAssert' | |
| 1.38 ± 0.01 times faster than './compile_package.sh swift-nio release SwiftAssert_LLVMAssert' | |
| Benchmark #1: ./compile_package.sh swift-syntax debug SwiftNoAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 24.211 s ± 0.139 s [User: 123.389 s, System: 11.159 s] | |
| Range (min … max): 23.967 s … 24.492 s 15 runs | |
| Benchmark #2: ./compile_package.sh swift-syntax debug SwiftAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 27.596 s ± 0.208 s [User: 139.876 s, System: 11.273 s] | |
| Range (min … max): 27.344 s … 28.036 s 15 runs | |
| Benchmark #3: ./compile_package.sh swift-syntax debug SwiftAssert_LLVMNoAssert | |
| Time (mean ± σ): 29.385 s ± 0.191 s [User: 156.033 s, System: 10.970 s] | |
| Range (min … max): 29.084 s … 29.807 s 15 runs | |
| Benchmark #4: ./compile_package.sh swift-syntax debug SwiftAssert_LLVMAssert_NoSILVerify | |
| Time (mean ± σ): 28.525 s ± 0.203 s [User: 146.483 s, System: 11.251 s] | |
| Range (min … max): 28.234 s … 29.086 s 15 runs | |
| Benchmark #5: ./compile_package.sh swift-syntax debug SwiftAssert_LLVMAssert | |
| Time (mean ± σ): 30.345 s ± 0.092 s [User: 163.234 s, System: 11.080 s] | |
| Range (min … max): 30.129 s … 30.482 s 15 runs | |
| Summary | |
| './compile_package.sh swift-syntax debug SwiftNoAssert_LLVMNoAssert_NoSILVerify' ran | |
| 1.14 ± 0.01 times faster than './compile_package.sh swift-syntax debug SwiftAssert_LLVMNoAssert_NoSILVerify' | |
| 1.18 ± 0.01 times faster than './compile_package.sh swift-syntax debug SwiftAssert_LLVMAssert_NoSILVerify' | |
| 1.21 ± 0.01 times faster than './compile_package.sh swift-syntax debug SwiftAssert_LLVMNoAssert' | |
| 1.25 ± 0.01 times faster than './compile_package.sh swift-syntax debug SwiftAssert_LLVMAssert' | |
| Benchmark #1: ./compile_package.sh swift-syntax release SwiftNoAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 131.296 s ± 2.658 s [User: 207.820 s, System: 7.311 s] | |
| Range (min … max): 127.739 s … 136.832 s 15 runs | |
| Benchmark #2: ./compile_package.sh swift-syntax release SwiftAssert_LLVMNoAssert_NoSILVerify | |
| Time (mean ± σ): 147.887 s ± 3.682 s [User: 225.534 s, System: 7.453 s] | |
| Range (min … max): 143.747 s … 155.613 s 15 runs | |
| Benchmark #3: ./compile_package.sh swift-syntax release SwiftAssert_LLVMNoAssert | |
| Time (mean ± σ): 165.385 s ± 2.418 s [User: 243.244 s, System: 8.137 s] | |
| Range (min … max): 162.419 s … 171.516 s 15 runs | |
| Benchmark #4: ./compile_package.sh swift-syntax release SwiftAssert_LLVMAssert_NoSILVerify | |
| Time (mean ± σ): 152.028 s ± 2.178 s [User: 251.361 s, System: 7.473 s] | |
| Range (min … max): 148.959 s … 155.948 s 15 runs | |
| Benchmark #5: ./compile_package.sh swift-syntax release SwiftAssert_LLVMAssert | |
| Time (mean ± σ): 169.388 s ± 2.605 s [User: 268.496 s, System: 7.513 s] | |
| Range (min … max): 164.706 s … 173.245 s 15 runs | |
| Summary | |
| './compile_package.sh swift-syntax release SwiftNoAssert_LLVMNoAssert_NoSILVerify' ran | |
| 1.13 ± 0.04 times faster than './compile_package.sh swift-syntax release SwiftAssert_LLVMNoAssert_NoSILVerify' | |
| 1.16 ± 0.03 times faster than './compile_package.sh swift-syntax release SwiftAssert_LLVMAssert_NoSILVerify' | |
| 1.26 ± 0.03 times faster than './compile_package.sh swift-syntax release SwiftAssert_LLVMNoAssert' | |
| 1.29 ± 0.03 times faster than './compile_package.sh swift-syntax release SwiftAssert_LLVMAssert' |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment