benchmarks-2024-10-18.md

Benchmarking rustls 0.23.15 vs OpenSSL 3.3.2 vs BoringSSL

System configuration

We ran the benchmarks on a bare-metal server with the following characteristics:

• OS: Debian 12 (Bookworm).
• C/C++ toolchains: GCC 12.2.0 and Clang 14.0.6.
• CPU: Xeon E-2386G (supporting AVX-512).
• Memory: 32GB.
• Extra configuration: hyper-threading disabled, dynamic frequency scaling disabled, cpu scaling governor set to performance for all cores.

Versions

The benchmarking tool used for both OpenSSL and BoringSSL was openssl-bench d5de57d9.

This was built from source with its makefile.

BoringSSL

The tested version of BoringSSL is 76968bb3d5, which was the most recent point on master when we started these measurements.

BoringSSL was built from source with CC=clang CXX=clang++ cmake -DCMAKE_BUILD_TYPE=Release. clang is used here to avoid potential performance deficits to GCC.

OpenSSL

The tested version of OpenSSL is 3.3.2, which was the latest release at the time of writing.

OpenSSL was built from source with ./Configure ; make -j12.

Rustls

The tested version of rustls was 0.23.15, which was the latest release at the time of writing. This was used with aws-lc-rs 1.10.0 / aws-lc-sys 0.22.0.

Additionally the following two commits were included, which affect the benchmark tool but do not affect the core crate:

• https://github.com/rustls/rustls/commit/13144a0aa391bbec55aa92ee020e88c2bb8c3ea8
• https://github.com/rustls/rustls/commit/b553880a5f5caf58bbd2c43e4031e8c55d6da486

Measurements

BoringSSL was tested with this command:

~/bench/openssl-bench
$ BENCH_MULTIPLIER=16 setarch -R make measure BORINGSSL=1

OpenSSL was tested with this command:

~/bench/openssl-bench
$ BENCH_MULTIPLIER=16 setarch -R make measure

rustls was tested with this command:

~/bench/rustls
$ BENCH_MULTIPLIER=16 setarch -R make -f admin/bench-measure.mk measure

Observations on results

AVX-512 support shows up twice in these results:

• rustls/aws-lc and OpenSSL's performance advantage in throughput tests is due to use of AVX-512F/VAES.
• rustls/aws-lc and OpenSSL's performance advantage in server-side full handshake tests is due to use of AVX-512IFMA-accelerated RSA.

This support was contributed to the respective projects by Intel.

TLS1.3 resumption is slower than TLS1.2 resumption because it includes a fresh key exchange.