Bench.scala

package p1

import org.openjdk.jmh.annotations.Benchmark

object Bench {
  val values: List[Either[Int, Int]] = (1 to 100_000).map{
    case n if n % 2 == 0 =>
      Right(n)
    case n =>
      Left(n)
  }.toList
}

class Bench {

  @Benchmark
  def testMap: List[Either[Int, Int]] = {
    Bench.values.map(_.map(_ + 1))
  }

  @Benchmark
  def testMatch: List[Either[Int, Int]] = {
    Bench.values.map {
      case Right(x) => Right(x + 1)
      case x => x
    }
  }

}

build.properties

sbt.version=1.9.6

build.sbt

scalaVersion := "2.13.12"

enablePlugins(JmhPlugin)

plugin.sbt

addSbtPlugin("pl.project13.scala" % "sbt-jmh" % "0.4.6")

result-jdk11.txt

[info] running (fork) org.openjdk.jmh.Main -i 10 -wi 10 -f1
[info] # JMH version: 1.37
[info] # VM version: JDK 11.0.20, OpenJDK 64-Bit Server VM, 11.0.20+8-LTS
[info] # VM invoker: /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 10 iterations, 10 s each
[info] # Measurement: 10 iterations, 10 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: p1.Bench.testMap
[info] # Run progress: 0.00% complete, ETA 00:06:40
[info] # Fork: 1 of 1
[info] # Warmup Iteration   1: 1818.384 ops/s
[info] # Warmup Iteration   2: 1957.069 ops/s
[info] # Warmup Iteration   3: 1959.441 ops/s
[info] # Warmup Iteration   4: 1970.433 ops/s
[info] # Warmup Iteration   5: 1964.057 ops/s
[info] # Warmup Iteration   6: 1961.854 ops/s
[info] # Warmup Iteration   7: 1962.892 ops/s
[info] # Warmup Iteration   8: 1968.470 ops/s
[info] # Warmup Iteration   9: 1964.067 ops/s
[info] # Warmup Iteration  10: 1967.253 ops/s
[info] Iteration   1: 1966.387 ops/s
[info] Iteration   2: 1965.214 ops/s
[info] Iteration   3: 1965.601 ops/s
[info] Iteration   4: 1963.393 ops/s
[info] Iteration   5: 1964.866 ops/s
[info] Iteration   6: 1963.521 ops/s
[info] Iteration   7: 1961.859 ops/s
[info] Iteration   8: 1959.481 ops/s
[info] Iteration   9: 1956.051 ops/s
[info] Iteration  10: 1956.241 ops/s
[info] Result "p1.Bench.testMap":
[info]   1962.261 ±(99.9%) 5.729 ops/s [Average]
[info]   (min, avg, max) = (1956.051, 1962.261, 1966.387), stdev = 3.789
[info]   CI (99.9%): [1956.532, 1967.991] (assumes normal distribution)
[info] # JMH version: 1.37
[info] # VM version: JDK 11.0.20, OpenJDK 64-Bit Server VM, 11.0.20+8-LTS
[info] # VM invoker: /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: full + dont-inline hint (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 10 iterations, 10 s each
[info] # Measurement: 10 iterations, 10 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: p1.Bench.testMatch
[info] # Run progress: 50.00% complete, ETA 00:03:20
[info] # Fork: 1 of 1
[info] # Warmup Iteration   1: 1894.626 ops/s
[info] # Warmup Iteration   2: 1996.605 ops/s
[info] # Warmup Iteration   3: 1997.674 ops/s
[info] # Warmup Iteration   4: 2007.801 ops/s
[info] # Warmup Iteration   5: 2007.718 ops/s
[info] # Warmup Iteration   6: 2000.964 ops/s
[info] # Warmup Iteration   7: 2006.272 ops/s
[info] # Warmup Iteration   8: 2006.694 ops/s
[info] # Warmup Iteration   9: 2003.821 ops/s
[info] # Warmup Iteration  10: 2004.876 ops/s
[info] Iteration   1: 2006.769 ops/s
[info] Iteration   2: 1996.594 ops/s
[info] Iteration   3: 2003.908 ops/s
[info] Iteration   4: 2004.176 ops/s
[info] Iteration   5: 2000.728 ops/s
[info] Iteration   6: 2007.642 ops/s
[info] Iteration   7: 1999.289 ops/s
[info] Iteration   8: 2000.309 ops/s
[info] Iteration   9: 1998.661 ops/s
[info] Iteration  10: 2001.688 ops/s
[info] Result "p1.Bench.testMatch":
[info]   2001.976 ±(99.9%) 5.410 ops/s [Average]
[info]   (min, avg, max) = (1996.594, 2001.976, 2007.642), stdev = 3.578
[info]   CI (99.9%): [1996.566, 2007.387] (assumes normal distribution)
[info] # Run complete. Total time: 00:06:40
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark         Mode  Cnt     Score   Error  Units
[info] Bench.testMap    thrpt   10  1962.261 ± 5.729  ops/s
[info] Bench.testMatch  thrpt   10  2001.976 ± 5.410  ops/s

result-jdk21.txt

[info] running (fork) org.openjdk.jmh.Main -i 10 -wi 10 -f1
[info] # JMH version: 1.37
[info] # VM version: JDK 21, OpenJDK 64-Bit Server VM, 21+35
[info] # VM invoker: /Library/Java/JavaVirtualMachines/zulu-21.jdk/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 10 iterations, 10 s each
[info] # Measurement: 10 iterations, 10 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: p1.Bench.testMap
[info] # Run progress: 0.00% complete, ETA 00:06:40
[info] # Fork: 1 of 1
[info] # Warmup Iteration   1: 2006.489 ops/s
[info] # Warmup Iteration   2: 2089.905 ops/s
[info] # Warmup Iteration   3: 2062.312 ops/s
[info] # Warmup Iteration   4: 2102.730 ops/s
[info] # Warmup Iteration   5: 2100.759 ops/s
[info] # Warmup Iteration   6: 2099.667 ops/s
[info] # Warmup Iteration   7: 2084.376 ops/s
[info] # Warmup Iteration   8: 2089.750 ops/s
[info] # Warmup Iteration   9: 2103.360 ops/s
[info] # Warmup Iteration  10: 2102.282 ops/s
[info] Iteration   1: 2107.658 ops/s
[info] Iteration   2: 2111.269 ops/s
[info] Iteration   3: 2113.334 ops/s
[info] Iteration   4: 2111.127 ops/s
[info] Iteration   5: 2105.565 ops/s
[info] Iteration   6: 2113.801 ops/s
[info] Iteration   7: 2116.826 ops/s
[info] Iteration   8: 2107.052 ops/s
[info] Iteration   9: 2108.084 ops/s
[info] Iteration  10: 2112.979 ops/s
[info] Result "p1.Bench.testMap":
[info]   2110.769 ±(99.9%) 5.416 ops/s [Average]
[info]   (min, avg, max) = (2105.565, 2110.769, 2116.826), stdev = 3.582
[info]   CI (99.9%): [2105.353, 2116.185] (assumes normal distribution)
[info] # JMH version: 1.37
[info] # VM version: JDK 21, OpenJDK 64-Bit Server VM, 21+35
[info] # VM invoker: /Library/Java/JavaVirtualMachines/zulu-21.jdk/Contents/Home/bin/java
[info] # VM options: <none>
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: 10 iterations, 10 s each
[info] # Measurement: 10 iterations, 10 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Throughput, ops/time
[info] # Benchmark: p1.Bench.testMatch
[info] # Run progress: 50.00% complete, ETA 00:03:20
[info] # Fork: 1 of 1
[info] # Warmup Iteration   1: 2101.750 ops/s
[info] # Warmup Iteration   2: 2163.365 ops/s
[info] # Warmup Iteration   3: 2161.632 ops/s
[info] # Warmup Iteration   4: 2167.240 ops/s
[info] # Warmup Iteration   5: 2164.650 ops/s
[info] # Warmup Iteration   6: 2170.152 ops/s
[info] # Warmup Iteration   7: 2172.058 ops/s
[info] # Warmup Iteration   8: 2176.147 ops/s
[info] # Warmup Iteration   9: 2162.778 ops/s
[info] # Warmup Iteration  10: 2170.632 ops/s
[info] Iteration   1: 2174.601 ops/s
[info] Iteration   2: 2163.280 ops/s
[info] Iteration   3: 2167.545 ops/s
[info] Iteration   4: 2167.317 ops/s
[info] Iteration   5: 2169.817 ops/s
[info] Iteration   6: 2167.505 ops/s
[info] Iteration   7: 2175.125 ops/s
[info] Iteration   8: 2166.187 ops/s
[info] Iteration   9: 2176.735 ops/s
[info] Iteration  10: 2184.316 ops/s
[info] Result "p1.Bench.testMatch":
[info]   2171.243 ±(99.9%) 9.581 ops/s [Average]
[info]   (min, avg, max) = (2163.280, 2171.243, 2184.316), stdev = 6.337
[info]   CI (99.9%): [2161.661, 2180.824] (assumes normal distribution)
[info] # Run complete. Total time: 00:06:40
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise
[info] extra caution when trusting the results, look into the generated code to check the benchmark still
[info] works, and factor in a small probability of new VM bugs. Additionally, while comparisons between
[info] different JVMs are already problematic, the performance difference caused by different Blackhole
[info] modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons.
[info] Benchmark         Mode  Cnt     Score   Error  Units
[info] Bench.testMap    thrpt   10  2110.769 ± 5.416  ops/s
[info] Bench.testMatch  thrpt   10  2171.243 ± 9.581  ops/s