Skip to content

Instantly share code, notes, and snippets.

@lotabout

lotabout/MyBenchmark.java

Last active December 24, 2023 05:54
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save lotabout/430f96f0829cc586c773643a9883d1ec to your computer and use it in GitHub Desktop.
Save lotabout/430f96f0829cc586c773643a9883d1ec to your computer and use it in GitHub Desktop.
Download ZIP
CompletableFuture.supplyAsync profile
Raw
README.md

Trying to findout the impact of java.util.concurrent.ForkJoinPool.common.parallelism for short-running lambdas.

supplyAsync will submit the task to the default ForkJoinPool, whose parallelism is controled by the above property. I'm interested on the impact of increasing the parallelism.

  • Number of tasks per iteration: 200
  • Will call the method 50 times.
  • Time to complete per task: ~1ms * 50 = 500ms
  • Total time to complete if all in serial: ~200ms * 50 = 10000ms
  • Total time in batch(8 core): ~25 * 50 = 1250ms
mvn clean package
java -jar target/benchmarks.jar -i 5 -bs 50 -f 3

Results(java 1.8)

The machine has 8 cores.

Benchmark                (N)  (parallelism)  Mode  Cnt     Score     Error  Units
MyBenchmark.supplyAsync  200              1    ss   15   574.706 ±  27.327  ms/op
MyBenchmark.supplyAsync  200              2    ss   15  6591.250 ±  64.085  ms/op
MyBenchmark.supplyAsync  200              3    ss   15  4419.905 ±  40.197  ms/op
MyBenchmark.supplyAsync  200              4    ss   15  3295.410 ±  52.130  ms/op
MyBenchmark.supplyAsync  200              8    ss   15  1667.734 ±  18.292  ms/op
MyBenchmark.supplyAsync  200             10    ss   15  1294.631 ±  24.595  ms/op
MyBenchmark.supplyAsync  200             12    ss   15  1100.135 ±  24.146  ms/op
MyBenchmark.supplyAsync  200             16    ss   15   852.595 ±  22.063  ms/op
MyBenchmark.supplyAsync  200             32    ss   15   462.703 ±   9.999  ms/op
MyBenchmark.supplyAsync  200             64    ss   15   259.228 ±   6.658  ms/op
MyBenchmark.supplyAsync  200            128    ss   15   142.215 ±   6.502  ms/op
MyBenchmark.supplyAsync  200            256    ss   15   164.186 ± 153.980  ms/op

Result (Java 1.11)

Benchmark                (N)  (parallelism)  Mode  Cnt     Score     Error  Units
MyBenchmark.supplyAsync  200              1    ss   15   676.703 ±  24.553  ms/op
MyBenchmark.supplyAsync  200              2    ss   15  6610.803 ± 159.791  ms/op
MyBenchmark.supplyAsync  200              3    ss   15  4416.303 ±  64.285  ms/op
MyBenchmark.supplyAsync  200              4    ss   15  3326.308 ±  24.697  ms/op
MyBenchmark.supplyAsync  200              8    ss   15  1657.020 ±  38.348  ms/op
MyBenchmark.supplyAsync  200             10    ss   15  1335.538 ±  28.533  ms/op
MyBenchmark.supplyAsync  200             12    ss   15  1126.361 ±  16.416  ms/op
MyBenchmark.supplyAsync  200             16    ss   15   851.601 ±  45.141  ms/op
MyBenchmark.supplyAsync  200             32    ss   15   461.067 ±  19.145  ms/op
MyBenchmark.supplyAsync  200             64    ss   15   274.158 ±  25.688  ms/op
MyBenchmark.supplyAsync  200            128    ss   15   159.518 ±  41.281  ms/op
MyBenchmark.supplyAsync  200            256    ss   15   138.416 ±  54.186  ms/op
Raw
MyBenchmark.java
package me.lotabout;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;
import org.openjdk.jmh.infra.Blackhole;
@BenchmarkMode({Mode.SingleShotTime})
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
@Warmup(iterations = 5)
public class MyBenchmark {
@Param({"200"})
private int N;
@Param({"1", "2", "3", "4", "8", "10", "12", "16", "32", "64", "128", "256"})
private String parallelism;
@Setup
public void setup() {
System.setProperty("java.util.concurrent.ForkJoinPool.common.parallelism", parallelism);
}
@Benchmark
public void supplyAsync(Blackhole bh) throws ExecutionException, InterruptedException {
List<Future> futures = new ArrayList<>();
for(int i=0; i<N; i++) {
final int x = i;
futures.add(CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
bh.consume(x);
return x;
}));
}
for (Future future: futures) {
future.get();
}
}
}
Raw
pom.xml
<!--
Copyright (c) 2005, 2014, Oracle and/or its affiliates. All rights reserved.
DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
This code is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License version 2 only, as
published by the Free Software Foundation. Oracle designates this
particular file as subject to the "Classpath" exception as provided
by Oracle in the LICENSE file that accompanied this code.
This code is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
version 2 for more details (a copy is included in the LICENSE file that
accompanied this code).
You should have received a copy of the GNU General Public License version
2 along with this work; if not, write to the Free Software Foundation,
Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
or visit www.oracle.com if you need additional information or have any
questions.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>me.lotabout</groupId>
<artifactId>uuidbench</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<name>Auto-generated JMH benchmark</name>
<prerequisites>
<maven>3.0</maven>
</prerequisites>
<dependencies>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>${jmh.version}</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>${jmh.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.uuid</groupId>
<artifactId>java-uuid-generator</artifactId>
<version>3.1.3</version>
</dependency>
</dependencies>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<jmh.version>1.21</jmh.version>
<javac.target>1.8</javac.target>
<uberjar.name>benchmarks</uberjar.name>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<compilerVersion>${javac.target}</compilerVersion>
<source>${javac.target}</source>
<target>${javac.target}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<finalName>${uberjar.name}</finalName>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>org.openjdk.jmh.Main</mainClass>
</transformer>
</transformers>
<filters>
<filter>
<!--
Shading signed JARs will fail without this.
http://stackoverflow.com/questions/999489/invalid-signature-file-when-attempting-to-run-a-jar
-->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>2.5</version>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.1</version>
</plugin>
<plugin>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.1</version>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
</plugin>
<plugin>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.9.1</version>
</plugin>
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
</plugin>
<plugin>
<artifactId>maven-site-plugin</artifactId>
<version>3.3</version>
</plugin>
<plugin>
<artifactId>maven-source-plugin</artifactId>
<version>2.2.1</version>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
Raw
Result-java11.txt
$ java -jar target/benchmarks.jar -i 5 -bs 50 -f 3
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/Users/jinzhouz/repos/uuidbench/target/benchmarks.jar) to field java.io.Console.cs
WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 1)
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 3
# Warmup Iteration 1: 29.923 ms/op
# Warmup Iteration 2: 24.570 ms/op
# Warmup Iteration 3: 21.004 ms/op
# Warmup Iteration 4: 13.650 ms/op
# Warmup Iteration 5: 17.158 ms/op
Iteration 1: 731.838 ms/op
Iteration 2: 680.072 ms/op
Iteration 3: 659.652 ms/op
Iteration 4: 658.217 ms/op
Iteration 5: 662.412 ms/op
# Run progress: 2.78% complete, ETA 00:02:22
# Fork: 2 of 3
# Warmup Iteration 1: 26.488 ms/op
# Warmup Iteration 2: 24.207 ms/op
# Warmup Iteration 3: 23.010 ms/op
# Warmup Iteration 4: 18.938 ms/op
# Warmup Iteration 5: 13.829 ms/op
Iteration 1: 695.674 ms/op
Iteration 2: 664.083 ms/op
Iteration 3: 663.561 ms/op
Iteration 4: 664.050 ms/op
Iteration 5: 662.390 ms/op
# Run progress: 5.56% complete, ETA 00:02:13
# Fork: 3 of 3
# Warmup Iteration 1: 24.573 ms/op
# Warmup Iteration 2: 22.075 ms/op
# Warmup Iteration 3: 20.732 ms/op
# Warmup Iteration 4: 16.955 ms/op
# Warmup Iteration 5: 14.177 ms/op
Iteration 1: 718.864 ms/op
Iteration 2: 692.421 ms/op
Iteration 3: 668.487 ms/op
Iteration 4: 659.645 ms/op
Iteration 5: 669.178 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 676.703 ±(99.9%) 24.553 ms/op
Histogram, ms/op:
[650.000, 655.000) = 0
[655.000, 660.000) = 3
[660.000, 665.000) = 5
[665.000, 670.000) = 2
[670.000, 675.000) = 0
[675.000, 680.000) = 0
[680.000, 685.000) = 1
[685.000, 690.000) = 0
[690.000, 695.000) = 1
[695.000, 700.000) = 1
[700.000, 705.000) = 0
[705.000, 710.000) = 0
[710.000, 715.000) = 0
[715.000, 720.000) = 1
[720.000, 725.000) = 0
[725.000, 730.000) = 0
[730.000, 735.000) = 1
Percentiles, ms/op:
p(0.0000) = 658.217 ms/op
p(50.0000) = 664.083 ms/op
p(90.0000) = 724.053 ms/op
p(95.0000) = 731.838 ms/op
p(99.0000) = 731.838 ms/op
p(99.9000) = 731.838 ms/op
p(99.9900) = 731.838 ms/op
p(99.9990) = 731.838 ms/op
p(99.9999) = 731.838 ms/op
p(100.0000) = 731.838 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 2)
# Run progress: 8.33% complete, ETA 00:02:09
# Fork: 1 of 3
# Warmup Iteration 1: 140.185 ms/op
# Warmup Iteration 2: 131.170 ms/op
# Warmup Iteration 3: 129.723 ms/op
# Warmup Iteration 4: 132.189 ms/op
# Warmup Iteration 5: 132.275 ms/op
Iteration 1: 6739.984 ms/op
Iteration 2: 6706.034 ms/op
Iteration 3: 6616.194 ms/op
Iteration 4: 6706.635 ms/op
Iteration 5: 6719.528 ms/op
# Run progress: 11.11% complete, ETA 00:06:10
# Fork: 2 of 3
# Warmup Iteration 1: 138.999 ms/op
# Warmup Iteration 2: 133.522 ms/op
# Warmup Iteration 3: 132.192 ms/op
# Warmup Iteration 4: 133.117 ms/op
# Warmup Iteration 5: 132.211 ms/op
Iteration 1: 6734.151 ms/op
Iteration 2: 6697.749 ms/op
Iteration 3: 6724.852 ms/op
Iteration 4: 6705.789 ms/op
Iteration 5: 6488.152 ms/op
# Run progress: 13.89% complete, ETA 00:08:20
# Fork: 3 of 3
# Warmup Iteration 1: 129.685 ms/op
# Warmup Iteration 2: 129.845 ms/op
# Warmup Iteration 3: 127.705 ms/op
# Warmup Iteration 4: 135.089 ms/op
# Warmup Iteration 5: 132.694 ms/op
Iteration 1: 6679.997 ms/op
Iteration 2: 6257.820 ms/op
Iteration 3: 6393.081 ms/op
Iteration 4: 6534.793 ms/op
Iteration 5: 6457.281 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 6610.803 ±(99.9%) 159.791 ms/op
Histogram, ms/op:
[6200.000, 6250.000) = 0
[6250.000, 6300.000) = 1
[6300.000, 6350.000) = 0
[6350.000, 6400.000) = 1
[6400.000, 6450.000) = 0
[6450.000, 6500.000) = 2
[6500.000, 6550.000) = 1
[6550.000, 6600.000) = 0
[6600.000, 6650.000) = 1
[6650.000, 6700.000) = 2
[6700.000, 6750.000) = 7
Percentiles, ms/op:
p(0.0000) = 6257.820 ms/op
p(50.0000) = 6697.749 ms/op
p(90.0000) = 6736.484 ms/op
p(95.0000) = 6739.984 ms/op
p(99.0000) = 6739.984 ms/op
p(99.9000) = 6739.984 ms/op
p(99.9900) = 6739.984 ms/op
p(99.9990) = 6739.984 ms/op
p(99.9999) = 6739.984 ms/op
p(100.0000) = 6739.984 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 3)
# Run progress: 16.67% complete, ETA 00:09:30
# Fork: 1 of 3
# Warmup Iteration 1: 96.392 ms/op
# Warmup Iteration 2: 90.328 ms/op
# Warmup Iteration 3: 87.055 ms/op
# Warmup Iteration 4: 90.499 ms/op
# Warmup Iteration 5: 90.518 ms/op
Iteration 1: 4510.368 ms/op
Iteration 2: 4487.674 ms/op
Iteration 3: 4305.732 ms/op
Iteration 4: 4457.493 ms/op
Iteration 5: 4395.070 ms/op
# Run progress: 19.44% complete, ETA 00:09:27
# Fork: 2 of 3
# Warmup Iteration 1: 97.779 ms/op
# Warmup Iteration 2: 88.364 ms/op
# Warmup Iteration 3: 88.401 ms/op
# Warmup Iteration 4: 86.685 ms/op
# Warmup Iteration 5: 89.945 ms/op
Iteration 1: 4374.563 ms/op
Iteration 2: 4374.589 ms/op
Iteration 3: 4453.806 ms/op
Iteration 4: 4448.381 ms/op
Iteration 5: 4404.049 ms/op
# Run progress: 22.22% complete, ETA 00:09:19
# Fork: 3 of 3
# Warmup Iteration 1: 96.621 ms/op
# Warmup Iteration 2: 88.904 ms/op
# Warmup Iteration 3: 90.093 ms/op
# Warmup Iteration 4: 87.745 ms/op
# Warmup Iteration 5: 85.799 ms/op
Iteration 1: 4402.979 ms/op
Iteration 2: 4449.899 ms/op
Iteration 3: 4356.593 ms/op
Iteration 4: 4336.430 ms/op
Iteration 5: 4486.922 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 4416.303 ±(99.9%) 64.285 ms/op
Histogram, ms/op:
[4300.000, 4325.000) = 1
[4325.000, 4350.000) = 1
[4350.000, 4375.000) = 3
[4375.000, 4400.000) = 1
[4400.000, 4425.000) = 2
[4425.000, 4450.000) = 2
[4450.000, 4475.000) = 2
[4475.000, 4500.000) = 2
[4500.000, 4525.000) = 1
[4525.000, 4550.000) = 0
[4550.000, 4575.000) = 0
Percentiles, ms/op:
p(0.0000) = 4305.732 ms/op
p(50.0000) = 4404.049 ms/op
p(90.0000) = 4496.752 ms/op
p(95.0000) = 4510.368 ms/op
p(99.0000) = 4510.368 ms/op
p(99.9000) = 4510.368 ms/op
p(99.9900) = 4510.368 ms/op
p(99.9990) = 4510.368 ms/op
p(99.9999) = 4510.368 ms/op
p(100.0000) = 4510.368 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 4)
# Run progress: 25.00% complete, ETA 00:09:08
# Fork: 1 of 3
# Warmup Iteration 1: 72.350 ms/op
# Warmup Iteration 2: 62.824 ms/op
# Warmup Iteration 3: 63.102 ms/op
# Warmup Iteration 4: 63.941 ms/op
# Warmup Iteration 5: 63.037 ms/op
Iteration 1: 3348.458 ms/op
Iteration 2: 3325.259 ms/op
Iteration 3: 3319.862 ms/op
Iteration 4: 3303.616 ms/op
Iteration 5: 3271.849 ms/op
# Run progress: 27.78% complete, ETA 00:08:39
# Fork: 2 of 3
# Warmup Iteration 1: 73.144 ms/op
# Warmup Iteration 2: 66.317 ms/op
# Warmup Iteration 3: 65.374 ms/op
# Warmup Iteration 4: 64.702 ms/op
# Warmup Iteration 5: 64.853 ms/op
Iteration 1: 3303.276 ms/op
Iteration 2: 3318.492 ms/op
Iteration 3: 3336.696 ms/op
Iteration 4: 3301.847 ms/op
Iteration 5: 3340.352 ms/op
# Run progress: 30.56% complete, ETA 00:08:13
# Fork: 3 of 3
# Warmup Iteration 1: 71.980 ms/op
# Warmup Iteration 2: 63.466 ms/op
# Warmup Iteration 3: 62.033 ms/op
# Warmup Iteration 4: 64.445 ms/op
# Warmup Iteration 5: 61.765 ms/op
Iteration 1: 3351.243 ms/op
Iteration 2: 3339.226 ms/op
Iteration 3: 3337.273 ms/op
Iteration 4: 3343.052 ms/op
Iteration 5: 3354.113 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 3326.308 ±(99.9%) 24.697 ms/op
Histogram, ms/op:
[3270.000, 3275.000) = 1
[3275.000, 3280.000) = 0
[3280.000, 3285.000) = 0
[3285.000, 3290.000) = 0
[3290.000, 3295.000) = 0
[3295.000, 3300.000) = 0
[3300.000, 3305.000) = 3
[3305.000, 3310.000) = 0
[3310.000, 3315.000) = 0
[3315.000, 3320.000) = 2
[3320.000, 3325.000) = 0
[3325.000, 3330.000) = 1
[3330.000, 3335.000) = 0
[3335.000, 3340.000) = 3
[3340.000, 3345.000) = 2
[3345.000, 3350.000) = 1
[3350.000, 3355.000) = 2
Percentiles, ms/op:
p(0.0000) = 3271.849 ms/op
p(50.0000) = 3336.696 ms/op
p(90.0000) = 3352.391 ms/op
p(95.0000) = 3354.113 ms/op
p(99.0000) = 3354.113 ms/op
p(99.9000) = 3354.113 ms/op
p(99.9900) = 3354.113 ms/op
p(99.9990) = 3354.113 ms/op
p(99.9999) = 3354.113 ms/op
p(100.0000) = 3354.113 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 8)
# Run progress: 33.33% complete, ETA 00:07:49
# Fork: 1 of 3
# Warmup Iteration 1: 40.113 ms/op
# Warmup Iteration 2: 33.929 ms/op
# Warmup Iteration 3: 32.826 ms/op
# Warmup Iteration 4: 32.359 ms/op
# Warmup Iteration 5: 32.234 ms/op
Iteration 1: 1692.496 ms/op
Iteration 2: 1677.722 ms/op
Iteration 3: 1700.319 ms/op
Iteration 4: 1664.503 ms/op
Iteration 5: 1671.851 ms/op
# Run progress: 36.11% complete, ETA 00:07:11
# Fork: 2 of 3
# Warmup Iteration 1: 36.425 ms/op
# Warmup Iteration 2: 31.728 ms/op
# Warmup Iteration 3: 33.225 ms/op
# Warmup Iteration 4: 29.931 ms/op
# Warmup Iteration 5: 32.371 ms/op
Iteration 1: 1694.388 ms/op
Iteration 2: 1636.490 ms/op
Iteration 3: 1572.411 ms/op
Iteration 4: 1658.613 ms/op
Iteration 5: 1660.353 ms/op
# Run progress: 38.89% complete, ETA 00:06:36
# Fork: 3 of 3
# Warmup Iteration 1: 37.315 ms/op
# Warmup Iteration 2: 30.683 ms/op
# Warmup Iteration 3: 31.973 ms/op
# Warmup Iteration 4: 34.185 ms/op
# Warmup Iteration 5: 31.195 ms/op
Iteration 1: 1682.401 ms/op
Iteration 2: 1657.127 ms/op
Iteration 3: 1595.339 ms/op
Iteration 4: 1630.184 ms/op
Iteration 5: 1661.101 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 1657.020 ±(99.9%) 38.348 ms/op
Histogram, ms/op:
[1500.000, 1525.000) = 0
[1525.000, 1550.000) = 0
[1550.000, 1575.000) = 1
[1575.000, 1600.000) = 1
[1600.000, 1625.000) = 0
[1625.000, 1650.000) = 2
[1650.000, 1675.000) = 6
[1675.000, 1700.000) = 4
[1700.000, 1725.000) = 1
[1725.000, 1750.000) = 0
[1750.000, 1775.000) = 0
Percentiles, ms/op:
p(0.0000) = 1572.411 ms/op
p(50.0000) = 1661.101 ms/op
p(90.0000) = 1696.761 ms/op
p(95.0000) = 1700.319 ms/op
p(99.0000) = 1700.319 ms/op
p(99.9000) = 1700.319 ms/op
p(99.9900) = 1700.319 ms/op
p(99.9990) = 1700.319 ms/op
p(99.9999) = 1700.319 ms/op
p(100.0000) = 1700.319 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 10)
# Run progress: 41.67% complete, ETA 00:06:05
# Fork: 1 of 3
# Warmup Iteration 1: 35.260 ms/op
# Warmup Iteration 2: 27.064 ms/op
# Warmup Iteration 3: 27.008 ms/op
# Warmup Iteration 4: 27.691 ms/op
# Warmup Iteration 5: 27.176 ms/op
Iteration 1: 1271.572 ms/op
Iteration 2: 1302.200 ms/op
Iteration 3: 1345.782 ms/op
Iteration 4: 1347.895 ms/op
Iteration 5: 1303.460 ms/op
# Run progress: 44.44% complete, ETA 00:05:35
# Fork: 2 of 3
# Warmup Iteration 1: 33.595 ms/op
# Warmup Iteration 2: 25.226 ms/op
# Warmup Iteration 3: 26.255 ms/op
# Warmup Iteration 4: 25.396 ms/op
# Warmup Iteration 5: 22.544 ms/op
Iteration 1: 1308.699 ms/op
Iteration 2: 1352.309 ms/op
Iteration 3: 1347.290 ms/op
Iteration 4: 1358.057 ms/op
Iteration 5: 1359.643 ms/op
# Run progress: 47.22% complete, ETA 00:05:07
# Fork: 3 of 3
# Warmup Iteration 1: 35.132 ms/op
# Warmup Iteration 2: 27.589 ms/op
# Warmup Iteration 3: 27.988 ms/op
# Warmup Iteration 4: 24.745 ms/op
# Warmup Iteration 5: 26.302 ms/op
Iteration 1: 1357.241 ms/op
Iteration 2: 1329.525 ms/op
Iteration 3: 1352.079 ms/op
Iteration 4: 1356.731 ms/op
Iteration 5: 1340.580 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 1335.538 ±(99.9%) 28.533 ms/op
Histogram, ms/op:
[1270.000, 1275.000) = 1
[1275.000, 1280.000) = 0
[1280.000, 1285.000) = 0
[1285.000, 1290.000) = 0
[1290.000, 1295.000) = 0
[1295.000, 1300.000) = 0
[1300.000, 1305.000) = 2
[1305.000, 1310.000) = 1
[1310.000, 1315.000) = 0
[1315.000, 1320.000) = 0
[1320.000, 1325.000) = 0
[1325.000, 1330.000) = 1
[1330.000, 1335.000) = 0
[1335.000, 1340.000) = 0
[1340.000, 1345.000) = 1
[1345.000, 1350.000) = 3
[1350.000, 1355.000) = 2
Percentiles, ms/op:
p(0.0000) = 1271.572 ms/op
p(50.0000) = 1347.290 ms/op
p(90.0000) = 1358.691 ms/op
p(95.0000) = 1359.643 ms/op
p(99.0000) = 1359.643 ms/op
p(99.9000) = 1359.643 ms/op
p(99.9900) = 1359.643 ms/op
p(99.9990) = 1359.643 ms/op
p(99.9999) = 1359.643 ms/op
p(100.0000) = 1359.643 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 12)
# Run progress: 50.00% complete, ETA 00:04:42
# Fork: 1 of 3
# Warmup Iteration 1: 29.426 ms/op
# Warmup Iteration 2: 21.218 ms/op
# Warmup Iteration 3: 21.978 ms/op
# Warmup Iteration 4: 21.656 ms/op
# Warmup Iteration 5: 21.506 ms/op
Iteration 1: 1105.011 ms/op
Iteration 2: 1149.315 ms/op
Iteration 3: 1126.171 ms/op
Iteration 4: 1112.617 ms/op
Iteration 5: 1102.000 ms/op
# Run progress: 52.78% complete, ETA 00:04:18
# Fork: 2 of 3
# Warmup Iteration 1: 26.584 ms/op
# Warmup Iteration 2: 21.882 ms/op
# Warmup Iteration 3: 21.735 ms/op
# Warmup Iteration 4: 23.136 ms/op
# Warmup Iteration 5: 22.626 ms/op
Iteration 1: 1125.969 ms/op
Iteration 2: 1150.653 ms/op
Iteration 3: 1126.801 ms/op
Iteration 4: 1137.289 ms/op
Iteration 5: 1103.224 ms/op
# Run progress: 55.56% complete, ETA 00:03:55
# Fork: 3 of 3
# Warmup Iteration 1: 28.159 ms/op
# Warmup Iteration 2: 22.536 ms/op
# Warmup Iteration 3: 22.718 ms/op
# Warmup Iteration 4: 22.936 ms/op
# Warmup Iteration 5: 19.405 ms/op
Iteration 1: 1132.547 ms/op
Iteration 2: 1120.220 ms/op
Iteration 3: 1137.624 ms/op
Iteration 4: 1133.069 ms/op
Iteration 5: 1132.901 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 1126.361 ±(99.9%) 16.416 ms/op
Histogram, ms/op:
[1100.000, 1105.000) = 2
[1105.000, 1110.000) = 1
[1110.000, 1115.000) = 1
[1115.000, 1120.000) = 0
[1120.000, 1125.000) = 1
[1125.000, 1130.000) = 3
[1130.000, 1135.000) = 3
[1135.000, 1140.000) = 2
[1140.000, 1145.000) = 0
[1145.000, 1150.000) = 1
[1150.000, 1155.000) = 1
Percentiles, ms/op:
p(0.0000) = 1102.000 ms/op
p(50.0000) = 1126.801 ms/op
p(90.0000) = 1149.850 ms/op
p(95.0000) = 1150.653 ms/op
p(99.0000) = 1150.653 ms/op
p(99.9000) = 1150.653 ms/op
p(99.9900) = 1150.653 ms/op
p(99.9990) = 1150.653 ms/op
p(99.9999) = 1150.653 ms/op
p(100.0000) = 1150.653 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 16)
# Run progress: 58.33% complete, ETA 00:03:34
# Fork: 1 of 3
# Warmup Iteration 1: 24.187 ms/op
# Warmup Iteration 2: 16.561 ms/op
# Warmup Iteration 3: 17.430 ms/op
# Warmup Iteration 4: 16.492 ms/op
# Warmup Iteration 5: 18.310 ms/op
Iteration 1: 872.771 ms/op
Iteration 2: 860.155 ms/op
Iteration 3: 884.153 ms/op
Iteration 4: 871.829 ms/op
Iteration 5: 880.792 ms/op
# Run progress: 61.11% complete, ETA 00:03:14
# Fork: 2 of 3
# Warmup Iteration 1: 22.097 ms/op
# Warmup Iteration 2: 16.872 ms/op
# Warmup Iteration 3: 17.362 ms/op
# Warmup Iteration 4: 16.869 ms/op
# Warmup Iteration 5: 18.179 ms/op
Iteration 1: 867.381 ms/op
Iteration 2: 856.489 ms/op
Iteration 3: 912.900 ms/op
Iteration 4: 849.730 ms/op
Iteration 5: 769.642 ms/op
# Run progress: 63.89% complete, ETA 00:02:55
# Fork: 3 of 3
# Warmup Iteration 1: 21.131 ms/op
# Warmup Iteration 2: 15.968 ms/op
# Warmup Iteration 3: 15.639 ms/op
# Warmup Iteration 4: 15.400 ms/op
# Warmup Iteration 5: 15.390 ms/op
Iteration 1: 776.816 ms/op
Iteration 2: 780.554 ms/op
Iteration 3: 877.399 ms/op
Iteration 4: 858.929 ms/op
Iteration 5: 854.469 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 851.601 ±(99.9%) 45.141 ms/op
Histogram, ms/op:
[ 700.000, 725.000) = 0
[ 725.000, 750.000) = 0
[ 750.000, 775.000) = 1
[ 775.000, 800.000) = 2
[ 800.000, 825.000) = 0
[ 825.000, 850.000) = 1
[ 850.000, 875.000) = 7
[ 875.000, 900.000) = 3
[ 900.000, 925.000) = 1
[ 925.000, 950.000) = 0
[ 950.000, 975.000) = 0
Percentiles, ms/op:
p(0.0000) = 769.642 ms/op
p(50.0000) = 860.155 ms/op
p(90.0000) = 895.652 ms/op
p(95.0000) = 912.900 ms/op
p(99.0000) = 912.900 ms/op
p(99.9000) = 912.900 ms/op
p(99.9900) = 912.900 ms/op
p(99.9990) = 912.900 ms/op
p(99.9999) = 912.900 ms/op
p(100.0000) = 912.900 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 32)
# Run progress: 66.67% complete, ETA 00:02:37
# Fork: 1 of 3
# Warmup Iteration 1: 15.092 ms/op
# Warmup Iteration 2: 8.714 ms/op
# Warmup Iteration 3: 8.804 ms/op
# Warmup Iteration 4: 8.504 ms/op
# Warmup Iteration 5: 9.861 ms/op
Iteration 1: 476.940 ms/op
Iteration 2: 472.434 ms/op
Iteration 3: 429.472 ms/op
Iteration 4: 432.433 ms/op
Iteration 5: 461.785 ms/op
# Run progress: 69.44% complete, ETA 00:02:19
# Fork: 2 of 3
# Warmup Iteration 1: 14.309 ms/op
# Warmup Iteration 2: 9.226 ms/op
# Warmup Iteration 3: 9.591 ms/op
# Warmup Iteration 4: 9.424 ms/op
# Warmup Iteration 5: 9.991 ms/op
Iteration 1: 457.461 ms/op
Iteration 2: 473.074 ms/op
Iteration 3: 459.145 ms/op
Iteration 4: 454.615 ms/op
Iteration 5: 455.805 ms/op
# Run progress: 72.22% complete, ETA 00:02:03
# Fork: 3 of 3
# Warmup Iteration 1: 14.482 ms/op
# Warmup Iteration 2: 8.895 ms/op
# Warmup Iteration 3: 9.418 ms/op
# Warmup Iteration 4: 8.785 ms/op
# Warmup Iteration 5: 9.847 ms/op
Iteration 1: 498.462 ms/op
Iteration 2: 475.958 ms/op
Iteration 3: 447.293 ms/op
Iteration 4: 471.733 ms/op
Iteration 5: 449.388 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 461.067 ±(99.9%) 19.145 ms/op
Histogram, ms/op:
[420.000, 425.000) = 0
[425.000, 430.000) = 1
[430.000, 435.000) = 1
[435.000, 440.000) = 0
[440.000, 445.000) = 0
[445.000, 450.000) = 2
[450.000, 455.000) = 1
[455.000, 460.000) = 3
[460.000, 465.000) = 1
[465.000, 470.000) = 0
[470.000, 475.000) = 3
[475.000, 480.000) = 2
[480.000, 485.000) = 0
[485.000, 490.000) = 0
[490.000, 495.000) = 0
Percentiles, ms/op:
p(0.0000) = 429.472 ms/op
p(50.0000) = 459.145 ms/op
p(90.0000) = 485.549 ms/op
p(95.0000) = 498.462 ms/op
p(99.0000) = 498.462 ms/op
p(99.9000) = 498.462 ms/op
p(99.9900) = 498.462 ms/op
p(99.9990) = 498.462 ms/op
p(99.9999) = 498.462 ms/op
p(100.0000) = 498.462 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 64)
# Run progress: 75.00% complete, ETA 00:01:47
# Fork: 1 of 3
# Warmup Iteration 1: 16.597 ms/op
# Warmup Iteration 2: 5.078 ms/op
# Warmup Iteration 3: 4.992 ms/op
# Warmup Iteration 4: 5.047 ms/op
# Warmup Iteration 5: 5.109 ms/op
Iteration 1: 276.051 ms/op
Iteration 2: 279.235 ms/op
Iteration 3: 275.952 ms/op
Iteration 4: 278.037 ms/op
Iteration 5: 268.921 ms/op
# Run progress: 77.78% complete, ETA 00:01:32
# Fork: 2 of 3
# Warmup Iteration 1: 16.784 ms/op
# Warmup Iteration 2: 4.882 ms/op
# Warmup Iteration 3: 5.183 ms/op
# Warmup Iteration 4: 4.975 ms/op
# Warmup Iteration 5: 5.359 ms/op
Iteration 1: 355.609 ms/op
Iteration 2: 253.921 ms/op
Iteration 3: 261.445 ms/op
Iteration 4: 274.004 ms/op
Iteration 5: 253.813 ms/op
# Run progress: 80.56% complete, ETA 00:01:18
# Fork: 3 of 3
# Warmup Iteration 1: 16.849 ms/op
# Warmup Iteration 2: 5.071 ms/op
# Warmup Iteration 3: 5.130 ms/op
# Warmup Iteration 4: 5.348 ms/op
# Warmup Iteration 5: 5.378 ms/op
Iteration 1: 261.712 ms/op
Iteration 2: 275.036 ms/op
Iteration 3: 264.588 ms/op
Iteration 4: 271.490 ms/op
Iteration 5: 262.552 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 274.158 ±(99.9%) 25.688 ms/op
Histogram, ms/op:
[200.000, 212.500) = 0
[212.500, 225.000) = 0
[225.000, 237.500) = 0
[237.500, 250.000) = 0
[250.000, 262.500) = 4
[262.500, 275.000) = 5
[275.000, 287.500) = 5
[287.500, 300.000) = 0
[300.000, 312.500) = 0
[312.500, 325.000) = 0
[325.000, 337.500) = 0
[337.500, 350.000) = 0
[350.000, 362.500) = 1
[362.500, 375.000) = 0
[375.000, 387.500) = 0
Percentiles, ms/op:
p(0.0000) = 253.813 ms/op
p(50.0000) = 271.490 ms/op
p(90.0000) = 309.785 ms/op
p(95.0000) = 355.609 ms/op
p(99.0000) = 355.609 ms/op
p(99.9000) = 355.609 ms/op
p(99.9900) = 355.609 ms/op
p(99.9990) = 355.609 ms/op
p(99.9999) = 355.609 ms/op
p(100.0000) = 355.609 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 128)
# Run progress: 83.33% complete, ETA 00:01:05
# Fork: 1 of 3
# Warmup Iteration 1: 17.659 ms/op
# Warmup Iteration 2: 4.869 ms/op
# Warmup Iteration 3: 3.969 ms/op
# Warmup Iteration 4: 4.677 ms/op
# Warmup Iteration 5: 4.300 ms/op
Iteration 1: 152.584 ms/op
Iteration 2: 145.654 ms/op
Iteration 3: 140.333 ms/op
Iteration 4: 143.816 ms/op
Iteration 5: 142.277 ms/op
# Run progress: 86.11% complete, ETA 00:00:53
# Fork: 2 of 3
# Warmup Iteration 1: 19.459 ms/op
# Warmup Iteration 2: 4.658 ms/op
# Warmup Iteration 3: 3.913 ms/op
# Warmup Iteration 4: 3.916 ms/op
# Warmup Iteration 5: 3.715 ms/op
Iteration 1: 284.569 ms/op
Iteration 2: 157.118 ms/op
Iteration 3: 145.948 ms/op
Iteration 4: 148.622 ms/op
Iteration 5: 143.351 ms/op
# Run progress: 88.89% complete, ETA 00:00:41
# Fork: 3 of 3
# Warmup Iteration 1: 16.439 ms/op
# Warmup Iteration 2: 4.491 ms/op
# Warmup Iteration 3: 3.931 ms/op
# Warmup Iteration 4: 4.788 ms/op
# Warmup Iteration 5: 4.264 ms/op
Iteration 1: 210.336 ms/op
Iteration 2: 143.842 ms/op
Iteration 3: 147.132 ms/op
Iteration 4: 147.189 ms/op
Iteration 5: 139.997 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 159.518 ±(99.9%) 41.281 ms/op
Histogram, ms/op:
[100.000, 112.500) = 0
[112.500, 125.000) = 0
[125.000, 137.500) = 0
[137.500, 150.000) = 11
[150.000, 162.500) = 2
[162.500, 175.000) = 0
[175.000, 187.500) = 0
[187.500, 200.000) = 0
[200.000, 212.500) = 1
[212.500, 225.000) = 0
[225.000, 237.500) = 0
[237.500, 250.000) = 0
[250.000, 262.500) = 0
[262.500, 275.000) = 0
[275.000, 287.500) = 1
Percentiles, ms/op:
p(0.0000) = 139.997 ms/op
p(50.0000) = 145.948 ms/op
p(90.0000) = 240.029 ms/op
p(95.0000) = 284.569 ms/op
p(99.0000) = 284.569 ms/op
p(99.9000) = 284.569 ms/op
p(99.9900) = 284.569 ms/op
p(99.9990) = 284.569 ms/op
p(99.9999) = 284.569 ms/op
p(100.0000) = 284.569 ms/op
# JMH version: 1.21
# VM version: JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9
# VM invoker: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 256)
# Run progress: 91.67% complete, ETA 00:00:30
# Fork: 1 of 3
# Warmup Iteration 1: 16.248 ms/op
# Warmup Iteration 2: 5.348 ms/op
# Warmup Iteration 3: 5.050 ms/op
# Warmup Iteration 4: 5.791 ms/op
# Warmup Iteration 5: 6.802 ms/op
Iteration 1: 131.076 ms/op
Iteration 2: 120.617 ms/op
Iteration 3: 112.147 ms/op
Iteration 4: 199.847 ms/op
Iteration 5: 243.348 ms/op
# Run progress: 94.44% complete, ETA 00:00:19
# Fork: 2 of 3
# Warmup Iteration 1: 16.366 ms/op
# Warmup Iteration 2: 5.215 ms/op
# Warmup Iteration 3: 5.191 ms/op
# Warmup Iteration 4: 4.723 ms/op
# Warmup Iteration 5: 4.738 ms/op
Iteration 1: 140.324 ms/op
Iteration 2: 115.986 ms/op
Iteration 3: 106.176 ms/op
Iteration 4: 103.859 ms/op
Iteration 5: 109.237 ms/op
# Run progress: 97.22% complete, ETA 00:00:09
# Fork: 3 of 3
# Warmup Iteration 1: 17.327 ms/op
# Warmup Iteration 2: 5.403 ms/op
# Warmup Iteration 3: 4.764 ms/op
# Warmup Iteration 4: 4.919 ms/op
# Warmup Iteration 5: 4.722 ms/op
Iteration 1: 253.502 ms/op
Iteration 2: 116.151 ms/op
Iteration 3: 107.322 ms/op
Iteration 4: 105.937 ms/op
Iteration 5: 110.709 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 138.416 ±(99.9%) 54.186 ms/op
Histogram, ms/op:
[100.000, 112.500) = 7
[112.500, 125.000) = 3
[125.000, 137.500) = 1
[137.500, 150.000) = 1
[150.000, 162.500) = 0
[162.500, 175.000) = 0
[175.000, 187.500) = 0
[187.500, 200.000) = 1
[200.000, 212.500) = 0
[212.500, 225.000) = 0
[225.000, 237.500) = 0
[237.500, 250.000) = 1
[250.000, 262.500) = 1
[262.500, 275.000) = 0
[275.000, 287.500) = 0
Percentiles, ms/op:
p(0.0000) = 103.859 ms/op
p(50.0000) = 115.986 ms/op
p(90.0000) = 247.410 ms/op
p(95.0000) = 253.502 ms/op
p(99.0000) = 253.502 ms/op
p(99.9000) = 253.502 ms/op
p(99.9900) = 253.502 ms/op
p(99.9990) = 253.502 ms/op
p(99.9999) = 253.502 ms/op
p(100.0000) = 253.502 ms/op
# Run complete. Total time: 00:05:35
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark (N) (parallelism) Mode Cnt Score Error Units
MyBenchmark.supplyAsync 200 1 ss 15 676.703 ± 24.553 ms/op
MyBenchmark.supplyAsync 200 2 ss 15 6610.803 ± 159.791 ms/op
MyBenchmark.supplyAsync 200 3 ss 15 4416.303 ± 64.285 ms/op
MyBenchmark.supplyAsync 200 4 ss 15 3326.308 ± 24.697 ms/op
MyBenchmark.supplyAsync 200 8 ss 15 1657.020 ± 38.348 ms/op
MyBenchmark.supplyAsync 200 10 ss 15 1335.538 ± 28.533 ms/op
MyBenchmark.supplyAsync 200 12 ss 15 1126.361 ± 16.416 ms/op
MyBenchmark.supplyAsync 200 16 ss 15 851.601 ± 45.141 ms/op
MyBenchmark.supplyAsync 200 32 ss 15 461.067 ± 19.145 ms/op
MyBenchmark.supplyAsync 200 64 ss 15 274.158 ± 25.688 ms/op
MyBenchmark.supplyAsync 200 128 ss 15 159.518 ± 41.281 ms/op
MyBenchmark.supplyAsync 200 256 ss 15 138.416 ± 54.186 ms/op
Raw
Result-java8.txt
$ java -jar target/benchmarks.jar -i 5 -bs 50 -f 3
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 1)
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 3
# Warmup Iteration 1: 24.993 ms/op
# Warmup Iteration 2: 18.187 ms/op
# Warmup Iteration 3: 19.156 ms/op
# Warmup Iteration 4: 16.652 ms/op
# Warmup Iteration 5: 13.651 ms/op
Iteration 1: 613.888 ms/op
Iteration 2: 593.104 ms/op
Iteration 3: 563.376 ms/op
Iteration 4: 551.973 ms/op
Iteration 5: 559.447 ms/op
# Run progress: 2.78% complete, ETA 00:02:03
# Fork: 2 of 3
# Warmup Iteration 1: 17.172 ms/op
# Warmup Iteration 2: 14.327 ms/op
# Warmup Iteration 3: 20.422 ms/op
# Warmup Iteration 4: 17.328 ms/op
# Warmup Iteration 5: 14.864 ms/op
Iteration 1: 642.294 ms/op
Iteration 2: 568.375 ms/op
Iteration 3: 552.501 ms/op
Iteration 4: 560.744 ms/op
Iteration 5: 566.118 ms/op
# Run progress: 5.56% complete, ETA 00:01:56
# Fork: 3 of 3
# Warmup Iteration 1: 19.366 ms/op
# Warmup Iteration 2: 19.298 ms/op
# Warmup Iteration 3: 13.464 ms/op
# Warmup Iteration 4: 13.824 ms/op
# Warmup Iteration 5: 10.565 ms/op
Iteration 1: 569.975 ms/op
Iteration 2: 558.004 ms/op
Iteration 3: 562.753 ms/op
Iteration 4: 562.511 ms/op
Iteration 5: 595.522 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 574.706 ±(99.9%) 27.327 ms/op
Histogram, ms/op:
[550.000, 560.000) = 4
[560.000, 570.000) = 7
[570.000, 580.000) = 0
[580.000, 590.000) = 0
[590.000, 600.000) = 2
[600.000, 610.000) = 0
[610.000, 620.000) = 1
[620.000, 630.000) = 0
[630.000, 640.000) = 0
Percentiles, ms/op:
p(0.0000) = 551.973 ms/op
p(50.0000) = 563.376 ms/op
p(90.0000) = 625.250 ms/op
p(95.0000) = 642.294 ms/op
p(99.0000) = 642.294 ms/op
p(99.9000) = 642.294 ms/op
p(99.9900) = 642.294 ms/op
p(99.9990) = 642.294 ms/op
p(99.9999) = 642.294 ms/op
p(100.0000) = 642.294 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 2)
# Run progress: 8.33% complete, ETA 00:01:50
# Fork: 1 of 3
# Warmup Iteration 1: 136.723 ms/op
# Warmup Iteration 2: 131.341 ms/op
# Warmup Iteration 3: 135.217 ms/op
# Warmup Iteration 4: 137.483 ms/op
# Warmup Iteration 5: 136.989 ms/op
Iteration 1: 6632.859 ms/op
Iteration 2: 6573.844 ms/op
Iteration 3: 6665.561 ms/op
Iteration 4: 6637.798 ms/op
Iteration 5: 6564.587 ms/op
# Run progress: 11.11% complete, ETA 00:05:53
# Fork: 2 of 3
# Warmup Iteration 1: 127.625 ms/op
# Warmup Iteration 2: 129.688 ms/op
# Warmup Iteration 3: 131.999 ms/op
# Warmup Iteration 4: 130.320 ms/op
# Warmup Iteration 5: 131.391 ms/op
Iteration 1: 6548.040 ms/op
Iteration 2: 6600.412 ms/op
Iteration 3: 6616.504 ms/op
Iteration 4: 6630.621 ms/op
Iteration 5: 6531.424 ms/op
# Run progress: 13.89% complete, ETA 00:08:03
# Fork: 3 of 3
# Warmup Iteration 1: 142.667 ms/op
# Warmup Iteration 2: 134.818 ms/op
# Warmup Iteration 3: 124.311 ms/op
# Warmup Iteration 4: 123.249 ms/op
# Warmup Iteration 5: 128.483 ms/op
Iteration 1: 6646.902 ms/op
Iteration 2: 6547.580 ms/op
Iteration 3: 6643.116 ms/op
Iteration 4: 6594.620 ms/op
Iteration 5: 6434.884 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 6591.250 ±(99.9%) 64.085 ms/op
Histogram, ms/op:
[6400.000, 6425.000) = 0
[6425.000, 6450.000) = 1
[6450.000, 6475.000) = 0
[6475.000, 6500.000) = 0
[6500.000, 6525.000) = 0
[6525.000, 6550.000) = 3
[6550.000, 6575.000) = 2
[6575.000, 6600.000) = 1
[6600.000, 6625.000) = 2
[6625.000, 6650.000) = 5
[6650.000, 6675.000) = 1
Percentiles, ms/op:
p(0.0000) = 6434.884 ms/op
p(50.0000) = 6600.412 ms/op
p(90.0000) = 6654.365 ms/op
p(95.0000) = 6665.561 ms/op
p(99.0000) = 6665.561 ms/op
p(99.9000) = 6665.561 ms/op
p(99.9900) = 6665.561 ms/op
p(99.9990) = 6665.561 ms/op
p(99.9999) = 6665.561 ms/op
p(100.0000) = 6665.561 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 3)
# Run progress: 16.67% complete, ETA 00:09:19
# Fork: 1 of 3
# Warmup Iteration 1: 95.524 ms/op
# Warmup Iteration 2: 85.803 ms/op
# Warmup Iteration 3: 91.085 ms/op
# Warmup Iteration 4: 86.538 ms/op
# Warmup Iteration 5: 89.448 ms/op
Iteration 1: 4435.074 ms/op
Iteration 2: 4474.462 ms/op
Iteration 3: 4434.499 ms/op
Iteration 4: 4418.624 ms/op
Iteration 5: 4413.202 ms/op
# Run progress: 19.44% complete, ETA 00:09:18
# Fork: 2 of 3
# Warmup Iteration 1: 92.388 ms/op
# Warmup Iteration 2: 88.867 ms/op
# Warmup Iteration 3: 86.573 ms/op
# Warmup Iteration 4: 87.251 ms/op
# Warmup Iteration 5: 90.873 ms/op
Iteration 1: 4458.517 ms/op
Iteration 2: 4443.958 ms/op
Iteration 3: 4354.801 ms/op
Iteration 4: 4436.101 ms/op
Iteration 5: 4449.096 ms/op
# Run progress: 22.22% complete, ETA 00:09:11
# Fork: 3 of 3
# Warmup Iteration 1: 95.621 ms/op
# Warmup Iteration 2: 90.589 ms/op
# Warmup Iteration 3: 89.000 ms/op
# Warmup Iteration 4: 89.518 ms/op
# Warmup Iteration 5: 84.106 ms/op
Iteration 1: 4433.808 ms/op
Iteration 2: 4389.230 ms/op
Iteration 3: 4435.038 ms/op
Iteration 4: 4378.717 ms/op
Iteration 5: 4343.451 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 4419.905 ±(99.9%) 40.197 ms/op
Histogram, ms/op:
[4300.000, 4312.500) = 0
[4312.500, 4325.000) = 0
[4325.000, 4337.500) = 0
[4337.500, 4350.000) = 1
[4350.000, 4362.500) = 1
[4362.500, 4375.000) = 0
[4375.000, 4387.500) = 1
[4387.500, 4400.000) = 1
[4400.000, 4412.500) = 0
[4412.500, 4425.000) = 2
[4425.000, 4437.500) = 5
[4437.500, 4450.000) = 2
[4450.000, 4462.500) = 1
[4462.500, 4475.000) = 1
[4475.000, 4487.500) = 0
Percentiles, ms/op:
p(0.0000) = 4343.451 ms/op
p(50.0000) = 4434.499 ms/op
p(90.0000) = 4464.895 ms/op
p(95.0000) = 4474.462 ms/op
p(99.0000) = 4474.462 ms/op
p(99.9000) = 4474.462 ms/op
p(99.9900) = 4474.462 ms/op
p(99.9990) = 4474.462 ms/op
p(99.9999) = 4474.462 ms/op
p(100.0000) = 4474.462 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 4)
# Run progress: 25.00% complete, ETA 00:09:01
# Fork: 1 of 3
# Warmup Iteration 1: 69.395 ms/op
# Warmup Iteration 2: 62.095 ms/op
# Warmup Iteration 3: 60.687 ms/op
# Warmup Iteration 4: 65.120 ms/op
# Warmup Iteration 5: 68.289 ms/op
Iteration 1: 3341.236 ms/op
Iteration 2: 3279.892 ms/op
Iteration 3: 3175.540 ms/op
Iteration 4: 3301.005 ms/op
Iteration 5: 3303.191 ms/op
# Run progress: 27.78% complete, ETA 00:08:33
# Fork: 2 of 3
# Warmup Iteration 1: 64.849 ms/op
# Warmup Iteration 2: 64.108 ms/op
# Warmup Iteration 3: 62.841 ms/op
# Warmup Iteration 4: 62.275 ms/op
# Warmup Iteration 5: 65.069 ms/op
Iteration 1: 3306.755 ms/op
Iteration 2: 3296.057 ms/op
Iteration 3: 3283.288 ms/op
Iteration 4: 3223.833 ms/op
Iteration 5: 3245.043 ms/op
# Run progress: 30.56% complete, ETA 00:08:07
# Fork: 3 of 3
# Warmup Iteration 1: 73.416 ms/op
# Warmup Iteration 2: 69.708 ms/op
# Warmup Iteration 3: 65.783 ms/op
# Warmup Iteration 4: 63.549 ms/op
# Warmup Iteration 5: 64.554 ms/op
Iteration 1: 3331.443 ms/op
Iteration 2: 3313.832 ms/op
Iteration 3: 3342.998 ms/op
Iteration 4: 3340.371 ms/op
Iteration 5: 3346.670 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 3295.410 ±(99.9%) 52.130 ms/op
Histogram, ms/op:
[3100.000, 3125.000) = 0
[3125.000, 3150.000) = 0
[3150.000, 3175.000) = 0
[3175.000, 3200.000) = 1
[3200.000, 3225.000) = 1
[3225.000, 3250.000) = 1
[3250.000, 3275.000) = 0
[3275.000, 3300.000) = 3
[3300.000, 3325.000) = 4
[3325.000, 3350.000) = 5
[3350.000, 3375.000) = 0
Percentiles, ms/op:
p(0.0000) = 3175.540 ms/op
p(50.0000) = 3303.191 ms/op
p(90.0000) = 3344.467 ms/op
p(95.0000) = 3346.670 ms/op
p(99.0000) = 3346.670 ms/op
p(99.9000) = 3346.670 ms/op
p(99.9900) = 3346.670 ms/op
p(99.9990) = 3346.670 ms/op
p(99.9999) = 3346.670 ms/op
p(100.0000) = 3346.670 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 8)
# Run progress: 33.33% complete, ETA 00:07:43
# Fork: 1 of 3
# Warmup Iteration 1: 37.225 ms/op
# Warmup Iteration 2: 34.292 ms/op
# Warmup Iteration 3: 35.265 ms/op
# Warmup Iteration 4: 33.148 ms/op
# Warmup Iteration 5: 34.229 ms/op
Iteration 1: 1674.614 ms/op
Iteration 2: 1688.233 ms/op
Iteration 3: 1687.546 ms/op
Iteration 4: 1673.540 ms/op
Iteration 5: 1652.633 ms/op
# Run progress: 36.11% complete, ETA 00:07:05
# Fork: 2 of 3
# Warmup Iteration 1: 38.889 ms/op
# Warmup Iteration 2: 31.116 ms/op
# Warmup Iteration 3: 32.601 ms/op
# Warmup Iteration 4: 34.354 ms/op
# Warmup Iteration 5: 33.143 ms/op
Iteration 1: 1650.746 ms/op
Iteration 2: 1677.614 ms/op
Iteration 3: 1659.499 ms/op
Iteration 4: 1656.991 ms/op
Iteration 5: 1667.468 ms/op
# Run progress: 38.89% complete, ETA 00:06:31
# Fork: 3 of 3
# Warmup Iteration 1: 37.528 ms/op
# Warmup Iteration 2: 34.162 ms/op
# Warmup Iteration 3: 33.204 ms/op
# Warmup Iteration 4: 31.836 ms/op
# Warmup Iteration 5: 33.630 ms/op
Iteration 1: 1691.690 ms/op
Iteration 2: 1673.602 ms/op
Iteration 3: 1647.641 ms/op
Iteration 4: 1632.920 ms/op
Iteration 5: 1681.271 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 1667.734 ±(99.9%) 18.292 ms/op
Histogram, ms/op:
[1630.000, 1635.000) = 1
[1635.000, 1640.000) = 0
[1640.000, 1645.000) = 0
[1645.000, 1650.000) = 1
[1650.000, 1655.000) = 2
[1655.000, 1660.000) = 2
[1660.000, 1665.000) = 0
[1665.000, 1670.000) = 1
[1670.000, 1675.000) = 3
[1675.000, 1680.000) = 1
[1680.000, 1685.000) = 1
[1685.000, 1690.000) = 2
[1690.000, 1695.000) = 1
Percentiles, ms/op:
p(0.0000) = 1632.920 ms/op
p(50.0000) = 1673.540 ms/op
p(90.0000) = 1689.615 ms/op
p(95.0000) = 1691.690 ms/op
p(99.0000) = 1691.690 ms/op
p(99.9000) = 1691.690 ms/op
p(99.9900) = 1691.690 ms/op
p(99.9990) = 1691.690 ms/op
p(99.9999) = 1691.690 ms/op
p(100.0000) = 1691.690 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 10)
# Run progress: 41.67% complete, ETA 00:06:01
# Fork: 1 of 3
# Warmup Iteration 1: 30.700 ms/op
# Warmup Iteration 2: 27.804 ms/op
# Warmup Iteration 3: 26.399 ms/op
# Warmup Iteration 4: 26.987 ms/op
# Warmup Iteration 5: 25.921 ms/op
Iteration 1: 1298.694 ms/op
Iteration 2: 1327.474 ms/op
Iteration 3: 1311.182 ms/op
Iteration 4: 1306.991 ms/op
Iteration 5: 1307.628 ms/op
# Run progress: 44.44% complete, ETA 00:05:31
# Fork: 2 of 3
# Warmup Iteration 1: 30.441 ms/op
# Warmup Iteration 2: 25.993 ms/op
# Warmup Iteration 3: 26.603 ms/op
# Warmup Iteration 4: 24.245 ms/op
# Warmup Iteration 5: 25.948 ms/op
Iteration 1: 1308.471 ms/op
Iteration 2: 1260.652 ms/op
Iteration 3: 1260.747 ms/op
Iteration 4: 1306.077 ms/op
Iteration 5: 1328.110 ms/op
# Run progress: 47.22% complete, ETA 00:05:04
# Fork: 3 of 3
# Warmup Iteration 1: 30.217 ms/op
# Warmup Iteration 2: 26.569 ms/op
# Warmup Iteration 3: 26.516 ms/op
# Warmup Iteration 4: 24.342 ms/op
# Warmup Iteration 5: 24.684 ms/op
Iteration 1: 1282.795 ms/op
Iteration 2: 1300.763 ms/op
Iteration 3: 1282.684 ms/op
Iteration 4: 1256.402 ms/op
Iteration 5: 1280.799 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 1294.631 ±(99.9%) 24.595 ms/op
Histogram, ms/op:
[1250.000, 1255.000) = 0
[1255.000, 1260.000) = 1
[1260.000, 1265.000) = 2
[1265.000, 1270.000) = 0
[1270.000, 1275.000) = 0
[1275.000, 1280.000) = 0
[1280.000, 1285.000) = 3
[1285.000, 1290.000) = 0
[1290.000, 1295.000) = 0
[1295.000, 1300.000) = 1
[1300.000, 1305.000) = 1
[1305.000, 1310.000) = 4
[1310.000, 1315.000) = 1
[1315.000, 1320.000) = 0
[1320.000, 1325.000) = 0
Percentiles, ms/op:
p(0.0000) = 1256.402 ms/op
p(50.0000) = 1300.763 ms/op
p(90.0000) = 1327.728 ms/op
p(95.0000) = 1328.110 ms/op
p(99.0000) = 1328.110 ms/op
p(99.9000) = 1328.110 ms/op
p(99.9900) = 1328.110 ms/op
p(99.9990) = 1328.110 ms/op
p(99.9999) = 1328.110 ms/op
p(100.0000) = 1328.110 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 12)
# Run progress: 50.00% complete, ETA 00:04:38
# Fork: 1 of 3
# Warmup Iteration 1: 25.549 ms/op
# Warmup Iteration 2: 22.359 ms/op
# Warmup Iteration 3: 22.313 ms/op
# Warmup Iteration 4: 21.410 ms/op
# Warmup Iteration 5: 21.575 ms/op
Iteration 1: 1097.967 ms/op
Iteration 2: 1116.401 ms/op
Iteration 3: 1045.636 ms/op
Iteration 4: 1063.058 ms/op
Iteration 5: 1083.852 ms/op
# Run progress: 52.78% complete, ETA 00:04:14
# Fork: 2 of 3
# Warmup Iteration 1: 26.062 ms/op
# Warmup Iteration 2: 22.021 ms/op
# Warmup Iteration 3: 21.212 ms/op
# Warmup Iteration 4: 21.146 ms/op
# Warmup Iteration 5: 20.796 ms/op
Iteration 1: 1108.755 ms/op
Iteration 2: 1123.621 ms/op
Iteration 3: 1096.366 ms/op
Iteration 4: 1092.752 ms/op
Iteration 5: 1128.194 ms/op
# Run progress: 55.56% complete, ETA 00:03:52
# Fork: 3 of 3
# Warmup Iteration 1: 27.039 ms/op
# Warmup Iteration 2: 22.191 ms/op
# Warmup Iteration 3: 22.534 ms/op
# Warmup Iteration 4: 22.015 ms/op
# Warmup Iteration 5: 22.853 ms/op
Iteration 1: 1117.286 ms/op
Iteration 2: 1119.162 ms/op
Iteration 3: 1108.516 ms/op
Iteration 4: 1105.442 ms/op
Iteration 5: 1095.012 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 1100.135 ±(99.9%) 24.146 ms/op
Histogram, ms/op:
[1040.000, 1045.000) = 0
[1045.000, 1050.000) = 1
[1050.000, 1055.000) = 0
[1055.000, 1060.000) = 0
[1060.000, 1065.000) = 1
[1065.000, 1070.000) = 0
[1070.000, 1075.000) = 0
[1075.000, 1080.000) = 0
[1080.000, 1085.000) = 1
[1085.000, 1090.000) = 0
[1090.000, 1095.000) = 1
[1095.000, 1100.000) = 3
[1100.000, 1105.000) = 0
[1105.000, 1110.000) = 3
[1110.000, 1115.000) = 0
[1115.000, 1120.000) = 3
[1120.000, 1125.000) = 1
Percentiles, ms/op:
p(0.0000) = 1045.636 ms/op
p(50.0000) = 1105.442 ms/op
p(90.0000) = 1125.450 ms/op
p(95.0000) = 1128.194 ms/op
p(99.0000) = 1128.194 ms/op
p(99.9000) = 1128.194 ms/op
p(99.9900) = 1128.194 ms/op
p(99.9990) = 1128.194 ms/op
p(99.9999) = 1128.194 ms/op
p(100.0000) = 1128.194 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 16)
# Run progress: 58.33% complete, ETA 00:03:31
# Fork: 1 of 3
# Warmup Iteration 1: 21.907 ms/op
# Warmup Iteration 2: 19.450 ms/op
# Warmup Iteration 3: 16.965 ms/op
# Warmup Iteration 4: 17.706 ms/op
# Warmup Iteration 5: 16.588 ms/op
Iteration 1: 868.400 ms/op
Iteration 2: 851.260 ms/op
Iteration 3: 844.678 ms/op
Iteration 4: 853.957 ms/op
Iteration 5: 853.934 ms/op
# Run progress: 61.11% complete, ETA 00:03:11
# Fork: 2 of 3
# Warmup Iteration 1: 20.580 ms/op
# Warmup Iteration 2: 17.846 ms/op
# Warmup Iteration 3: 17.050 ms/op
# Warmup Iteration 4: 18.343 ms/op
# Warmup Iteration 5: 18.241 ms/op
Iteration 1: 841.637 ms/op
Iteration 2: 862.456 ms/op
Iteration 3: 819.892 ms/op
Iteration 4: 834.720 ms/op
Iteration 5: 855.296 ms/op
# Run progress: 63.89% complete, ETA 00:02:52
# Fork: 3 of 3
# Warmup Iteration 1: 19.925 ms/op
# Warmup Iteration 2: 15.551 ms/op
# Warmup Iteration 3: 16.667 ms/op
# Warmup Iteration 4: 16.792 ms/op
# Warmup Iteration 5: 17.859 ms/op
Iteration 1: 885.718 ms/op
Iteration 2: 888.873 ms/op
Iteration 3: 867.056 ms/op
Iteration 4: 815.778 ms/op
Iteration 5: 845.270 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 852.595 ±(99.9%) 22.063 ms/op
Histogram, ms/op:
[810.000, 815.000) = 0
[815.000, 820.000) = 2
[820.000, 825.000) = 0
[825.000, 830.000) = 0
[830.000, 835.000) = 1
[835.000, 840.000) = 0
[840.000, 845.000) = 2
[845.000, 850.000) = 1
[850.000, 855.000) = 3
[855.000, 860.000) = 1
[860.000, 865.000) = 1
[865.000, 870.000) = 2
[870.000, 875.000) = 0
[875.000, 880.000) = 0
[880.000, 885.000) = 0
Percentiles, ms/op:
p(0.0000) = 815.778 ms/op
p(50.0000) = 853.934 ms/op
p(90.0000) = 886.980 ms/op
p(95.0000) = 888.873 ms/op
p(99.0000) = 888.873 ms/op
p(99.9000) = 888.873 ms/op
p(99.9900) = 888.873 ms/op
p(99.9990) = 888.873 ms/op
p(99.9999) = 888.873 ms/op
p(100.0000) = 888.873 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 32)
# Run progress: 66.67% complete, ETA 00:02:35
# Fork: 1 of 3
# Warmup Iteration 1: 13.597 ms/op
# Warmup Iteration 2: 10.923 ms/op
# Warmup Iteration 3: 9.706 ms/op
# Warmup Iteration 4: 9.420 ms/op
# Warmup Iteration 5: 9.354 ms/op
Iteration 1: 478.279 ms/op
Iteration 2: 480.373 ms/op
Iteration 3: 463.662 ms/op
Iteration 4: 460.174 ms/op
Iteration 5: 451.356 ms/op
# Run progress: 69.44% complete, ETA 00:02:17
# Fork: 2 of 3
# Warmup Iteration 1: 13.845 ms/op
# Warmup Iteration 2: 12.461 ms/op
# Warmup Iteration 3: 9.545 ms/op
# Warmup Iteration 4: 9.930 ms/op
# Warmup Iteration 5: 9.276 ms/op
Iteration 1: 451.655 ms/op
Iteration 2: 465.714 ms/op
Iteration 3: 460.197 ms/op
Iteration 4: 457.352 ms/op
Iteration 5: 460.038 ms/op
# Run progress: 72.22% complete, ETA 00:02:01
# Fork: 3 of 3
# Warmup Iteration 1: 14.027 ms/op
# Warmup Iteration 2: 9.559 ms/op
# Warmup Iteration 3: 8.492 ms/op
# Warmup Iteration 4: 8.687 ms/op
# Warmup Iteration 5: 8.928 ms/op
Iteration 1: 459.844 ms/op
Iteration 2: 478.366 ms/op
Iteration 3: 461.546 ms/op
Iteration 4: 453.185 ms/op
Iteration 5: 458.808 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 462.703 ±(99.9%) 9.999 ms/op
Histogram, ms/op:
[450.000, 452.500) = 2
[452.500, 455.000) = 1
[455.000, 457.500) = 1
[457.500, 460.000) = 2
[460.000, 462.500) = 4
[462.500, 465.000) = 1
[465.000, 467.500) = 1
[467.500, 470.000) = 0
[470.000, 472.500) = 0
[472.500, 475.000) = 0
[475.000, 477.500) = 0
[477.500, 480.000) = 2
[480.000, 482.500) = 1
[482.500, 485.000) = 0
[485.000, 487.500) = 0
Percentiles, ms/op:
p(0.0000) = 451.356 ms/op
p(50.0000) = 460.174 ms/op
p(90.0000) = 479.169 ms/op
p(95.0000) = 480.373 ms/op
p(99.0000) = 480.373 ms/op
p(99.9000) = 480.373 ms/op
p(99.9900) = 480.373 ms/op
p(99.9990) = 480.373 ms/op
p(99.9999) = 480.373 ms/op
p(100.0000) = 480.373 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 64)
# Run progress: 75.00% complete, ETA 00:01:46
# Fork: 1 of 3
# Warmup Iteration 1: 15.478 ms/op
# Warmup Iteration 2: 4.982 ms/op
# Warmup Iteration 3: 5.118 ms/op
# Warmup Iteration 4: 5.237 ms/op
# Warmup Iteration 5: 5.030 ms/op
Iteration 1: 251.656 ms/op
Iteration 2: 265.500 ms/op
Iteration 3: 258.723 ms/op
Iteration 4: 256.243 ms/op
Iteration 5: 255.338 ms/op
# Run progress: 77.78% complete, ETA 00:01:31
# Fork: 2 of 3
# Warmup Iteration 1: 18.797 ms/op
# Warmup Iteration 2: 6.272 ms/op
# Warmup Iteration 3: 4.905 ms/op
# Warmup Iteration 4: 4.979 ms/op
# Warmup Iteration 5: 5.063 ms/op
Iteration 1: 259.664 ms/op
Iteration 2: 273.808 ms/op
Iteration 3: 267.144 ms/op
Iteration 4: 265.301 ms/op
Iteration 5: 258.136 ms/op
# Run progress: 80.56% complete, ETA 00:01:17
# Fork: 3 of 3
# Warmup Iteration 1: 15.761 ms/op
# Warmup Iteration 2: 6.045 ms/op
# Warmup Iteration 3: 6.442 ms/op
# Warmup Iteration 4: 5.106 ms/op
# Warmup Iteration 5: 5.102 ms/op
Iteration 1: 253.595 ms/op
Iteration 2: 253.119 ms/op
Iteration 3: 257.354 ms/op
Iteration 4: 253.247 ms/op
Iteration 5: 259.585 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 259.228 ±(99.9%) 6.658 ms/op
Histogram, ms/op:
[250.000, 252.500) = 1
[252.500, 255.000) = 3
[255.000, 257.500) = 3
[257.500, 260.000) = 4
[260.000, 262.500) = 0
[262.500, 265.000) = 0
[265.000, 267.500) = 3
[267.500, 270.000) = 0
[270.000, 272.500) = 0
[272.500, 275.000) = 1
[275.000, 277.500) = 0
Percentiles, ms/op:
p(0.0000) = 251.656 ms/op
p(50.0000) = 258.136 ms/op
p(90.0000) = 269.810 ms/op
p(95.0000) = 273.808 ms/op
p(99.0000) = 273.808 ms/op
p(99.9000) = 273.808 ms/op
p(99.9900) = 273.808 ms/op
p(99.9990) = 273.808 ms/op
p(99.9999) = 273.808 ms/op
p(100.0000) = 273.808 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 128)
# Run progress: 83.33% complete, ETA 00:01:04
# Fork: 1 of 3
# Warmup Iteration 1: 21.805 ms/op
# Warmup Iteration 2: 4.302 ms/op
# Warmup Iteration 3: 5.247 ms/op
# Warmup Iteration 4: 2.913 ms/op
# Warmup Iteration 5: 2.888 ms/op
Iteration 1: 145.937 ms/op
Iteration 2: 149.597 ms/op
Iteration 3: 153.050 ms/op
Iteration 4: 137.560 ms/op
Iteration 5: 135.283 ms/op
# Run progress: 86.11% complete, ETA 00:00:52
# Fork: 2 of 3
# Warmup Iteration 1: 19.478 ms/op
# Warmup Iteration 2: 4.322 ms/op
# Warmup Iteration 3: 5.057 ms/op
# Warmup Iteration 4: 4.578 ms/op
# Warmup Iteration 5: 4.696 ms/op
Iteration 1: 143.142 ms/op
Iteration 2: 141.474 ms/op
Iteration 3: 144.636 ms/op
Iteration 4: 141.199 ms/op
Iteration 5: 146.226 ms/op
# Run progress: 88.89% complete, ETA 00:00:40
# Fork: 3 of 3
# Warmup Iteration 1: 22.280 ms/op
# Warmup Iteration 2: 4.812 ms/op
# Warmup Iteration 3: 5.316 ms/op
# Warmup Iteration 4: 4.046 ms/op
# Warmup Iteration 5: 4.280 ms/op
Iteration 1: 151.236 ms/op
Iteration 2: 135.365 ms/op
Iteration 3: 136.527 ms/op
Iteration 4: 135.857 ms/op
Iteration 5: 136.144 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 142.215 ±(99.9%) 6.502 ms/op
Histogram, ms/op:
[130.000, 132.500) = 0
[132.500, 135.000) = 0
[135.000, 137.500) = 5
[137.500, 140.000) = 1
[140.000, 142.500) = 2
[142.500, 145.000) = 2
[145.000, 147.500) = 2
[147.500, 150.000) = 1
[150.000, 152.500) = 1
[152.500, 155.000) = 1
[155.000, 157.500) = 0
Percentiles, ms/op:
p(0.0000) = 135.283 ms/op
p(50.0000) = 141.474 ms/op
p(90.0000) = 151.962 ms/op
p(95.0000) = 153.050 ms/op
p(99.0000) = 153.050 ms/op
p(99.9000) = 153.050 ms/op
p(99.9900) = 153.050 ms/op
p(99.9990) = 153.050 ms/op
p(99.9999) = 153.050 ms/op
p(100.0000) = 153.050 ms/op
# JMH version: 1.21
# VM version: JDK 1.8.0_121, Java HotSpot(TM) 64-Bit Server VM, 25.121-b13
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 5 iterations, single-shot each
# Measurement: 5 iterations, single-shot each, 50 calls per op
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: me.lotabout.MyBenchmark.supplyAsync
# Parameters: (N = 200, parallelism = 256)
# Run progress: 91.67% complete, ETA 00:00:29
# Fork: 1 of 3
# Warmup Iteration 1: 27.229 ms/op
# Warmup Iteration 2: 9.791 ms/op
# Warmup Iteration 3: 9.600 ms/op
# Warmup Iteration 4: 9.304 ms/op
# Warmup Iteration 5: 3.098 ms/op
Iteration 1: 126.999 ms/op
Iteration 2: 107.726 ms/op
Iteration 3: 106.326 ms/op
Iteration 4: 102.371 ms/op
Iteration 5: 106.237 ms/op
# Run progress: 94.44% complete, ETA 00:00:19
# Fork: 2 of 3
# Warmup Iteration 1: 28.846 ms/op
# Warmup Iteration 2: 10.913 ms/op
# Warmup Iteration 3: 11.490 ms/op
# Warmup Iteration 4: 9.134 ms/op
# Warmup Iteration 5: 3.326 ms/op
Iteration 1: 192.659 ms/op
Iteration 2: 658.216 ms/op
Iteration 3: 105.330 ms/op
Iteration 4: 105.198 ms/op
Iteration 5: 109.281 ms/op
# Run progress: 97.22% complete, ETA 00:00:09
# Fork: 3 of 3
# Warmup Iteration 1: 34.203 ms/op
# Warmup Iteration 2: 11.308 ms/op
# Warmup Iteration 3: 8.781 ms/op
# Warmup Iteration 4: 2.912 ms/op
# Warmup Iteration 5: 2.054 ms/op
Iteration 1: 161.697 ms/op
Iteration 2: 264.726 ms/op
Iteration 3: 105.796 ms/op
Iteration 4: 104.531 ms/op
Iteration 5: 105.695 ms/op
Result "me.lotabout.MyBenchmark.supplyAsync":
N = 15
mean = 164.186 ±(99.9%) 153.980 ms/op
Histogram, ms/op:
[100.000, 150.000) = 11
[150.000, 200.000) = 2
[200.000, 250.000) = 0
[250.000, 300.000) = 1
[300.000, 350.000) = 0
[350.000, 400.000) = 0
[400.000, 450.000) = 0
[450.000, 500.000) = 0
[500.000, 550.000) = 0
[550.000, 600.000) = 0
[600.000, 650.000) = 0
Percentiles, ms/op:
p(0.0000) = 102.371 ms/op
p(50.0000) = 106.326 ms/op
p(90.0000) = 422.122 ms/op
p(95.0000) = 658.216 ms/op
p(99.0000) = 658.216 ms/op
p(99.9000) = 658.216 ms/op
p(99.9900) = 658.216 ms/op
p(99.9990) = 658.216 ms/op
p(99.9999) = 658.216 ms/op
p(100.0000) = 658.216 ms/op
# Run complete. Total time: 00:05:30
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark (N) (parallelism) Mode Cnt Score Error Units
MyBenchmark.supplyAsync 200 1 ss 15 574.706 ± 27.327 ms/op
MyBenchmark.supplyAsync 200 2 ss 15 6591.250 ± 64.085 ms/op
MyBenchmark.supplyAsync 200 3 ss 15 4419.905 ± 40.197 ms/op
MyBenchmark.supplyAsync 200 4 ss 15 3295.410 ± 52.130 ms/op
MyBenchmark.supplyAsync 200 8 ss 15 1667.734 ± 18.292 ms/op
MyBenchmark.supplyAsync 200 10 ss 15 1294.631 ± 24.595 ms/op
MyBenchmark.supplyAsync 200 12 ss 15 1100.135 ± 24.146 ms/op
MyBenchmark.supplyAsync 200 16 ss 15 852.595 ± 22.063 ms/op
MyBenchmark.supplyAsync 200 32 ss 15 462.703 ± 9.999 ms/op
MyBenchmark.supplyAsync 200 64 ss 15 259.228 ± 6.658 ms/op
MyBenchmark.supplyAsync 200 128 ss 15 142.215 ± 6.502 ms/op
MyBenchmark.supplyAsync 200 256 ss 15 164.186 ± 153.980 ms/op
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment