Skip to content

Instantly share code, notes, and snippets.

@danielmitterdorfer

danielmitterdorfer/bulk_index_throughput.png Secret

Last active August 8, 2016 13:46
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save danielmitterdorfer/2de0dfd61092eaad84958319cfc0d61b to your computer and use it in GitHub Desktop.
Save danielmitterdorfer/2de0dfd61092eaad84958319cfc0d61b to your computer and use it in GitHub Desktop.
Download ZIP
REST Client Benchmark
Raw
results.md

Goal

We want to compare the (single-threaded) performance of the transport client against the new REST client.

For that we chose two typical operations: bulk indexing and a search operation (which search operation does not really matter because we want to benchmark the client, not the server, although it is better to choose searches that execute fast in order to avoid hiding differences in measurement noise).

The performance characteristics are throughput (how many operations/s can we achieve?) and service time (how long does the respective method call on the client take (including roundtrip time to the server)?). We intentionally do not account for coordinated omission as we assume a closed system (the client is used as part of a typical (web) application). That's also the reason why we determine "service time" rather than "latency".

Note that service time is measured at defined throughput levels (and not just hammering Elasticsearch as hard as we can). The benchmark driver attempts to reach a specific throughput, called "target throughput". In addition to target throughput, we report the actually achieved throughput.

System under test

Elasticsearch client implementations and the respective Elasticsearch request handling code (i.e. bulk indexing action and search action).

Benchmark Setup

  • System: Linux 4.6.4-1-ARCH
  • JVM: Oracle Java 1.8.0_92-b14
  • CPU: Intel(R) Xeon(R) CPU E3-1270 v5 @ 3.60GHz (CPU frequency for all cores locked at 3.4 GHz, performance CPU governor)
  • One ES node with default production settings and 4GB heap
  • Client pinned to core 0 with taskset.
  • Benchmarked version of Elasticsearch: git hash bf51247ec00a40b0aeb66b4a14ba93dc5556cfe3

To avoid network overhead, we benchmarked on the same machine and pinned the (single threaded) client benchmark driver (see below).

Bulk Index benchmark

  • Each benchmark is run 3 consecutive times. Between benchmark trials we restart Elasticsearch and also physically delete the index.
  • 40% of all bulk requests are treated as warmup iterations which are not taken into account when reporting metrics.
  • We vary the bulk size between 500 docs per bulk request and 50,000 docs per bulk request and don't limit throughput.

Search benchmark

  • Precondition: We rely on a prepopulated index from the bulk index benchmark.
  • Each benchmark is run 5 consecutive times in the same JVM.
  • We run 10,000 warmup iterations which are not taken into account when reporting metrics and 10,000 measurement iterations.
  • We vary the target throughput between 100 and 100,000 requests per second.

Command line parameters (REST):

taskset -c 0 java -XX:+UnlockDiagnosticVMOptions -Xms4096M -Xmx4096M -XX:+UseConcMarkSweepGC -verbose:gc -XX:+PrintGCDetails -XX:+PrintSafepointStatistics -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCCause -XX:GuaranteedSafepointInterval=3600000 -jar client-benchmarks-5.0.0-alpha5-SNAPSHOT-all.jar rest 127.0.0.1 documents.json geonames type 8647880 5000 "{ \"query\": { \"match_phrase\": { \"name\": \"Sankt Georgen\" } } }\""

Command line parameters (transport):

taskset -c 0 java -XX:+UnlockDiagnosticVMOptions -Xms4096M -Xmx4096M  -XX:+UseConcMarkSweepGC -verbose:gc -XX:+PrintGCDetails -XX:+PrintSafepointStatistics -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCCause -XX:GuaranteedSafepointInterval=3600000 -jar client-benchmarks-5.0.0-alpha5-SNAPSHOT-all.jar transport 127.0.0.1 documents.json geonames type 8647880 5000 "{ \"match_phrase\": { \"name\": \"Sankt Georgen\" } }\""

For smaller bulk sizes, the bulk size parameter has been adjusted accordingly and we also did not run the search benchmark again.

Summary / Interpretation

Bulk Indexing

Both clients are roughly on par w.r.t. throughput and latency.

Indexing Throughput

Search

For a realistic scenario, we have included search request processing in the benchmark. Both clients can saturate a single node Elasticsearch cluster as achieved throughput flattens around 2,500 operations/s.

Tail service times do not increase significantly further with higher target throughputs indicating that we did not push the client to its limits but are rather constrained by request processing in Elasticsearch.

Below is a comparison of the service times for a target throughput of 100 operations/s. For each client type, we have chosen the trial run with the worst 99.99th percentile service time. As the service time does not change significantly with higher target throughputs, we only present the results for a target throughput of 100 operations/s.

Search Service Time

As we can see, the service times of both clients are comparable.

We also took one test where we stub out the search request handling code in Elasticsearch. This test showed that the transport client can reach a throughput of roughly 17,000 operations / s and the REST client a bit less, roughly 13,000 operations / s. Note that these values still include the roundtrip to Elasticsearch. We have just replaced the search code in TransportSearchAction with the following stub:

listener.onResponse(new SearchResponse(new InternalSearchResponse(
    new InternalSearchHits(
        new InternalSearchHit[0], 0L, 0.0f),
    new InternalAggregations(Collections.emptyList()),
    new Suggest(Collections.emptyList()), new SearchProfileShardResults(Collections.emptyMap()), false, false), "", 1, 1, 0, new ShardSearchFailure[0]));

We'd expect that at least part of the difference in throughput can be accounted to different code paths that are exercised in Elasticsearch but did not investigate further.

Raw results

Bulk indexing benchmark

REST client

Bulk size = 500

----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 14.161295 ops/s, p90 = 89.827060 ms, p95 = 93.597530 ms, p99 = 122.945063 ms, p99.9 = 416.324246 ms, p99.99 = 738.925592 ms
success count = 10377, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 14.238887 ops/s, p90 = 89.441558 ms, p95 = 93.634877 ms, p99 = 124.507943 ms, p99.9 = 431.485352 ms, p99.99 = 732.643461 ms
success count = 10377, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 14.373727 ops/s, p90 = 88.621217 ms, p95 = 92.740257 ms, p99 = 122.718190 ms, p99.9 = 328.315777 ms, p99.99 = 793.512759 ms
success count = 10377, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------

Bulk Size = 5,000

-----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 5.324823 ops/s, p90 = 229.735356 ms, p95 = 264.828318 ms, p99 = 437.578298 ms, p99.9 = 813.742307 ms, p99.99 = 819.001147 ms
success count = 1038, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 5.221142 ops/s, p90 = 241.546441 ms, p95 = 269.865622 ms, p99 = 448.660294 ms, p99.9 = 733.270223 ms, p99.99 = 735.158014 ms
success count = 1038, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 5.287249 ops/s, p90 = 234.703333 ms, p95 = 270.863615 ms, p99 = 417.647984 ms, p99.9 = 816.044974 ms, p99.99 = 817.854353 ms
success count = 1038, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------------

Bulk size = 50,000

Note: For this benchmark we have allocated an 8GB heap for the benchmark client with -Xms8192M -Xmx8192M.

----------------------------------------------------------------------------------------------------------------------------------------------
Throughput = 0.699820 ops/s, p90 = 1712.092796 ms, p95 = 1816.040775 ms, p99 = 2087.240439 ms, p99.9 = 2097.032488 ms, p99.99 = 2097.032488 ms
success count = 104, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------
Throughput = 0.703642 ops/s, p90 = 1705.658244 ms, p95 = 1881.572634 ms, p99 = 2118.300104 ms, p99.9 = 2122.255816 ms, p99.99 = 2122.255816 ms
success count = 104, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------
Throughput = 0.699942 ops/s, p90 = 1673.373163 ms, p95 = 1814.511778 ms, p99 = 2120.784529 ms, p99.9 = 2124.665858 ms, p99.99 = 2124.665858 ms
success count = 104, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------------
Transport client

Bulk Size = 500

----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 14.958733 ops/s, p90 = 87.350785 ms, p95 = 91.282197 ms, p99 = 118.075608 ms, p99.9 = 325.896215 ms, p99.99 = 620.186538 ms
success count = 10377, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 15.024396 ops/s, p90 = 86.719914 ms, p95 = 90.588683 ms, p99 = 117.241998 ms, p99.9 = 351.608185 ms, p99.99 = 686.587708 ms
success count = 10377, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 14.535440 ops/s, p90 = 88.089208 ms, p95 = 92.005229 ms, p99 = 121.615030 ms, p99.9 = 306.872181 ms, p99.99 = 676.198084 ms
success count = 10377, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------

Bulk Size = 5,000

-----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 5.321748 ops/s, p90 = 230.694256 ms, p95 = 268.908635 ms, p99 = 442.016533 ms, p99.9 = 906.763333 ms, p99.99 = 911.951069 ms
success count = 1038, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 5.439259 ops/s, p90 = 222.891606 ms, p95 = 264.541269 ms, p99 = 406.944071 ms, p99.9 = 784.375289 ms, p99.99 = 789.581406 ms
success count = 1038, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------------------------------------------------------
Throughput = 5.412568 ops/s, p90 = 225.970822 ms, p95 = 274.699452 ms, p99 = 408.132151 ms, p99.9 = 726.572544 ms, p99.99 = 731.448762 ms
success count = 1038, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------------

Bulk Size = 50,000

Note: For this benchmark we have allocated an 8GB heap for the benchmark client with -Xms8192M -Xmx8192M.

----------------------------------------------------------------------------------------------------------------------------------------------
Throughput = 0.733697 ops/s, p90 = 1675.221855 ms, p95 = 1809.757996 ms, p99 = 2251.328932 ms, p99.9 = 2257.920282 ms, p99.99 = 2257.920282 ms
success count = 104, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------
Throughput = 0.747903 ops/s, p90 = 1613.517281 ms, p95 = 1850.680329 ms, p99 = 2247.524457 ms, p99.9 = 2265.070337 ms, p99.99 = 2265.070337 ms
success count = 104, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------
Throughput = 0.754228 ops/s, p90 = 1666.424191 ms, p95 = 1791.406626 ms, p99 = 2352.237733 ms, p99.9 = 2354.475733 ms, p99.99 = 2354.475733 ms
success count = 104, error count = 0
----------------------------------------------------------------------------------------------------------------------------------------------

Search benchmark

REST client
=============
 Trial run 1
=============
Target throughput = 100 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
----------------------------------------------------------------------------------------------------------------------------------
Throughput = 99.858802 ops/s, p90 = 0.577605 ms, p95 = 0.615335 ms, p99 = 0.692266 ms, p99.9 = 15.437697 ms, p99.99 = 20.454479 ms
success count = 10000, error count = 0
----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 1000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
-----------------------------------------------------------------------------------------------------------------------------------
Throughput = 962.197830 ops/s, p90 = 0.619316 ms, p95 = 0.669246 ms, p99 = 1.003487 ms, p99.9 = 15.247768 ms, p99.99 = 19.833752 ms
success count = 10000, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 10000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2422.277065 ops/s, p90 = 0.452784 ms, p95 = 0.510404 ms, p99 = 0.895995 ms, p99.9 = 15.173094 ms, p99.99 = 22.741543 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Target throughput = 100000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2384.805735 ops/s, p90 = 0.458614 ms, p95 = 0.522185 ms, p99 = 0.898706 ms, p99.9 = 16.468813 ms, p99.99 = 23.202884 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
=============
 Trial run 2
=============
Target throughput = 100 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
----------------------------------------------------------------------------------------------------------------------------------
Throughput = 99.845087 ops/s, p90 = 0.586176 ms, p95 = 0.617089 ms, p99 = 0.695661 ms, p99.9 = 16.207999 ms, p99.99 = 19.452390 ms
success count = 10000, error count = 0
----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 1000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
-----------------------------------------------------------------------------------------------------------------------------------
Throughput = 964.138522 ops/s, p90 = 0.617298 ms, p95 = 0.660951 ms, p99 = 0.949524 ms, p99.9 = 15.594627 ms, p99.99 = 18.195841 ms
success count = 10000, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 10000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2437.853362 ops/s, p90 = 0.461444 ms, p95 = 0.516686 ms, p99 = 0.898849 ms, p99.9 = 14.831436 ms, p99.99 = 17.939168 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Target throughput = 100000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2477.069745 ops/s, p90 = 0.430508 ms, p95 = 0.486526 ms, p99 = 0.928777 ms, p99.9 = 15.224350 ms, p99.99 = 24.936530 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
=============
 Trial run 3
=============
Target throughput = 100 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
----------------------------------------------------------------------------------------------------------------------------------
Throughput = 99.864214 ops/s, p90 = 0.588257 ms, p95 = 0.627751 ms, p99 = 0.693613 ms, p99.9 = 15.532338 ms, p99.99 = 19.504201 ms
success count = 10000, error count = 0
----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 1000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
-----------------------------------------------------------------------------------------------------------------------------------
Throughput = 964.208115 ops/s, p90 = 0.621514 ms, p95 = 0.668300 ms, p99 = 0.961835 ms, p99.9 = 14.950091 ms, p99.99 = 17.690016 ms
success count = 10000, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 10000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2449.903500 ops/s, p90 = 0.456502 ms, p95 = 0.513315 ms, p99 = 0.895950 ms, p99.9 = 14.966800 ms, p99.99 = 17.655408 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Target throughput = 100000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2499.910876 ops/s, p90 = 0.436453 ms, p95 = 0.498843 ms, p99 = 0.915429 ms, p99.9 = 14.334258 ms, p99.99 = 18.833310 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
=============
 Trial run 4
=============
Target throughput = 100 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
----------------------------------------------------------------------------------------------------------------------------------
Throughput = 99.861475 ops/s, p90 = 0.591075 ms, p95 = 0.626712 ms, p99 = 0.691141 ms, p99.9 = 15.498463 ms, p99.99 = 20.188814 ms
success count = 10000, error count = 0
----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 1000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
-----------------------------------------------------------------------------------------------------------------------------------
Throughput = 961.250414 ops/s, p90 = 0.624529 ms, p95 = 0.667464 ms, p99 = 0.932602 ms, p99.9 = 16.127658 ms, p99.99 = 21.446496 ms
success count = 10000, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 10000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2506.002504 ops/s, p90 = 0.438028 ms, p95 = 0.480228 ms, p99 = 0.892466 ms, p99.9 = 15.079973 ms, p99.99 = 18.667954 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Target throughput = 100000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2489.974347 ops/s, p90 = 0.427624 ms, p95 = 0.483491 ms, p99 = 0.923890 ms, p99.9 = 14.972515 ms, p99.99 = 16.089594 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
=============
 Trial run 5
=============
Target throughput = 100 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
----------------------------------------------------------------------------------------------------------------------------------
Throughput = 99.858199 ops/s, p90 = 0.584442 ms, p95 = 0.622223 ms, p99 = 0.700156 ms, p99.9 = 15.427299 ms, p99.99 = 19.636319 ms
success count = 10000, error count = 0
----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 1000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
-----------------------------------------------------------------------------------------------------------------------------------
Throughput = 963.760902 ops/s, p90 = 0.621503 ms, p95 = 0.663147 ms, p99 = 0.960264 ms, p99.9 = 15.638613 ms, p99.99 = 19.442293 ms
success count = 10000, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 10000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2483.680476 ops/s, p90 = 0.438673 ms, p95 = 0.488318 ms, p99 = 0.904511 ms, p99.9 = 14.709731 ms, p99.99 = 18.306186 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Target throughput = 100000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2478.254070 ops/s, p90 = 0.440038 ms, p95 = 0.494122 ms, p99 = 0.915670 ms, p99.9 = 14.538895 ms, p99.99 = 17.918478 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Transport client
=============
 Trial run 1
=============
Target throughput = 100 ops / s
Operation: search
----------------------------------------------------------------------------------------------------------------------------------
Throughput = 99.841584 ops/s, p90 = 0.605054 ms, p95 = 0.634399 ms, p99 = 0.691421 ms, p99.9 = 15.816485 ms, p99.99 = 22.199781 ms
success count = 10000, error count = 0
----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 1000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
-----------------------------------------------------------------------------------------------------------------------------------
Throughput = 960.268341 ops/s, p90 = 0.617379 ms, p95 = 0.681679 ms, p99 = 0.934950 ms, p99.9 = 15.580209 ms, p99.99 = 20.581855 ms
success count = 10000, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 10000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2610.393852 ops/s, p90 = 0.400153 ms, p95 = 0.449177 ms, p99 = 0.853240 ms, p99.9 = 15.302408 ms, p99.99 = 17.259946 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Target throughput = 100000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2534.724913 ops/s, p90 = 0.410033 ms, p95 = 0.464485 ms, p99 = 0.882938 ms, p99.9 = 15.413459 ms, p99.99 = 51.305936 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
=============
 Trial run 2
=============
Target throughput = 100 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
----------------------------------------------------------------------------------------------------------------------------------
Throughput = 99.829328 ops/s, p90 = 0.612975 ms, p95 = 0.637624 ms, p99 = 0.717228 ms, p99.9 = 15.700880 ms, p99.99 = 23.600368 ms
success count = 10000, error count = 0
----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 1000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
-----------------------------------------------------------------------------------------------------------------------------------
Throughput = 961.438666 ops/s, p90 = 0.614163 ms, p95 = 0.681844 ms, p99 = 0.940075 ms, p99.9 = 15.705636 ms, p99.99 = 22.791300 ms
success count = 10000, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 10000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2671.025272 ops/s, p90 = 0.390803 ms, p95 = 0.430519 ms, p99 = 0.848583 ms, p99.9 = 15.032098 ms, p99.99 = 19.187328 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Target throughput = 100000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2636.988070 ops/s, p90 = 0.394995 ms, p95 = 0.443977 ms, p99 = 0.898199 ms, p99.9 = 14.616566 ms, p99.99 = 16.635615 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
=============
 Trial run 3
=============
Target throughput = 100 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
----------------------------------------------------------------------------------------------------------------------------------
Throughput = 99.842786 ops/s, p90 = 0.603736 ms, p95 = 0.632936 ms, p99 = 0.711477 ms, p99.9 = 15.744591 ms, p99.99 = 22.125040 ms
success count = 10000, error count = 0
----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 1000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
-----------------------------------------------------------------------------------------------------------------------------------
Throughput = 959.843143 ops/s, p90 = 0.594134 ms, p95 = 0.666851 ms, p99 = 0.924774 ms, p99.9 = 15.762936 ms, p99.99 = 23.275361 ms
success count = 10000, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 10000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2662.748510 ops/s, p90 = 0.395715 ms, p95 = 0.446048 ms, p99 = 0.874397 ms, p99.9 = 14.783485 ms, p99.99 = 16.557632 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Target throughput = 100000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2638.403185 ops/s, p90 = 0.394474 ms, p95 = 0.434318 ms, p99 = 0.868906 ms, p99.9 = 15.340628 ms, p99.99 = 19.759272 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
=============
 Trial run 4
=============
Target throughput = 100 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
----------------------------------------------------------------------------------------------------------------------------------
Throughput = 99.851666 ops/s, p90 = 0.580134 ms, p95 = 0.623238 ms, p99 = 0.692727 ms, p99.9 = 15.591430 ms, p99.99 = 19.818205 ms
success count = 10000, error count = 0
----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 1000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
-----------------------------------------------------------------------------------------------------------------------------------
Throughput = 962.127098 ops/s, p90 = 0.624135 ms, p95 = 0.682212 ms, p99 = 0.931517 ms, p99.9 = 15.452340 ms, p99.99 = 19.614005 ms
success count = 10000, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 10000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2670.142295 ops/s, p90 = 0.385294 ms, p95 = 0.424333 ms, p99 = 0.858917 ms, p99.9 = 15.536577 ms, p99.99 = 17.470832 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Target throughput = 100000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2646.184062 ops/s, p90 = 0.394717 ms, p95 = 0.437751 ms, p99 = 0.821088 ms, p99.9 = 15.219609 ms, p99.99 = 16.651344 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
=============
 Trial run 5
=============
Target throughput = 100 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
----------------------------------------------------------------------------------------------------------------------------------
Throughput = 99.828664 ops/s, p90 = 0.596241 ms, p95 = 0.631620 ms, p99 = 0.694086 ms, p99.9 = 15.926534 ms, p99.99 = 20.853050 ms
success count = 10000, error count = 0
----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 1000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
-----------------------------------------------------------------------------------------------------------------------------------
Throughput = 962.148618 ops/s, p90 = 0.617395 ms, p95 = 0.677396 ms, p99 = 0.908102 ms, p99.9 = 15.516983 ms, p99.99 = 20.751288 ms
success count = 10000, error count = 0
-----------------------------------------------------------------------------------------------------------------------------------
Target throughput = 10000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2679.511059 ops/s, p90 = 0.389382 ms, p95 = 0.430864 ms, p99 = 0.852091 ms, p99.9 = 14.879085 ms, p99.99 = 18.278016 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Target throughput = 100000 ops / s
Running SearchBenchmarkTask with 10000 warmup iterations and 10000 iterations.
------------------------------------------------------------------------------------------------------------------------------------
Throughput = 2639.028731 ops/s, p90 = 0.394373 ms, p95 = 0.446464 ms, p99 = 0.891780 ms, p99.9 = 15.411441 ms, p99.99 = 18.081259 ms
success count = 10000, error count = 0
------------------------------------------------------------------------------------------------------------------------------------
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment