Skip to content

Instantly share code, notes, and snippets.

@skhatri

skhatri/loadtest.md

Created March 30, 2020 06:15
Show Gist options
  • Save skhatri/bdba17d19058ec29face971e94b77782 to your computer and use it in GitHub Desktop.
Save skhatri/bdba17d19058ec29face971e94b77782 to your computer and use it in GitHub Desktop.
Download ZIP
microservices-starter-java load test
Raw
loadtest.md

Load Test Results

source: https://github.com/skhatri/microservices-starter-java

Load Testing

400k requests with 50 connections and 20 concurrent streams

h2load -n400000 -c50 -m20 https://localhost:8080/todo/search

jetty

java -Xms512m -Xmx512m -Dserver.jetty.max-threads=50  -Dserver.jetty.acceptors=15 -jar jetty.jar

Results with 50 threads

finished in 137.01s, 2919.49 req/s, 4.02MB/s
requests: 400000 total, 400000 started, 400000 done, 400000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 400000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 551.38MB (578159017) total, 3.58MB (3756267) headers (space savings 89.45%), 537.49MB (563600000) data
                     min         max         mean         sd        +/- sd
time for request:      716us       3.57s    334.92ms    314.13ms    59.61%
time for connect:    66.52ms    274.29ms    114.18ms     64.77ms    86.00%
time to 1st byte:   662.54ms       2.46s       1.39s    689.28ms    40.00%
req/s           :      58.39       66.06       59.63        1.61    86.00%

Jetty with default setup

java -Xms512m -Xmx512m -jar jetty.jar

Results with default

finished in 160.19s, 2497.04 req/s, 3.44MB/s
requests: 400000 total, 400000 started, 400000 done, 400000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 400000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 551.40MB (578188067) total, 3.61MB (3785317) headers (space savings 89.37%), 537.49MB (563600000) data
                     min         max         mean         sd        +/- sd
time for request:     1.03ms       3.83s    398.42ms    242.03ms    85.10%
time for connect:    78.85ms    121.84ms     97.06ms     12.89ms    58.00%
time to 1st byte:      1.11s       2.50s       1.38s    291.90ms    88.00%
req/s           :      49.94       51.12       50.09        0.21    88.00%

netty

Reactor-Netty

Netty with 32 threads

java -Xms512m -Xmx512m -Dreactor.netty.ioWorkerCount=32 -jar reactor-netty.jar

Results

finished in 93.85s, 4261.99 req/s, 5.85MB/s
requests: 400000 total, 400000 started, 400000 done, 400000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 400000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 548.77MB (575422300) total, 1.15MB (1201100) headers (space savings 94.64%), 537.49MB (563600000) data
                     min         max         mean         sd        +/- sd
time for request:     8.30ms       3.02s    211.84ms    183.09ms    90.30%
time for connect:   245.39ms       3.89s       1.21s       1.38s    74.00%
time to 1st byte:      1.43s       5.05s       2.60s       1.36s    74.00%
req/s           :      85.24      120.40       95.03       14.80    72.00%

Reactor-Netty Default setup

java -Xms512m -Xmx512m -jar reactor-netty.jar

Result

finished in 91.84s, 4355.60 req/s, 5.98MB/s
requests: 400000 total, 400000 started, 400000 done, 400000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 400000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 548.77MB (575422300) total, 1.15MB (1201100) headers (space savings 94.64%), 537.49MB (563600000) data
                     min         max         mean         sd        +/- sd
time for request:     8.66ms    658.12ms    204.29ms     79.04ms    86.19%
time for connect:   179.38ms      15.15s       6.11s       4.74s    56.00%
time to 1st byte:   636.42ms      15.70s       6.51s       4.81s    58.00%
req/s           :      87.12       97.25       91.16        2.84    66.00%

tomcat

Tomcat with 50 threads

java -Xms512m -Xmx512m -Dserver.tomcat.max-threads=50 -jar tomcat.jar

Result

finished in 170.04s, 2352.41 req/s, 3.30MB/s
requests: 400000 total, 400000 started, 400000 done, 400000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 400000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 561.92MB (589216250) total, 14.11MB (14800600) headers (space savings 58.43%), 537.49MB (563600000) data
                     min         max         mean         sd        +/- sd
time for request:    11.97ms       2.50s    418.76ms    159.01ms    85.31%
time for connect:      2.07s       2.11s       2.09s      8.47ms    70.00%
time to 1st byte:      2.66s       4.04s       3.34s    427.47ms    58.00%
req/s           :      47.05       47.39       47.14        0.08    66.00%

Tomcat with Default setup

java -Xms512m -Xmx512m -jar tomcat.jar

Result

finished in 210.51s, 1900.17 req/s, 2.67MB/s
requests: 400000 total, 400000 started, 400000 done, 400000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 400000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 561.92MB (589219514) total, 14.11MB (14800600) headers (space savings 58.43%), 537.49MB (563600000) data
                     min         max         mean         sd        +/- sd
time for request:     1.26ms       3.59s    516.48ms    287.44ms    80.40%
time for connect:      1.33s       1.38s       1.35s     13.90ms    76.00%
time to 1st byte:      2.43s       2.47s       2.45s     10.32ms    56.00%
req/s           :      38.00       40.15       38.43        0.45    86.00%

undertow

Undertow with 15 workers 30 io threads

java -Xms512m -Xmx512m -Dserver.undertow.worker-threads=15 -Dserver.undertow.io-threads=30 -jar undertow.jar

Result

OOM - Did not complete

Undertwo with Default setup

java -Xms512m -Xmx512m -jar undertow.jar

Result

finished in 123.57s, 3237.14 req/s, 4.54MB/s
requests: 400000 total, 400000 started, 400000 done, 400000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 400000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 560.76MB (588002700) total, 12.97MB (13600600) headers (space savings 61.80%), 537.49MB (563600000) data
                     min         max         mean         sd        +/- sd
time for request:     8.93ms      40.33s    241.06ms       2.20s    98.81%
time for connect:   931.28ms       8.82s       4.11s       2.02s    70.00%
time to 1st byte:      1.50s      31.30s      16.31s       7.38s    70.00%
req/s           :      64.74      273.65       88.77       42.27    90.00%

Time Based Testing

50 Connections with 20 concurrent streams for 120 seconds after a warm up period of 5 seconds.

h2load -c50 -m20 --duration=120 --warm-up-time=5 https://localhost:8080/todo/search

jetty

java -Xms512m -Xmx512m -Dserver.jetty.max-threads=100  -Dserver.jetty.acceptors=50 -jar jetty.jar

finished in 125.12s, 3041.32 req/s, 4.19MB/s
requests: 364958 total, 364958 started, 364958 done, 364958 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 364922 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 503.02MB (527451789) total, 3.33MB (3491445) headers (space savings 89.45%), 499.47MB (523732345) data
                     min         max         mean         sd        +/- sd
time for request:      774us       1.57s    329.19ms    203.09ms    63.59%
time for connect:        0us         0us         0us         0us     0.00%
time to 1st byte:        0us         0us         0us         0us     0.00%
req/s           :      58.28       65.92       60.78        1.66    72.00%

netty

java -Xms512m -Xmx512m -jar reactor-netty.jar

Result

finished in 125.01s, 4567.83 req/s, 6.27MB/s
requests: 548140 total, 548660 started, 548140 done, 548140 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 548140 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 752.00MB (788528024) total, 1.58MB (1660280) headers (space savings 94.64%), 743.16MB (779261540) data
                     min         max         mean         sd        +/- sd
time for request:   135.55ms    747.36ms    214.46ms     71.15ms    90.92%
time for connect:   129.13ms      10.07s       4.50s       3.06s    53.85%
time to 1st byte:   470.84ms      10.62s       4.95s       3.14s    53.85%
req/s           :      75.67       98.49       91.35        5.87    62.00%

tomcat

java -Xms512m -Xmx512m -Dserver.tomcat.max-threads=50 -jar tomcat.jar

finished in 125.01s, 1960.42 req/s, 2.75MB/s
requests: 235251 total, 235251 started, 235251 done, 235251 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 235251 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 330.48MB (346534379) total, 8.37MB (8772153) headers (space savings 58.42%), 318.56MB (334030221) data
                     min         max         mean         sd        +/- sd
time for request:    81.72ms       2.32s    512.02ms    182.56ms    79.50%
time for connect:        0us         0us         0us         0us     0.00%
time to 1st byte:        0us         0us         0us         0us     0.00%
req/s           :      38.79       39.71       39.21        0.18    70.00%

undertow

java -Xms512m -Xmx512m -jar undertow.jar

finished in 125.01s, 2181.38 req/s, 3.06MB/s
requests: 261765 total, 262685 started, 261765 done, 261765 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 261765 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 366.97MB (384797034) total, 8.54MB (8959838) headers (space savings 61.80%), 354.08MB (371281363) data
                     min         max         mean         sd        +/- sd
time for request:     9.28ms      53.46s    252.67ms       2.42s    99.09%
time for connect:   711.92ms       2.56s       1.15s    748.47ms    78.26%
time to 1st byte:      9.56s      30.69s      20.88s       6.07s    63.04%
req/s           :      10.97      375.89       43.63       76.74    92.00%

Non-SSL using Best Performer from Above

time based: http1.1 with 50 connections and 20 pipelining requests

h2load --h1 -c50 -m20 --duration=120 --warm-up-time=5 http://localhost:8080/todo/search

java -Xms512m -Xmx512m -Dserver.http2.enabled=false -Dserver.ssl.enabled=false -jar reactor-netty.jar

finished in 125.00s, 2634.49 req/s, 3.72MB/s
requests: 316139 total, 316139 started, 316139 done, 316139 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 316139 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 446.81MB (468517998) total, 14.34MB (15033306) headers (space savings 0.00%), 439.14MB (460476699) data
                     min         max         mean         sd        +/- sd
time for request:    21.85ms      12.38s    384.53ms    316.74ms    98.20%
time for connect:        0us         0us         0us         0us     0.00%
time to 1st byte:        0us         0us         0us         0us     0.00%
req/s           :      48.76       55.17       52.69        2.02    54.00%

load http1.1 with 400k requests over 50 connections and 20 pipelining requests

h2load --h1 -n400000 -c50 -m20 http://localhost:8080/todo/search

finished in 169.68s, 2357.34 req/s, 3.33MB/s
requests: 400000 total, 400000 started, 400000 done, 400000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 400000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 565.34MB (592800000) total, 17.55MB (18400000) headers (space savings 0.00%), 537.49MB (563600000) data
                     min         max         mean         sd        +/- sd
time for request:    18.44ms      14.05s    404.07ms    372.73ms    98.26%
time for connect:      239us      2.88ms      1.62ms       740us    58.00%
time to 1st byte:   413.62ms      13.99s       5.92s       4.06s    58.00%
req/s           :      47.15       54.26       49.53        2.00    58.00%

400k requests with 100 connections and 10 pipelining requests

h2load --h1 -n400000 -c100 -m10 http://localhost:8080/todo/search

finished in 133.20s, 3003.10 req/s, 4.24MB/s
requests: 400000 total, 400000 started, 400000 done, 400000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 400000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 565.34MB (592800000) total, 17.55MB (18400000) headers (space savings 0.00%), 537.49MB (563600000) data
                     min         max         mean         sd        +/- sd
time for request:     3.27ms      11.50s    314.87ms    268.55ms    98.15%
time for connect:      458us      4.93ms      2.75ms      1.23ms    59.00%
time to 1st byte:     7.94ms      11.50s       3.77s       3.46s    64.00%
req/s           :      30.03       34.79       31.83        1.58    53.00%

400k requests with 100 connections

h2load --h1 -n400000 -c100 -m1 http://localhost:8080/todo/search

finished in 88.49s, 4520.20 req/s, 6.39MB/s
requests: 400000 total, 400000 started, 400000 done, 400000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 400000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 565.34MB (592800000) total, 17.55MB (18400000) headers (space savings 0.00%), 537.49MB (563600000) data
                     min         max         mean         sd        +/- sd
time for request:      446us       1.81s     21.95ms     16.06ms    93.52%
time for connect:      396us     19.62ms     11.20ms      5.94ms    75.00%
time to 1st byte:    22.45ms       1.81s    560.64ms    557.05ms    77.00%
req/s           :      45.21       45.94       45.56        0.18    71.00%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment