Skip to content

Instantly share code, notes, and snippets.

@matsumotory

matsumotory/environment

Last active August 29, 2015 13:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save matsumotory/9702123 to your computer and use it in GitHub Desktop.
Save matsumotory/9702123 to your computer and use it in GitHub Desktop.
Download ZIP
mruby-http2 no-tls benchmark. GitHub: https://github.com/matsumoto-r/mruby-http2
Raw
environment
(Fedora19 on VMWare)
$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 60
model name : Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
stepping : 3
microcode : 0x9
cpu MHz : 3498.029
cache size : 8192 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm ida arat epb xsaveopt pln pts dtherm fsgsbase smep
bogomips : 6996.05
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 60
model name : Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
stepping : 3
microcode : 0x9
cpu MHz : 3498.029
cache size : 8192 KB
physical id : 2
siblings : 1
core id : 0
cpu cores : 1
apicid : 2
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm ida arat epb xsaveopt pln pts dtherm fsgsbase smep
bogomips : 6996.05
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
$ free -m
total used free shared buffers cached
Mem: 7979 1063 6915 0 214 543
-/+ buffers/cache: 305 7673
Swap: 2079 0 2079
Raw
h2load_benchmark
$ h2load -n1000000 -c100 -m100 http://127.0.0.1:8080/index.html
starting benchmark...
spawning thread #0: 100 concurrent clients, 1000000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done
finished in 2 sec, 970 millisec and 170 microsec, 336681 req/s, 8220 kbytes/s
requests: 1000000 total, 1000000 started, 1000000 done, 1000000 succeeded, 0 failed, 0 errored
status codes: 1000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 49003700 bytes total, 1600 bytes headers, 25000000 bytes data
Raw
http2_server.rb
root_dir = "/usr/local/trusterd"
s = HTTP2::Server.new({
:port => 8080,
:key => "#{root_dir}/ssl/server.key",
:crt => "#{root_dir}/ssl/server.crt",
:document_root => "#{root_dir}/htdocs",
:server_name => "mruby-http2 server",
# optional default
# :debug => false
# :tls => true
# :daemon => false,
:tls => false,
})
s.run
# ./bin/mruby http2_server.rb
Raw
index.html
hello mruby-http2 world.
Raw
nghttp _verbose
$ nghttp -v http://127.0.0.1:8080/index.html
[ 0.001] send SETTINGS frame <length=10, flags=0x00, stream_id=0>
(niv=2)
[SETTINGS_MAX_CONCURRENT_STREAMS(3):100]
[SETTINGS_INITIAL_WINDOW_SIZE(4):65535]
[ 0.001] send HEADERS frame <length=48, flags=0x05, stream_id=1>
; END_STREAM | END_HEADERS
(padlen=0)
; Open new stream
:authority: 127.0.0.1:8080
:method: GET
:path: /index.html
:scheme: http
accept: */*
accept-encoding: gzip, deflate
user-agent: nghttp2/0.4.0-DEV
[ 0.002] recv SETTINGS frame <length=5, flags=0x00, stream_id=0>
(niv=1)
[SETTINGS_MAX_CONCURRENT_STREAMS(3):100]
[ 0.002] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
; ACK
(niv=0)
[ 0.003] (stream_id=1) :status: 200
[ 0.003] (stream_id=1) server: mruby-http2 server
[ 0.003] recv HEADERS frame <length=16, flags=0x04, stream_id=1>
; END_HEADERS
(padlen=0)
; First response header
hello mruby-http2 world.
[ 0.003] recv DATA frame <length=25, flags=0x00, stream_id=1>
[ 0.003] recv DATA frame <length=0, flags=0x01, stream_id=1>
; END_STREAM
[ 0.003] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
; ACK
(niv=0)
[ 0.003] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
(last_stream_id=0, error_code=NO_ERROR(0), opaque_data(0)=[])
@matsumotory
Copy link
Author

10並列でもう一回やってみた。236666 req/sでた。10並列だとこれくらい。

$ h2load -n100000 -c10 -m10 http://127.0.0.1:8080/index.html
starting benchmark...
spawning thread #0: 10 concurrent clients, 100000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 0 sec, 422 millisec and 535 microsec, 236666 req/s, 5778 kbytes/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 4900370 bytes total, 160 bytes headers, 2500000 bytes data

@matsumotory
Copy link
Author

同時接続数を10から100へ

$ h2load -n100000 -c100 -m100 http://127.0.0.1:8080/index.html
starting benchmark...
spawning thread #0: 100 concurrent clients, 100000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 0 sec, 336 millisec and 498 microsec, 297178 req/s, 7259 kbytes/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 4903700 bytes total, 1600 bytes headers, 2500000 bytes data

297178 req/sになった。さらに速い。

@matsumotory
Copy link
Author

ちなみに同環境でのnginx
conf

worker_processes  2;

events {
    worker_connections  1024;
    accept_mutex_delay 100ms;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile on;
    tcp_nopush on;
    access_log off;

    #gzip on;
    #gzip_comp_level 9;

    keepalive_requests 100000;
    keepalive_timeout 65;

    server {
        listen       81;
        server_name  localhost;

        location / {
            root   html;
            index  index.html index.htm;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

同時接続数100 + keepalive

$ weighttp -c 100 -n 100000 -k http://127.0.0.1:81/index.html
weighttp - a lightweight and simple webserver benchmarking tool

starting benchmark...
spawning thread #1: 100 concurrent requests, 100000 total requests
progress:  10% done
progress:  20% done
progress:  30% done
progress:  40% done
progress:  50% done
progress:  60% done
progress:  70% done
progress:  80% done
progress:  90% done
progress: 100% done

finished in 2 sec, 323 millisec and 572 microsec, 43037 req/s, 10969 kbyte/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 26100000 bytes total, 23600000 bytes http, 2500000 bytes data

同時接続数10 + keepalive

$ weighttp -c 10 -n 100000 -k http://127.0.0.1:81/index.html
weighttp - a lightweight and simple webserver benchmarking tool

starting benchmark...
spawning thread #1: 10 concurrent requests, 100000 total requests
progress:  10% done
progress:  20% done
progress:  30% done
progress:  40% done
progress:  50% done
progress:  60% done
progress:  70% done
progress:  80% done
progress:  90% done
progress: 100% done

finished in 2 sec, 334 millisec and 225 microsec, 42840 req/s, 10919 kbyte/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 26100000 bytes total, 23600000 bytes http, 2500000 bytes data

この条件だと圧倒的にmruby-http2速い。不安。

@matsumotory
Copy link
Author

総接続数100万同時接続数100でmruby-http2に対して再ベンチ

$ h2load -n1000000 -c100 -m100 http://127.0.0.1:8080/index.html
starting benchmark...
spawning thread #0: 100 concurrent clients, 1000000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3 sec, 15 millisec and 202 microsec, 331652 req/s, 8097 kbytes/s
requests: 1000000 total, 1000000 started, 1000000 done, 1000000 succeeded, 0 failed, 0 errored
status codes: 1000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 49003700 bytes total, 1600 bytes headers, 25000000 bytes data

30万超えちゃった。

@matsumotory
Copy link
Author

mruby-http2に対して総接続数1000万同時接続数100

$ h2load -n10000000 -c100 -m100 http://127.0.0.1:8080/index.html
starting benchmark...
spawning thread #0: 100 concurrent clients, 10000000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 30 sec, 466 millisec and 926 microsec, 328224 req/s, 8013 kbytes/s
requests: 10000000 total, 10000000 started, 10000000 done, 10000000 succeeded, 0 failed, 0 errored
status codes: 10000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 490003700 bytes total, 1600 bytes headers, 250000000 bytes data

@matsumotory
Copy link
Author

mruby-http2に対して総接続数1000万同時接続数500

$ h2load -n10000000 -c500 -m500 http://127.0.0.1:8080/index.html
starting benchmark...
spawning thread #0: 500 concurrent clients, 10000000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 31 sec, 291 millisec and 346 microsec, 319577 req/s, 7802 kbytes/s
requests: 10000000 total, 10000000 started, 10000000 done, 10000000 succeeded, 0 failed, 0 errored
status codes: 10000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 490018500 bytes total, 8000 bytes headers, 250000000 bytes data

@matsumotory
Copy link
Author 

$ h2load -n1000000 -c1000 -m1000 http://127.0.0.1:8080/index.html
starting benchmark...
spawning thread #0: 1000 concurrent clients, 1000000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3 sec, 523 millisec and 701 microsec, 283792 req/s, 6932 kbytes/s
requests: 1000000 total, 1000000 started, 1000000 done, 1000000 succeeded, 0 failed, 0 errored
status codes: 1000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 49037000 bytes total, 16000 bytes headers, 25000000 bytes data

@matsumotory
Copy link
Author

Max performance: 336681 req/s

$ h2load -n1000000 -c100 -m100 http://127.0.0.1:8080/index.html
starting benchmark...
spawning thread #0: 100 concurrent clients, 1000000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 2 sec, 970 millisec and 170 microsec, 336681 req/s, 8220 kbytes/s
requests: 1000000 total, 1000000 started, 1000000 done, 1000000 succeeded, 0 failed, 0 errored
status codes: 1000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 49003700 bytes total, 1600 bytes headers, 25000000 bytes data

@matsumotory
Copy link
Author

2014/03/24 commit: 9b702aeb345aa79e05f0088a9b5860ce08d54012

config

root_dir = "/usr/local/trusterd"

s = HTTP2::Server.new({
  :port           => 8080,
  :key            => "#{root_dir}/ssl/server.key",
  :crt            => "#{root_dir}/ssl/server.crt",
  :document_root  => "#{root_dir}/htdocs",
  :server_name    => "mruby-http2 server",

  # optional default
  # :debug => false
  # :tls => true
  #:debug => true,
  :tls => false,
  #:daemon => true,
  #:callback => true,
})

s.set_map_to_strage_cb {
  #p s.uri
  #p s.request.filename
  if s.request.uri == "/index.html"
    s.request.filename = "#{root_dir}/htdocs/hoge"
  end

  #p s.request.filename
}
s.run

result

$ h2load -n1000000 -c100 -m100 http://127.0.0.1:8080/index.html
starting benchmark...
spawning thread #0: 100 concurrent clients, 1000000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3 sec, 512 millisec and 701 microsec, 284681 req/s, 6950 kbytes/s
requests: 1000000 total, 1000000 started, 1000000 done, 1000000 succeeded, 0 failed, 0 errored
status codes: 1000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 49003700 bytes total, 1600 bytes headers, 25000000 bytes data

@matsumotory
Copy link
Author

2014/03/24 commit: 9b702aeb345aa79e05f0088a9b5860ce08d54012

config: enabled callback(location setting)

root_dir = "/usr/local/trusterd"

s = HTTP2::Server.new({
  :port           => 8080,
  :key            => "#{root_dir}/ssl/server.key",
  :crt            => "#{root_dir}/ssl/server.crt",
  :document_root  => "#{root_dir}/htdocs",
  :server_name    => "mruby-http2 server",

  # optional default
  # :debug => false
  # :tls => true
  #:debug => true,
  :tls => false,
  #:daemon => true,
  :callback => true,
})

s.set_map_to_strage_cb {

  if s.request.uri == "/index.html"
    s.request.filename = "#{root_dir}/htdocs/hoge"
  end

}
s.run

result

$ h2load -n1000000 -c100 -m100 http://127.0.0.1:8080/index.html
starting benchmark...
spawning thread #0: 100 concurrent clients, 1000000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 4 sec, 11 millisec and 118 microsec, 249307 req/s, 1217 kbytes/s
requests: 1000000 total, 1000000 started, 1000000 done, 1000000 succeeded, 0 failed, 0 errored
status codes: 1000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 29003700 bytes total, 1600 bytes headers, 5000000 bytes data

@sawanoboly
Copy link

VirtualBoxでTLSなしをやってみた。146232 req/s

index.htmlをtmpfsにしてみたらちょっと(+30,000)早くなった。

ENV

  • Ubuntu12.04 on virtualbox
# cat /proc/cpuinfo 
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 69
model name      : Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
stepping        : 1
microcode       : 0x19
cpu MHz         : 2762.728
cache size      : 6144 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc up rep_good nopl pni monitor ssse3 lahf_lm
bogomips        : 5525.45
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

build

  • build mruby with cookbook[mruby]

Before build

apt-get install autoconf automake autotools-dev libtool pkg-config zlib1g-dev libcunit1-dev libssl-dev libxml2-dev libevent-dev libjansson-dev libjemalloc-dev
mount -t tmpfs -o size=64m /dev/shm /var/tmp

Server

root_dir = "/var/tmp/www"  # tmpfs

s = HTTP2::Server.new({
  :port           => 8080,
  :document_root  => "#{root_dir}/htdocs",
  :server_name    => "mruby-http2 server",
  :tls => false,
})

s.run

Result

# ./h2load -n100000 -c10 -m10 http://127.0.0.1:8080/index.html
-t: warning: the number of threads is greater than hardware cores.
starting benchmark...
spawning thread #0: 10 concurrent clients, 100000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 0 sec, 683 millisec and 844 microsec, 146232 req/s, 6997 kbytes/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 4900370 bytes total, 160 bytes headers, 2500000 bytes data

@sawanoboly
Copy link

リクエスト数を10倍にしたらCPUを使いきっていたけど、これじゃクライアントの負荷込みになるな。

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----                                                                                            [4/25]
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 2  0  13848  11040   5640 169312    0    0     0     0  254 20729 40 34 25  0
 1  0  13848  11040   5640 169312    0    0     0     0  307 32957 48 52  0  0
 1  0  13848  11040   5640 169312    0    0     0     0  313 32056 51 49  0  0
 1  0  13848  10980   5640 169312    0    0     0     0  306 31870 45 55  0  0
 1  0  13848  11040   5640 169312    0    0     0     0  328 32145 49 51  0  0
 2  0  13848  11040   5640 169312    0    0     0     0  328 32418 52 48  0  0
 0  0  13848  11888   5640 169312    0    0     0     0  206 19315 31 29 41  0

@sawanoboly
Copy link

VirtualBoxのホストオンリーネットワーク越しにテスト。

ENV

  • Server: Ubuntu12.04 on virtualbox (10.33.35.201)
  • Client: Ubuntu12.04 on virtualbox (10.33.35.202)

Result

c# ./h2load -n1000000 -c10 -m10 http://10.33.35.201:8080/index.html
-t: warning: the number of threads is greater than hardware cores.
starting benchmark...
spawning thread #0: 10 concurrent clients, 1000000 total requests
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 14 sec, 274 millisec and 632 microsec, 70054 req/s, 3352 kbytes/s
requests: 1000000 total, 1000000 started, 1000000 done, 1000000 succeeded, 0 failed, 0 errored
status codes: 1000000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 49000370 bytes total, 160 bytes headers, 25000000 bytes data

VMstat

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 2  0      0 102428  43508  38640    0    0   183    11  383   88  1  3 96  0
 0  0      0 102368  43508  38640    0    0     0     0   19   47  0  0 100  0
 1  0      0 102368  43508  38640    0    0     0     0   79   74  0  0 100  0
 1  0      0 102248  43508  38640    0    0     0     0 12433  295 16 83  1  0
 1  0      0 102248  43508  38640    0    0     0     0 12345  361 18 81  1  0
 1  0      0 102308  43508  38640    0    0     0     0 11123  345 18 77  5  0
 2  0      0 102248  43508  38640    0    0     0     0 12247  336 21 79  0  0
 1  0      0 102248  43508  38640    0    0     0     0 12572  327 16 83  1  0
 1  0      0 102248  43508  38640    0    0     0     0 12872  344 17 83  0  0
 1  0      0 102308  43508  38640    0    0     0     0 12742  388 16 84  0  0
 1  0      0 102248  43508  38640    0    0     0     0 13163  305 14 85  1  0
 1  0      0 102308  43508  38640    0    0     0     0 12845  269 14 86  0  0
 1  0      0 102248  43508  38640    0    0     0     0 12985  321 17 83  0  0
 1  0      0 102248  43508  38640    0    0     0     0 12729  339 18 81  1  0
 1  0      0 102248  43508  38640    0    0     0     0 12794  285 15 85  0  0
 1  0      0 102248  43508  38640    0    0     0     0 12049  465 14 86  0  0
 1  0      0 102308  43508  38640    0    0     0     0 13684  357 18 81  1  0
 0  0      0 102248  43508  38640    0    0     0     0 2248  278  4 18 78  0
 0  0      0 102248  43508  38640    0    0     0     0   16   36  0  0 100  0

@matsumotory
Copy link
Author

ありがとうございます！nginxとか比較対象（厳密にはプロトコルが違うので比較にならないが）があると目安になってわかりやすそうですね。

@matsumotory
Copy link
Author

誘導... matsumotory/mruby-http2#1

日本語でもOKです。ここだと通知とか無くて気付けなさそうですし。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment