SugarCRM on HHVM Benchmarking

benchmark.p

# output as png image
set terminal png size 1280,800

set output "benchmark.png"
set title "SugarCRM Benchmarking"

# y-axis grid
set grid y

# x-axis label
set xlabel "request"

# y-axis label
set ylabel "response time (ms)"

# plot data from data files
plot \
"mod_php.csv" using 9 smooth sbezier with lines title "Apache 2.4.6 + mod_php" lw 4, \
"php5_fpm.csv" using 9 smooth sbezier with lines title "nginx 1.4.1 + php5-fpm" lw 4, \
"php.csv" using 9 smooth sbezier with lines title "PHP 5.5 built-in server" lw 4, \
"hhvm.csv" using 9 smooth sbezier with lines title "HHVM 2.2.0" lw 4

benchmark.png

benchmark.sh

ab \
  -n 1000 \
  -c 10 \
  -g output.csv \
  -C PHPSESSID=2h0rrktahtmb6hk5l8ke1gnaf2 \
  http://sugarcrm.local/?module=Accounts \
  > output.txt

# do the above for all configurations

gnuplot benchmark.p

hhvm.txt

This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking sugarcrm.local (be patient)


Server Software:        
Server Hostname:        sugarcrm.local
Server Port:            8080

Document Path:          /?module=Accounts
Document Length:        249353 bytes

Concurrency Level:      10
Time taken for tests:   26.570 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      249888000 bytes
HTML transferred:       249353000 bytes
Requests per second:    37.64 [#/sec] (mean)
Time per request:       265.700 [ms] (mean)
Time per request:       26.570 [ms] (mean, across all concurrent requests)
Transfer rate:          9184.47 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   111  265 118.0    235     925
Waiting:      111  264 118.0    234     924
Total:        112  265 118.0    235     925

Percentage of the requests served within a certain time (ms)
  50%    235
  66%    263
  75%    284
  80%    309
  90%    400
  95%    503
  98%    611
  99%    800
 100%    925 (longest request)

mod_php.txt

This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking sugarcrm.local (be patient)


Server Software:        Apache/2.4.6
Server Hostname:        sugarcrm.local
Server Port:            80

Document Path:          /?module=Accounts
Document Length:        249353 bytes

Concurrency Level:      10
Time taken for tests:   148.671 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      250047000 bytes
HTML transferred:       249353000 bytes
Requests per second:    6.73 [#/sec] (mean)
Time per request:       1486.711 [ms] (mean)
Time per request:       148.671 [ms] (mean, across all concurrent requests)
Transfer rate:          1642.46 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.5      0       5
Processing:   247 1480 949.7   1304    6832
Waiting:      234 1465 949.8   1289    6818
Total:        247 1480 949.7   1304    6832

Percentage of the requests served within a certain time (ms)
  50%   1304
  66%   1681
  75%   1933
  80%   2122
  90%   2751
  95%   3468
  98%   4135
  99%   4490
 100%   6832 (longest request)

php.txt

This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking sugarcrm.local (be patient)


Server Software:        
Server Hostname:        sugarcrm.local
Server Port:            8282

Document Path:          /?module=Accounts
Document Length:        249353 bytes

Concurrency Level:      10
Time taken for tests:   248.616 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      249988000 bytes
HTML transferred:       249353000 bytes
Requests per second:    4.02 [#/sec] (mean)
Time per request:       2486.155 [ms] (mean)
Time per request:       248.616 [ms] (mean, across all concurrent requests)
Transfer rate:          981.95 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:   243 2475 162.2   2471    2957
Waiting:      233 2461 161.7   2457    2946
Total:        244 2475 162.1   2471    2957

Percentage of the requests served within a certain time (ms)
  50%   2471
  66%   2506
  75%   2530
  80%   2550
  90%   2628
  95%   2697
  98%   2773
  99%   2808
 100%   2957 (longest request)

php5_fpm.txt

This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking sugarcrm.local (be patient)


Server Software:        nginx/1.4.1
Server Hostname:        sugarcrm.local
Server Port:            8181

Document Path:          /?module=Accounts
Document Length:        249353 bytes

Concurrency Level:      10
Time taken for tests:   153.489 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      250028000 bytes
HTML transferred:       249353000 bytes
Requests per second:    6.52 [#/sec] (mean)
Time per request:       1534.890 [ms] (mean)
Time per request:       153.489 [ms] (mean, across all concurrent requests)
Transfer rate:          1590.79 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       4
Processing:   280 1528 461.1   1434    3764
Waiting:      267 1513 461.1   1417    3752
Total:        283 1528 461.1   1434    3764

Percentage of the requests served within a certain time (ms)
  50%   1434
  66%   1615
  75%   1745
  80%   1873
  90%   2160
  95%   2438
  98%   2821
  99%   3000
 100%   3764 (longest request)