Skip to content

Instantly share code, notes, and snippets.

@bigeagle
Created August 29, 2013 01:18
Show Gist options
  • Save bigeagle/6373276 to your computer and use it in GitHub Desktop.
Save bigeagle/6373276 to your computer and use it in GitHub Desktop.
#!/usr/bin/env python2
# -*- coding:utf-8 -*-
import tornado.ioloop
import tornado.web
import tornado.gen
import motor
import json
class PersistantHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
def get(self):
coll = self.settings['_pdb'].test_coll
coll.find().to_list(30, callback=self.on_fetch)
def on_fetch(self, records, err):
for r in records:
self.write(json.dumps(
{k: v for k, v in r.iteritems() if k != '_id'}))
self.write('<br />')
self.finish()
class GenPersistantHandler(tornado.web.RequestHandler):
@tornado.gen.coroutine
def get(self):
coll = self.settings['_pdb'].test_coll
cursor = coll.find()
for r in (yield motor.Op(cursor.to_list, 30)):
self.write(json.dumps(
{k: v for k, v in r.iteritems() if k != '_id'}))
self.write('<br />')
self.finish()
class SyncRequestHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
def get(self):
self.conn = motor.MotorClient().open_sync()
db = self.conn['tornado-test']
coll = db.test_coll
coll.find().to_list(30, callback=self.on_fetch)
def on_fetch(self, records, err):
for r in records:
self.write(json.dumps(
{k: v for k, v in r.iteritems() if k != '_id'}))
self.write('<br />')
self.finish()
self.conn.close()
class AsyncRequestHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
def get(self):
motor.MotorClient().open(self.on_open)
def on_open(self, conn, err):
self.conn = conn
db = self.conn['tornado-test']
coll = db.test_coll
coll.find().to_list(30, callback=self.on_fetch)
def on_fetch(self, records, err):
for r in records:
self.write(json.dumps(
{k: v for k, v in r.iteritems() if k != '_id'}))
self.write('<br />')
self.finish()
self.conn.close()
class GenAsyncRequestHandler(tornado.web.RequestHandler):
@tornado.gen.coroutine
def get(self):
_ = yield tornado.gen.Task(motor.MotorClient().open, )
self.conn, err = _.args
db = self.conn['tornado-test']
coll = db.test_coll
cursor = coll.find()
for r in (yield motor.Op(cursor.to_list, 30)):
self.write(json.dumps(
{k: v for k, v in r.iteritems() if k != '_id'}))
self.write('<br />')
self.conn.close()
self.finish()
if __name__ == "__main__":
db = motor.MotorClient().open_sync()['tornado-test']
application = tornado.web.Application([
(r"/p", PersistantHandler),
(r"/gp", GenPersistantHandler),
(r"/s", SyncRequestHandler),
(r"/a", AsyncRequestHandler),
(r"/ga", GenAsyncRequestHandler),
], _pdb=db, debug=True)
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
# vim: ts=4 sw=4 sts=4 expandtab
@bigeagle
Copy link
Author

Test persistent client performance

ab -n 1000 -c 200 http://localhost:8888/p

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        TornadoServer/3.1
Server Hostname:        localhost
Server Port:            8888

Document Path:          /p
Document Length:        1372 bytes

Concurrency Level:      200
Time taken for tests:   2.396 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      1567000 bytes
HTML transferred:       1372000 bytes
Requests per second:    417.42 [#/sec] (mean)
Time per request:       479.133 [ms] (mean)
Time per request:       2.396 [ms] (mean, across all concurrent requests)
Transfer rate:          638.77 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   15 113.4      0    1002
Processing:   208  436  72.1    457     597
Waiting:      208  436  72.1    457     597
Total:        217  451 135.5    460    1535

Percentage of the requests served within a certain time (ms)
  50%    460
  66%    476
  75%    483
  80%    492
  90%    502
  95%    541
  98%    591
  99%   1444
 100%   1535 (longest request)

Test one-asynchronous-open-client-per-request performance

ab -n 1000 -c 200 http://localhost:8888/a
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        TornadoServer/3.1
Server Hostname:        localhost
Server Port:            8888

Document Path:          /a
Document Length:        1372 bytes

Concurrency Level:      200
Time taken for tests:   3.702 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      1567000 bytes
HTML transferred:       1372000 bytes
Requests per second:    270.15 [#/sec] (mean)
Time per request:       740.335 [ms] (mean)
Time per request:       3.702 [ms] (mean, across all concurrent requests)
Transfer rate:          413.40 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    4   7.6      0      22
Processing:   412  709  92.2    722     847
Waiting:      412  709  92.2    721     847
Total:        431  713  86.0    722     847

Percentage of the requests served within a certain time (ms)
  50%    722
  66%    760
  75%    763
  80%    765
  90%    801
  95%    839
  98%    843
  99%    845
 100%    847 (longest request)

test Test one-block-open-client-per-request performance

ab -n 1000 -c 200 http://localhost:8888/s
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        TornadoServer/3.1
Server Hostname:        localhost
Server Port:            8888

Document Path:          /s
Document Length:        1372 bytes

Concurrency Level:      200
Time taken for tests:   5.089 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      1567000 bytes
HTML transferred:       1372000 bytes
Requests per second:    196.50 [#/sec] (mean)
Time per request:       1017.797 [ms] (mean)
Time per request:       5.089 [ms] (mean, across all concurrent requests)
Transfer rate:          300.70 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   68 244.4      0    1004
Processing:   465  888 218.7    858    1354
Waiting:      465  888 218.7    858    1354
Total:        465  956 276.0    866    1665

Percentage of the requests served within a certain time (ms)
  50%    866
  66%   1031
  75%   1099
  80%   1235
  90%   1252
  95%   1656
  98%   1659
  99%   1660
 100%   1665 (longest request)

Obviously global motorClient has the best performance

@bigeagle
Copy link
Author

Performance with concurrency of 20:

ab -n 1000 -c 20 http://localhost:8888/p
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        TornadoServer/3.1
Server Hostname:        localhost
Server Port:            8888

Document Path:          /p
Document Length:        1372 bytes

Concurrency Level:      20
Time taken for tests:   2.189 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      1567000 bytes
HTML transferred:       1372000 bytes
Requests per second:    456.92 [#/sec] (mean)
Time per request:       43.772 [ms] (mean)
Time per request:       2.189 [ms] (mean, across all concurrent requests)
Transfer rate:          699.21 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.7      0       7
Processing:     3   43   7.1     41      72
Waiting:        1   43   7.1     41      72
Total:          8   43   7.0     41      72

Percentage of the requests served within a certain time (ms)
  50%     41
  66%     47
  75%     50
  80%     51
  90%     53
  95%     55
  98%     58
  99%     60
 100%     72 (longest request)

Concurrency 200:

ab -n 1000 -c 200 http://localhost:8888/p
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        TornadoServer/3.1
Server Hostname:        localhost
Server Port:            8888

Document Path:          /p
Document Length:        1372 bytes

Concurrency Level:      200
Time taken for tests:   2.390 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      1567000 bytes
HTML transferred:       1372000 bytes
Requests per second:    418.39 [#/sec] (mean)
Time per request:       478.020 [ms] (mean)
Time per request:       2.390 [ms] (mean, across all concurrent requests)
Transfer rate:          640.26 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   4.3      0      14
Processing:   214  445  64.3    459     600
Waiting:      214  445  64.3    458     600
Total:        226  447  63.5    459     600

Percentage of the requests served within a certain time (ms)
  50%    459
  66%    475
  75%    479
  80%    480
  90%    499
  95%    538
  98%    567
  99%    575
 100%    600 (longest request)

concurrency 1000:

ab -n 1000 -c 1000 http://localhost:8888/p
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        TornadoServer/3.1
Server Hostname:        localhost
Server Port:            8888

Document Path:          /p
Document Length:        1372 bytes

Concurrency Level:      1000
Time taken for tests:   2.437 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      1567000 bytes
HTML transferred:       1372000 bytes
Requests per second:    410.31 [#/sec] (mean)
Time per request:       2437.197 [ms] (mean)
Time per request:       2.437 [ms] (mean, across all concurrent requests)
Transfer rate:          627.88 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0  166 357.8     14    1005
Processing:   277  893 239.3    947    1264
Waiting:      277  893 239.4    947    1264
Total:        297 1059 477.4    975    2269

Percentage of the requests served within a certain time (ms)
  50%    975
  66%   1047
  75%   1084
  80%   1208
  90%   2012
  95%   2070
  98%   2238
  99%   2253
 100%   2269 (longest request)

time per request does not increase much with the increase of concurrency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment