Skip to content

Instantly share code, notes, and snippets.

@gubatron
Created August 17, 2014 01:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gubatron/24c798a38360e6127f77 to your computer and use it in GitHub Desktop.
Save gubatron/24c798a38360e6127f77 to your computer and use it in GitHub Desktop.
Testing the cost of invoking ioloop.instance()
#My theory is that if ioloop.instance() is to be called for every request the websocket might receive
#this is costly as the instance() method has a dictionary lookup in there.
#Plus there's also the cost of allocating memory for the stack of the instance() method and then returning back to
#the method which invokes it, vs just keeping the reference once and reusing it.
#So here's some test code
import cProfile
from zmq.eventloop import ioloop
ioloop.install()
def testingWithIndirectCalls():
i=100000
while i > 0:
i-=1
loop = ioloop.IOLoop.instance()
loop.current()
def testingWithReferenceCalls(loop):
i=100000
while i > 0:
i-=1
loop.current()
if __name__ == '__main__':
print "Test with a call to IOLoop.instance() everytime"
cProfile.run('testingWithIndirectCalls()')
print "\n\n"
print "="*100
print "\n\n"
#vs
print "Test keeping the reference to IOLoop.instance() and reusing the reference"
cProfile.run('testingWithReferenceCalls(ioloop.IOLoop.instance())')
'''Test Results:
Test with a call to IOLoop.instance() everytime
1100044 function calls in 0.678 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.678 0.678 <string>:1(<module>)
1 0.000 0.000 0.000 0.000 ioloop.py:108(register)
200000 0.075 0.000 0.104 0.000 ioloop.py:130(instance)
1 0.000 0.000 0.000 0.000 ioloop.py:131(initialize)
100000 0.104 0.000 0.386 0.000 ioloop.py:135(instance)
100000 0.074 0.000 0.203 0.000 ioloop.py:169(current)
100002 0.013 0.000 0.013 0.000 ioloop.py:210(configurable_base)
1 0.000 0.000 0.000 0.000 ioloop.py:226(initialize)
1 0.000 0.000 0.000 0.000 ioloop.py:587(split_fd)
1 0.000 0.000 0.000 0.000 ioloop.py:636(initialize)
1 0.000 0.000 0.000 0.000 ioloop.py:674(add_handler)
1 0.000 0.000 0.000 0.000 ioloop.py:81(__init__)
1 0.000 0.000 0.000 0.000 ioloop.py:84(_map_events)
1 0.000 0.000 0.000 0.000 poll.py:21(__init__)
1 0.000 0.000 0.000 0.000 poll.py:28(register)
2 0.000 0.000 0.000 0.000 posix.py:27(set_close_exec)
2 0.000 0.000 0.000 0.000 posix.py:32(_set_nonblocking)
1 0.000 0.000 0.000 0.000 posix.py:38(__init__)
1 0.000 0.000 0.000 0.000 posix.py:47(fileno)
1 0.000 0.000 0.000 0.000 stack_context.py:253(wrap)
1 0.089 0.089 0.678 0.678 stack_test.py:6(testingWithIndirectCalls)
1 0.000 0.000 0.000 0.000 util.py:194(__new__)
100000 0.154 0.000 0.228 0.000 util.py:231(configure)
1 0.000 0.000 0.000 0.000 util.py:247(configured_class)
1 0.000 0.000 0.000 0.000 {built-in method __new__ of type object at 0x10d7c27c8}
8 0.000 0.000 0.000 0.000 {fcntl.fcntl}
100000 0.079 0.000 0.079 0.000 {getattr}
200003 0.028 0.000 0.028 0.000 {hasattr}
100000 0.040 0.000 0.040 0.000 {isinstance}
100000 0.021 0.000 0.021 0.000 {issubclass}
1 0.000 0.000 0.000 0.000 {len}
1 0.000 0.000 0.000 0.000 {method 'append' of 'list' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
1 0.000 0.000 0.000 0.000 {method 'fileno' of 'file' objects}
1 0.000 0.000 0.000 0.000 {method 'update' of 'dict' objects}
2 0.000 0.000 0.000 0.000 {posix.fdopen}
1 0.000 0.000 0.000 0.000 {posix.pipe}
1 0.000 0.000 0.000 0.000 {thread.allocate_lock}
====================================================================================================
Test keeping the reference to IOLoop.instance() and reusing the reference
400010 function calls in 0.220 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.220 0.220 <string>:1(<module>)
100001 0.036 0.000 0.047 0.000 ioloop.py:130(instance)
1 0.000 0.000 0.000 0.000 ioloop.py:135(instance)
100000 0.061 0.000 0.181 0.000 ioloop.py:169(current)
1 0.000 0.000 0.000 0.000 ioloop.py:210(configurable_base)
1 0.039 0.039 0.220 0.220 stack_test.py:13(testingWithReferenceCalls)
1 0.000 0.000 0.000 0.000 util.py:231(configure)
100000 0.073 0.000 0.073 0.000 {getattr}
100001 0.011 0.000 0.011 0.000 {hasattr}
1 0.000 0.000 0.000 0.000 {isinstance}
1 0.000 0.000 0.000 0.000 {issubclass}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
'''
@gubatron
Copy link
Author

Making that call to Ioloop.instance() is indeed very expensive, there's a dictionary lookup in there when they check for the singleton instance...

The result is a 3x speedup, python does only 400,010 function calls when keeping the reference, vs 1,100,044 function calls when calling instance() everytime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment