profiling.md

Profiling!

A simple way to get line profiling on your python functions.

1. pip install line_profiler
2. Decorate your function with @profile (no import needed)
3. Launch a shell with kernprof -l manage.py shell
4. Call the function you want to profile. Preferably several times! (%timeit is great for running it several times)
5. Exit the shell. You'll see a dumped file created called manage.py.lprof
6. Load the data with python -m line_profiler manage.py.lprof
7. Explore the results!

Here's an example of the results you would get:

Total time: 0.441951 s
File: /rpc.py
Function: get_user_information at line 342

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   342                                           @profile
   343                                           def get_user_information(user):
... (docstring redacted)
   357        82          798      9.7      0.2      profile = user.profile
   358        82        80686    984.0     18.3      groups = user.groups.all().values_list('name', flat=True)
   359        82        69605    848.8     15.7      labs = (LABS_GROUP_NAME in groups)
   360
   361        82         1713     20.9      0.4      default_business = profile.get_default_business()
   362
   363        82          160      2.0      0.0      return {
   364        82          338      4.1      0.1          'username': user.username,
   365        82          119      1.5      0.0          'email': user.email,
   366        82          104      1.3      0.0          'first_name': user.first_name,
   367        82          162      2.0      0.0          'last_name': user.last_name,
   368        82          199      2.4      0.0          'is_active': bool(user.is_active),
   369        82          116      1.4      0.0          'is_superuser': bool(user.is_superuser),
   370        82          123      1.5      0.0          'is_staff': bool(user.is_staff),
   371        82          206      2.5      0.0          'last_login': user.last_login,
   372        82       159332   1943.1     36.1          'has_social_auth': user.social_auth.exists(),
   373        82       126393   1541.4     28.6          'is_origin_user': bool(user.is_origin_user()),
   374        82         1622     19.8      0.4          'is_not_origin_user': !bool(user.is_origin_user()),
   375        82           92      1.1      0.0          'labs': labs,
   376        82           94      1.1      0.0          'groups': groups,
   377        82           89      1.1      0.0          'default_identity_business_id': default_business.identity_business_id if default_business else None
   378                                               }

Basically the line_profiler adds a tool to your environment called kernprof which will wrap the python executable and add profiling to any thing that you decorate with the @profile decorator. It's worth noting though that you won't be able to run your modules containing @profile except when run through kernprof, since that decorator doesn't exist by default.

Here's a quick and dirty hack that would allow you to use @profile with or without kernprof, definitely don't commit this though ;)

try:
    profile
except NameError:
    profile = lamda x: x

For more details, check out https://github.com/rkern/line_profiler/