Skip to content

Instantly share code, notes, and snippets.

@brydavis

brydavis/demo.py

Last active August 10, 2019 17:43
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save brydavis/0af699668d59d21e78fcf0b5d2f934a6 to your computer and use it in GitHub Desktop.
Save brydavis/0af699668d59d21e78fcf0b5d2f934a6 to your computer and use it in GitHub Desktop.
Download ZIP
UWPCE - Lesson 06
Raw
demo.py
import time
import math
# from timeit import timeit
# print(timeit('[x for x in range(100)]', number=1000))
# print(timeit('[x for x in range(100)]', number=1000))
# print(timeit(lambda: [x for x in range(100)], number=1000))
def do_math():
for n in range(1000, 1000000):
math.sqrt(n)
def take_break():
# baseline overhead
time.sleep(2)
def do_work():
for n in range(1000,10000):
# [x for x in range(n)]
list(range(n))
if __name__ == "__main__":
do_math()
take_break()
do_work()
take_break()
Display the source blob
Display the rendered blob
Raw
notes-lesson06.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Profiling and Performance\n",
"\n",
"What is profiling?\n",
"Capturing metadata about how a program behaves\n",
"Mostly around time and compute utilization\n",
"\n",
"Why profile?\n",
"Understand how fast a program or parts of a program run, so you can identify if it can run faster\n",
"\n",
"When is a good time to optimize?\n",
"https://xkcd.com/1205/\n",
"\n",
"\n",
"https://superuser.com/questions/228056/windows-equivalent-to-unix-time-command\n",
"\n",
"https://imgs.xkcd.com/comics/is_it_worth_the_time_2x.png \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"\n",
"## time\n",
"\n",
" $ time python demo.py\n",
"\n",
" real 0.00\n",
" user 0.00\n",
" sys 0.00\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## gnu-time\n",
"\n",
"\n",
" $ brew install gnu-time\n",
" ...\n",
"\n",
" $ gtime --verbose python demo.py\n",
"\n",
"\tCommand being timed: \"find /Users/bryan/Documents/\"\n",
"\tUser time (seconds): 0.00\n",
"\tSystem time (seconds): 0.00\n",
"\tPercent of CPU this job got: 75%\n",
"\tElapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00\n",
"\tAverage shared text size (kbytes): 0\n",
"\tAverage unshared data size (kbytes): 0\n",
"\tAverage stack size (kbytes): 0\n",
"\tAverage total size (kbytes): 0\n",
"\tMaximum resident set size (kbytes): 852\n",
"\tAverage resident set size (kbytes): 0\n",
"\tMajor (requiring I/O) page faults: 0\n",
"\tMinor (reclaiming a frame) page faults: 339\n",
"\tVoluntary context switches: 1\n",
"\tInvoluntary context switches: 2\n",
"\tSwaps: 0\n",
"\tFile system inputs: 0\n",
"\tFile system outputs: 0\n",
"\tSocket messages sent: 0\n",
"\tSocket messages received: 0\n",
"\tSignals delivered: 0\n",
"\tPage size (bytes): 4096\n",
"\tExit status: 0\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## timeit\n",
"https://docs.python.org/3.7/library/timeit.html\n",
"\n",
"\n",
" $ python -m timeit '\"-\".join(str(n) for n in range(100))'\n",
"\n",
" # add timeit to your code\n",
" # use at line level and run program\n",
" \n",
" $ python demo.py\n",
" \n",
" \n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## cProfile\n",
"\n",
" $ python -m cProfile demo.py\n",
" \n",
" 999224 function calls in 4.969 seconds\n",
"\n",
" Ordered by: standard name\n",
"\n",
" ncalls tottime percall cumtime percall filename:lineno(function)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:103(release)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:143(__init__)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:147(__enter__)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:151(__exit__)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:157(_get_module_lock)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:176(cb)\n",
" 2 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:211(_call_with_frames_removed)\n",
" 19 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:222(_verbose_message)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:307(__init__)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:311(__enter__)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:318(__exit__)\n",
" 4 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:321(<genexpr>)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:369(__init__)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:403(cached)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:416(parent)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:424(has_location)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:504(_init_module_attrs)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:576(module_from_spec)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:58(__init__)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:663(_load_unlocked)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:719(find_spec)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:78(acquire)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:792(find_spec)\n",
" 3 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:855(__enter__)\n",
" 3 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:859(__exit__)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:882(_find_spec)\n",
" 1 0.000 0.000 0.001 0.001 <frozen importlib._bootstrap>:948(_find_and_load_unlocked)\n",
" 1 0.000 0.000 0.001 0.001 <frozen importlib._bootstrap>:978(_find_and_load)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:1029(__init__)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:1040(create_module)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:1048(exec_module)\n",
" 5 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:1203(_path_importer_cache)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:1240(_get_spec)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:1272(find_spec)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:1351(_get_spec)\n",
" 4 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:1356(find_spec)\n",
" 4 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:36(_relax_case)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:369(_get_cached)\n",
" 16 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:56(_path_join)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:574(spec_from_file_location)\n",
" 16 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:58(<listcomp>)\n",
" 5 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:74(_path_stat)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:84(_path_is_mode_type)\n",
" 1 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap_external>:93(_path_isfile)\n",
" 1 0.000 0.000 4.969 4.969 demo.py:1(<module>)\n",
" 1 0.122 0.122 0.193 0.193 demo.py:10(do_math)\n",
" 2 0.000 0.000 4.002 2.001 demo.py:14(take_break)\n",
" 1 0.773 0.773 0.773 0.773 demo.py:18(do_work)\n",
" 5 0.000 0.000 0.000 0.000 {built-in method _imp.acquire_lock}\n",
" 1 0.000 0.000 0.000 0.000 {built-in method _imp.create_dynamic}\n",
" 1 0.000 0.000 0.000 0.000 {built-in method _imp.exec_dynamic}\n",
" 1 0.000 0.000 0.000 0.000 {built-in method _imp.is_builtin}\n",
" 1 0.000 0.000 0.000 0.000 {built-in method _imp.is_frozen}\n",
" 5 0.000 0.000 0.000 0.000 {built-in method _imp.release_lock}\n",
" 2 0.000 0.000 0.000 0.000 {built-in method _thread.allocate_lock}\n",
" 2 0.000 0.000 0.000 0.000 {built-in method _thread.get_ident}\n",
" 1 0.000 0.000 0.000 0.000 {built-in method builtins.any}\n",
" 1 0.000 0.000 4.969 4.969 {built-in method builtins.exec}\n",
" 6 0.000 0.000 0.000 0.000 {built-in method builtins.getattr}\n",
" 6 0.000 0.000 0.000 0.000 {built-in method builtins.hasattr}\n",
" 5 0.000 0.000 0.000 0.000 {built-in method builtins.isinstance}\n",
" 999000 0.071 0.000 0.071 0.000 {built-in method math.sqrt}\n",
" 1 0.000 0.000 0.000 0.000 {built-in method posix.fspath}\n",
" 1 0.000 0.000 0.000 0.000 {built-in method posix.getcwd}\n",
" 5 0.000 0.000 0.000 0.000 {built-in method posix.stat}\n",
" 2 4.002 2.001 4.002 2.001 {built-in method time.sleep}\n",
" 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n",
" 2 0.000 0.000 0.000 0.000 {method 'endswith' of 'str' objects}\n",
" 2 0.000 0.000 0.000 0.000 {method 'get' of 'dict' objects}\n",
" 16 0.000 0.000 0.000 0.000 {method 'join' of 'str' objects}\n",
" 6 0.000 0.000 0.000 0.000 {method 'rpartition' of 'str' objects}\n",
" 32 0.000 0.000 0.000 0.000 {method 'rstrip' of 'str' objects}\n",
" \n",
"\n",
"\n",
"##### ncalls\n",
"for the number of calls,\n",
"\n",
"##### tottime\n",
"\n",
"for the total time spent in the given function (and excluding time made in calls to sub-functions)\n",
"\n",
"##### percall\n",
"\n",
"is the quotient of tottime divided by ncalls\n",
"\n",
"##### cumtime\n",
"\n",
"is the cumulative time spent in this and all subfunctions (from invocation till exit). This figure is accurate even for recursive functions.\n",
"\n",
"##### percall\n",
"\n",
"is the quotient of cumtime divided by primitive calls\n",
"\n",
"##### filename:lineno(function)\n",
"\n",
"provides the respective data of each function"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br><br>\n",
"## PyPy\n",
"\n",
"https://pypy.org/download.html\n",
"\n",
"Drop in replacement for Python.\n",
"\n",
"Usually a couple releases behind standard Python (aka CPython).\n",
"So, compatibility is main issue.\n",
"\n",
" time ~/Downloads/pypy3.6/bin/pypy3 demo.py \n",
"\n",
"\n",
"## Cython\n",
"\n",
"https://www.infoworld.com/article/3250299/what-is-cython-python-at-the-speed-of-c.html\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment