Skip to content

Instantly share code, notes, and snippets.

@anandology

anandology/day1.ipynb

Last active August 29, 2015 13:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save anandology/9223073 to your computer and use it in GitHub Desktop.
Save anandology/9223073 to your computer and use it in GitHub Desktop.
Download ZIP
Advanced Python Training at LinkedIn -- Feb 26 - Mar 1, 2014
Display the source blob
Display the rendered blob
Raw
day1.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
day2.ipynb
{
"metadata": {
"name": ""
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Advanced Python Training - Day 2\n",
"Linked In<br/>\n",
"Feb 26 - Mar 1, 2014<br/>\n",
"<a href=\"http://anandology.com\">Anand Chitipothu</a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"My solutions to the Flask projects that we will be working during this course will be available at <https://bitbucket.org/anandology/flask-training.git>\n",
"\n",
" git clone https://bitbucket.org/anandology/flask-training.git\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Decorators (Continued..)"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file trace2.py\n",
"import functools\n",
"import os\n",
"\n",
"level = 0\n",
"def trace(f):\n",
" # Don't decorate if the DEBUG is not set in the environ\n",
" if not os.environ.get('DEBUG'):\n",
" return f\n",
"\n",
" @functools.wraps(f)\n",
" def g(*args):\n",
" global level\n",
" \n",
" # print only if DEBUG is enabled in the environment\n",
" #if os.environ.get('DEBUG'):\n",
" # print \"| \" * level + \"|-- \" + f.__name__, args\n",
" level += 1\n",
" value = f(*args)\n",
" level -= 1 \n",
" return value\n",
" return g"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting trace2.py\n"
]
}
],
"prompt_number": 9
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file squares2.py\n",
"from trace2 import trace\n",
"\n",
"@trace\n",
"def square(x):\n",
" \"\"\"Computes square of a number.\"\"\"\n",
" return x*x\n",
"\n",
"@trace\n",
"def sum_of_squares(x, y):\n",
" return square(x)+square(y)\n",
"\n",
"if __name__ == \"__main__\":\n",
" print sum_of_squares(3, 4)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting squares2.py\n"
]
}
],
"prompt_number": 10
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"!DEBUG=1 python squares2.py"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"25\r\n"
]
}
],
"prompt_number": 11
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"!python squares2.py"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"25\r\n"
]
}
],
"prompt_number": 12
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Problem** Write a decorator function `with_retries` that continue to retry for 5 times if there is any exception raised when executing the function.\n",
"\n",
" @with_retries\n",
" def wget(url):\n",
" return urllib2.urlopen(url).read()\n",
" \n",
" wget(\"http://google.com/no-such-page\")\n",
" \n",
"This should print:\n",
"\n",
" Failed to download. retrying...\n",
" Failed to download. retrying...\n",
" Failed to download. retrying... \n",
" Failed to download. retrying... \n",
" Failed to download. retrying... \n",
" Giving up."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file with_retries1.py\n",
"import functools\n",
"\n",
"def with_retries(f):\n",
" retries = 5\n",
" @functools.wraps(f)\n",
" def g(*args, **kwargs):\n",
" for i in range(retries):\n",
" try:\n",
" return f(*args, **kwargs)\n",
" except:\n",
" print \"Failed to download. retrying...\"\n",
" print \"Giving up\"\n",
" return g\n",
"\n",
"if __name__ == \"__main__\":\n",
" import urllib2\n",
" \n",
" @with_retries\n",
" def wget(url):\n",
" return urllib2.urlopen(url)\n",
" \n",
" wget(\"http://google.com/no-such-page\")\n",
" wget(url=\"http://google.com/no-such-page\")\n",
" "
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting with_retries1.py\n"
]
}
],
"prompt_number": 17
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"!python with_retries1.py"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Failed to download. retrying...\r\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Failed to download. retrying...\r\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Failed to download. retrying...\r\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Failed to download. retrying...\r\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Failed to download. retrying...\r\n",
"Giving up\r\n"
]
}
],
"prompt_number": 16
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now can we make the `with_retries` decorator take number of retries and delay between retry as arguments?\n",
"\n",
" @with_retries(retries=3, delay=5.0)\n",
" def wget(url):\n",
" return urllib2.urlopen(url)\n",
" \n",
"This is equivalant to:\n",
"\n",
" def wget(url):\n",
" return urllib2.urlopen(url)\n",
"\n",
" wget = with_retries(retries=3, delay=5.0)(wget)\n",
" \n",
"Or:\n",
"\n",
" decorator = with_retries(retries=3, delay=5.0)\n",
" wget = decorator(wget)\n",
" \n",
" \n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file with_retries2.py\n",
"import urllib2\n",
"import time\n",
"import functools\n",
"\n",
"def with_retries(retries=5, delay=0.0):\n",
" def decorator(f):\n",
" \"\"\"This is the earlier with_retries that we wrote earlier.\n",
" \n",
" With an additional delay.\n",
" \"\"\"\n",
" @functools.wraps(f)\n",
" def g(*args, **kwargs):\n",
" for i in range(retries):\n",
" try:\n",
" return f(*args, **kwargs)\n",
" except:\n",
" print time.asctime(), \"Failed to download. retrying...\"\n",
" # No need for delay in the last iteration\n",
" if i != retries-1:\n",
" time.sleep(delay)\n",
" print time.asctime(), \"Giving up\"\n",
" return g\n",
" return decorator\n",
"\n",
"@with_retries(retries=3, delay=2.0)\n",
"def wget(url):\n",
" return urllib2.urlopen(url)\n",
"\n",
"if __name__ == \"__main__\":\n",
" wget(\"http://localhost/no-such-page\")"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting with_retries2.py\n"
]
}
],
"prompt_number": 31
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"!python with_retries2.py"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Thu Feb 27 10:18:54 2014 Failed to download. retrying...\r\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Thu Feb 27 10:18:56 2014 Failed to download. retrying...\r\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Thu Feb 27 10:18:58 2014 Failed to download. retrying...\r\n",
"Thu Feb 27 10:18:58 2014 Giving up\r\n"
]
}
],
"prompt_number": 32
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file with_retries3.py\n",
"\"\"\"Trying to simplify with_retries2.py\n",
"\"\"\"\n",
"import urllib2\n",
"import time\n",
"import functools\n",
"\n",
"def with_retries(retries=5, delay=0.0):\n",
" return lambda f: with_retries_decorator(f, retries, delay)\n",
" # The above lambda is equivalant to:\n",
" # def decor(f):\n",
" # return with_retries_decorator(f, retries, delay)\n",
" # return decor\n",
"\n",
"def with_retries_decorator(f, retries, delay):\n",
" \"\"\"This is the earlier with_retries that we wrote earlier.\n",
" \n",
" With an additional delay.\n",
" \"\"\"\n",
" @functools.wraps(f)\n",
" def g(*args, **kwargs):\n",
" for i in range(retries):\n",
" try:\n",
" return f(*args, **kwargs)\n",
" except:\n",
" print time.asctime(), \"Failed to download. retrying...\"\n",
" # No need for delay in the last iteration\n",
" if i != retries-1:\n",
" time.sleep(delay)\n",
" print time.asctime(), \"Giving up\"\n",
" return g\n",
"\n",
"@with_retries(retries=3, delay=2.0)\n",
"def wget(url):\n",
" return urllib2.urlopen(url)\n",
"\n",
"if __name__ == \"__main__\":\n",
" wget(\"http://localhost/no-such-page\")\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting with_retries3.py\n"
]
}
],
"prompt_number": 39
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"!python with_retries3.py"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Thu Feb 27 10:36:54 2014 Failed to download. retrying...\r\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Thu Feb 27 10:36:56 2014 Failed to download. retrying...\r\n"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Thu Feb 27 10:36:58 2014 Failed to download. retrying...\r\n",
"Thu Feb 27 10:36:58 2014 Giving up\r\n"
]
}
],
"prompt_number": 40
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file with_retries4.py\n",
"\"\"\"Further simplification of with_retries3.py\n",
"\"\"\"\n",
"import urllib2\n",
"import time\n",
"import functools\n",
"\n",
"def with_retries(f=None, retries=5, delay=0.0):\n",
" print \"with_retries\", f, retries, delay\n",
" if f is None:\n",
" # return a decorator\n",
" return lambda f2: with_retries(f2, retries, delay)\n",
" \n",
" @functools.wraps(f)\n",
" def g(*args, **kwargs):\n",
" for i in range(retries):\n",
" try:\n",
" return f(*args, **kwargs)\n",
" except:\n",
" print time.asctime(), \"Failed to download. retrying...\"\n",
" # No need for delay in the last iteration\n",
" if i != retries-1:\n",
" time.sleep(delay)\n",
" print time.asctime(), \"Giving up\"\n",
" return g\n",
"\n",
"@with_retries\n",
"def wget(url):\n",
" return urllib2.urlopen(url)\n",
"\n",
"if __name__ == \"__main__\":\n",
" wget(\"http://localhost/no-such-page\")\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting with_retries4.py\n"
]
}
],
"prompt_number": 46
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"!python with_retries4.py"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"with_retries <function wget at 0x10046d7d0> 5 0.0\r\n",
"Thu Feb 27 10:46:39 2014 Failed to download. retrying...\r\n",
"Thu Feb 27 10:46:39 2014 Failed to download. retrying...\r\n",
"Thu Feb 27 10:46:39 2014 Failed to download. retrying...\r\n",
"Thu Feb 27 10:46:39 2014 Failed to download. retrying...\r\n",
"Thu Feb 27 10:46:39 2014 Failed to download. retrying...\r\n",
"Thu Feb 27 10:46:39 2014 Giving up\r\n"
]
}
],
"prompt_number": 48
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Problem** Write a decorator `ignore_exceptions` that ignores any exeption raised by the function. If optional argument exctype is passed, it should ignore only exceptions of that type.\n",
"\n",
" @ignore_exception(exctype=IOError)\n",
" def read_config():\n",
" return open(\"config.txt\").read()\n",
" \n",
" @ignore_exception(exctype=IOError)\n",
" def f(value):\n",
" return int(value)\n",
" \n",
" # f(\"not-integer\") should raise ValueError because we have asked ignore_exception to ignore only exceptions of type IOError."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"def log_function(f):\n",
" print f.__name__, \"is defined\"\n",
" return f\n",
"\n",
"@log_function\n",
"def square(x):\n",
" return x*x\n",
"\n",
"print square(4)\n",
" "
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"square is defined\n",
"16\n"
]
}
],
"prompt_number": 49
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example: Web Framework**"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file fakeweb.py\n",
"\"\"\"A Flask like simple web framework.\n",
"\"\"\"\n",
"\n",
"before_hooks = []\n",
"after_hooks = []\n",
"path_mapping = []\n",
"\n",
"def route(path):\n",
" def decorator(f):\n",
" path_mapping.append((path, f))\n",
" # print \"route\", path, f\n",
" return f\n",
" return decorator\n",
"\n",
"def find_func(path):\n",
" for _path, func in path_mapping:\n",
" if _path == path:\n",
" return func\n",
"\n",
"def request(path):\n",
" #print \"request\", path\n",
" for h in before_hooks:\n",
" h(path)\n",
" \n",
" func = find_func(path)\n",
" if func:\n",
" response = func()\n",
" else:\n",
" response = \"Not Found\"\n",
" \n",
" for h in after_hooks:\n",
" response = h(response)\n",
" return response\n",
" \n",
"def after_request(f):\n",
" \"\"\"Registers the function f and for every request,\n",
" calls the function f with response as argument\n",
" and uses the return value of that as final response.\n",
" \"\"\"\n",
" after_hooks.append(f)\n",
" return f\n",
" \n",
" \n",
"def before_request(f):\n",
" before_hooks.append(f)\n",
" return f\n",
"\n",
"def wsgifunc(env, start_response):\n",
" path = env['PATH_INFO']\n",
" start_response(\"200 OK\", [(\"Content-type\", \"text/plain\")])\n",
" return request(path)\n",
"\n",
"def run(port=8080):\n",
" from wsgiref.simple_server import make_server\n",
" server = make_server(\"localhost\", port, wsgifunc)\n",
" server.serve_forever()\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting fakeweb.py\n"
]
}
],
"prompt_number": 103
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file webapp.py\n",
"import time\n",
"from fakeweb import (route, request, \n",
" after_request, before_request, run)\n",
"\n",
"@before_request\n",
"def log_request(path):\n",
" print time.asctime(), \"request for\", path\n",
" \n",
"#@notfound\n",
"#def my_notfound():\n",
"# return \"404 - Not Found\"\n",
"\n",
"@after_request\n",
"def uppercase(response):\n",
" return response.upper()\n",
" \n",
"@after_request\n",
"def make_layout(response):\n",
" line = \"=\" * 20\n",
" return \"{0}\\n{1}\\n{0}\\n\".format(line, response.center(20))\n",
"\n",
"@route(path=\"/\")\n",
"def index():\n",
" return \"Welcome\"\n",
"\n",
"@route(path=\"/hello\")\n",
"def hello():\n",
" return \"Hello\"\n",
"\n",
"if __name__ == \"__main__\":\n",
" import sys\n",
" if \"--web\" in sys.argv:\n",
" run()\n",
" else:\n",
" print request(\"/\") # should print Welcome\n",
" print request(\"/hello\") # should print Hello\n",
" print request(\"/xxx\") # should print Not Found\n",
" "
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting webapp.py\n"
]
}
],
"prompt_number": 101
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"!python webapp.py"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Thu Feb 27 12:58:59 2014 request for /\r\n",
"====================\r\n",
" WELCOME \r\n",
"====================\r\n",
"\r\n",
"Thu Feb 27 12:58:59 2014 request for /hello\r\n",
"====================\r\n",
" HELLO \r\n",
"====================\r\n",
"\r\n",
"Thu Feb 27 12:58:59 2014 request for /xxx\r\n",
"====================\r\n",
" NOT FOUND \r\n",
"====================\r\n",
"\r\n"
]
}
],
"prompt_number": 102
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Introducton to Flask"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file helloworld.py\n",
"from flask import Flask\n",
"\n",
"app = Flask(__name__)\n",
"\n",
"@app.route(\"/\")\n",
"def hello():\n",
" return \"Hello, world!\"\n",
"\n",
"if __name__ == \"__main__\":\n",
" app.run()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Writing helloworld.py\n"
]
}
],
"prompt_number": 104
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file helloworld2.py\n",
"from flask import Flask\n",
"\n",
"app = Flask(__name__)\n",
"\n",
"@app.route(\"/\")\n",
"@app.route(\"/<name>\")\n",
"def hello(name=None):\n",
" name = name or \"World\"\n",
" return \"Hello, {0}!\\n\".format(name)\n",
"\n",
"if __name__ == \"__main__\":\n",
" app.run()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting helloworld2.py\n"
]
}
],
"prompt_number": 107
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Problem** Write a flask app that to greet people. Both the message and the name of the person must be taken from the URL.\n",
"\n",
"Example:\n",
"\n",
" /hello/python -> Hello Python!\n",
" /good_morning/everyone -> Good Morning Everyone!\n",
" /happy_birthday/alice -> Happy Birthday Alice!\n",
" \n",
"The underscores in the message should be converted to spaces and both message and name should be coverted to title case. \n",
"\n",
"\n",
" "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file helloworld3.py\n",
"from flask import Flask, request, make_response\n",
"\n",
"app = Flask(__name__)\n",
"\n",
"@app.route(\"/\")\n",
"@app.route(\"/<name>\")\n",
"def hello(name=None): \n",
" repeat = int(request.args.get(\"repeat\", 1))\n",
" name = name or \"World\"\n",
" content = \"Hello, {0}!\\n\".format(name) * repeat\n",
" \n",
" # To set a header, we need to make a response object first.\n",
" response = make_response(content)\n",
" response.content_type = \"text/plain\"\n",
" return response\n",
"\n",
"if __name__ == \"__main__\":\n",
" app.run()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting helloworld3.py\n"
]
}
],
"prompt_number": 113
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file adder.py\n",
"\n",
"from flask import Flask, request, render_template\n",
"\n",
"app = Flask(__name__)\n",
"\n",
"@app.route(\"/\")\n",
"def index():\n",
" return render_template(\"adder.html\")\n",
"\n",
"@app.route(\"/add\")\n",
"def add():\n",
" x = request.args.get(\"x\")\n",
" y = request.args.get(\"y\")\n",
" # TODO: error handling\n",
" z = int(x) + int(y)\n",
" return render_template(\"adder_result.html\",\n",
" x=x, y=y, z=z)\n",
" \n",
" \n",
"if __name__ == \"__main__\":\n",
" app.run()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting adder.py\n"
]
}
],
"prompt_number": 119
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file templates/adder.html\n",
"<!doctype html>\n",
"<html>\n",
" <head><title>Adder</title></head>\n",
" <body>\n",
" <form method=\"GET\" action=\"/add\">\n",
" X: <input type=\"text\" name=\"x\" /><br/>\n",
" Y: <input type=\"text\" name=\"y\" /><br/>\n",
" <input type=\"submit\" name=\"action\" value=\"Add\"/>\n",
" </form>\n",
" </body>\n",
"</html>"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting templates/adder.html\n"
]
}
],
"prompt_number": 116
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%%file templates/adder_result.html\n",
"<!doctype html>\n",
"<html>\n",
" <head><title>Adder</title></head>\n",
" <body>\n",
" <a href=\"/\">&larr; Home</a><br/><br/>\n",
" \n",
" {{x}} + {{y}} = {{z}}\n",
" </body>\n",
"</html>"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Overwriting templates/adder_result.html\n"
]
}
],
"prompt_number": 122
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {}
}
]
}
Display the source blob
Display the rendered blob
Raw
day3.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
day4.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment