Telemetry tutorial

Jupyter.ipynb

{"nbformat_minor": 0, "cells": [{"source": "###Jupyter tutorial", "cell_type": "markdown", "metadata": {}}, {"source": "Jupyter is a command shell for interactive computing in multiple programming languages, originally developed for the Python programming language, that offers introspection, rich media, shell syntax, tab completion, and history.", "cell_type": "markdown", "metadata": {}}, {"source": "You don\u2019t need to know anything beyond Python to start using IPython \u2013 just type commands as you would at the standard Python prompt. TAB completion works in many contexts!", "cell_type": "markdown", "metadata": {}}, {"execution_count": 1, "cell_type": "code", "source": "3 + 3", "outputs": [{"execution_count": 1, "output_type": "execute_result", "data": {"text/plain": "6"}, "metadata": {}}], "metadata": {"collapsed": false, "trusted": true}}, {"execution_count": 2, "cell_type": "code", "source": "def fib(n):\n    if n==1 or n==2: return 1\n    else: return fib(n-1) + fib(n-2)\n    \nfib(10)", "outputs": [{"execution_count": 2, "output_type": "execute_result", "data": {"text/plain": "55"}, "metadata": {}}], "metadata": {"collapsed": false, "trusted": true}}, {"source": "#####Getting help", "cell_type": "markdown", "metadata": {}}, {"execution_count": 3, "cell_type": "code", "source": "%quickref", "outputs": [], "metadata": {"collapsed": true, "trusted": true}}, {"source": "Typing object_name? will print all sorts of details about any object, including docstrings, function definition lines (for call arguments) and constructor details for classes. ", "cell_type": "markdown", "metadata": {}}, {"execution_count": 4, "cell_type": "code", "source": "dir?", "outputs": [], "metadata": {"collapsed": true, "trusted": true}}, {"execution_count": 5, "cell_type": "code", "source": "help(dir)", "outputs": [{"output_type": "stream", "name": "stdout", "text": "Help on built-in function dir in module __builtin__:\n\ndir(...)\n    dir([object]) -> list of strings\n    \n    If called without an argument, return the names in the current scope.\n    Else, return an alphabetized list of names comprising (some of) the attributes\n    of the given object, and of attributes reachable from it.\n    If the object supplies a method named __dir__, it will be used; otherwise\n    the default dir() logic is used and returns:\n      for a module object: the module's attributes.\n      for a class object:  its attributes, and recursively the attributes\n        of its bases.\n      for any other object: its attributes, its class's attributes, and\n        recursively the attributes of its class's base classes.\n\n"}], "metadata": {"collapsed": false, "trusted": true}}, {"source": "#####Magic functions", "cell_type": "markdown", "metadata": {}}, {"source": "IPython has a set of predefined \u2018magic functions\u2019 that you can call with a command line style syntax. There are two kinds of magics, line-oriented and cell-oriented. Line magics are prefixed with the % character and work much like OS command-line calls: they get as an argument the rest of the line, where arguments are passed without parentheses or quotes. Cell magics are prefixed with a double %%, and they are functions that get as an argument not only the rest of the line, but also the lines below it in a separate argument.", "cell_type": "markdown", "metadata": {}}, {"source": "The magic commands %pdoc, %pdef, %psource and %pfile will respectively print the docstring, function definition line, full source code and the complete file for any object (when they can be found). If automagic is on (it is by default), you don\u2019t need to type the \u2018%\u2019 explicitly. See this section for more.", "cell_type": "markdown", "metadata": {}}, {"execution_count": 6, "cell_type": "code", "source": "%pdef fib", "outputs": [{"output_type": "stream", "name": "stdout", "text": " \u001b[0mfib\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mn\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n "}], "metadata": {"collapsed": false, "trusted": true}}, {"execution_count": 7, "cell_type": "code", "source": "%psource fib", "outputs": [], "metadata": {"collapsed": false, "trusted": true}}, {"source": "To time the execution of a python statement or expression:", "cell_type": "markdown", "metadata": {}}, {"execution_count": 8, "cell_type": "code", "source": "%timeit x = 10", "outputs": [{"output_type": "stream", "name": "stdout", "text": "10000000 loops, best of 3: 22.9 ns per loop\n"}], "metadata": {"collapsed": false, "trusted": true}}, {"execution_count": 9, "cell_type": "code", "source": "%time x = 10", "outputs": [{"output_type": "stream", "name": "stdout", "text": "CPU times: user 0 ns, sys: 0 ns, total: 0 ns\nWall time: 10 \u00b5s\n"}], "metadata": {"collapsed": false, "trusted": true}}, {"source": "To run a statement through the python profiler:", "cell_type": "markdown", "metadata": {}}, {"execution_count": 10, "cell_type": "code", "source": "%prun x = 10", "outputs": [{"output_type": "stream", "name": "stdout", "text": " "}], "metadata": {"collapsed": false, "trusted": true}}, {"source": "To enable debugging:", "cell_type": "markdown", "metadata": {}}, {"execution_count": 11, "cell_type": "code", "source": "%pdb", "outputs": [{"output_type": "stream", "name": "stdout", "text": "Automatic pdb calling has been turned ON\n"}], "metadata": {"collapsed": false, "trusted": true}}, {"source": "Use the Python debugger, pdb. The %pdb command allows you to toggle on and off the automatic invocation of an IPython-enhanced pdb debugger (with coloring, tab completion and more) at any uncaught exception. The advantage of this is that pdb starts inside the function where the exception occurred, with all data still available. You can print variables, see code, execute statements and even walk up and down the call stack to track down the true source of the problem (which often is many layers in the stack above where the exception gets triggered", "cell_type": "markdown", "metadata": {}}, {"execution_count": 12, "cell_type": "code", "source": "def foo(n):\n    raise Exception\n\nprint foo(5)", "outputs": [{"ename": "Exception", "evalue": "", "traceback": ["\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mException\u001b[0m                                 Traceback (most recent call last)", "\u001b[1;32m<ipython-input-12-8a693f586f8c>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      2\u001b[0m     \u001b[1;32mraise\u001b[0m \u001b[0mException\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 4\u001b[1;33m \u001b[1;32mprint\u001b[0m \u001b[0mfoo\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m5\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[1;32m<ipython-input-12-8a693f586f8c>\u001b[0m in \u001b[0;36mfoo\u001b[1;34m(n)\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mfoo\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mn\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m     \u001b[1;32mraise\u001b[0m \u001b[0mException\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      3\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[1;32mprint\u001b[0m \u001b[0mfoo\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m5\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", "\u001b[1;31mException\u001b[0m: "], "output_type": "error"}, {"output_type": "stream", "name": "stdout", "text": "> \u001b[1;32m<ipython-input-12-8a693f586f8c>\u001b[0m(2)\u001b[0;36mfoo\u001b[1;34m()\u001b[0m\n\u001b[1;32m      1 \u001b[1;33m\u001b[1;32mdef\u001b[0m \u001b[0mfoo\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mn\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m----> 2 \u001b[1;33m    \u001b[1;32mraise\u001b[0m \u001b[0mException\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3 \u001b[1;33m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\n\nKeyboardInterrupt\n"}], "metadata": {"collapsed": false, "trusted": true}}, {"source": "######System commands", "cell_type": "markdown", "metadata": {}}, {"execution_count": 13, "cell_type": "code", "source": "!echo \"foobar\" > /tmp/foo", "outputs": [], "metadata": {"collapsed": false, "trusted": true}}, {"execution_count": 14, "cell_type": "code", "source": "!cat /tmp/foo", "outputs": [{"output_type": "stream", "name": "stdout", "text": "foobar\r\n"}], "metadata": {"collapsed": false, "trusted": true}}, {"execution_count": 15, "cell_type": "code", "source": "out = !ls /", "outputs": [], "metadata": {"collapsed": true, "trusted": true}}, {"execution_count": 16, "cell_type": "code", "source": "print out", "outputs": [{"output_type": "stream", "name": "stdout", "text": "['backports', 'bin', 'boot', 'dev', 'etc', 'home', 'include', 'lib', 'lib64', 'local', 'lost+found', 'media', 'mnt', 'mnt1', 'opt', 'proc', 'root', 'sbin', 'selinux', 'share', 'srv', 'sys', 'tmp', 'usr', 'var']\n"}], "metadata": {"collapsed": false, "trusted": true}}, {"source": "#### Exercises", "cell_type": "markdown", "metadata": {}}, {"source": "1. Take the User Interface Tour (Help menu)", "cell_type": "markdown", "metadata": {}}], "nbformat": 4, "metadata": {"kernelspec": {"display_name": "Python 2", "name": "python2", "language": "python"}, "language_info": {"mimetype": "text/x-python", "nbconvert_exporter": "python", "version": "2.7.9", "name": "python", "file_extension": ".py", "pygments_lexer": "ipython2", "codemirror_mode": {"version": 2, "name": "ipython"}}}}

pandas.ipynb

Spark.ipynb