FFI.ipynb

Idioms.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Whirlwind Tour of Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variables\n",
    "Variables must start with a-Z or `_`, but numbers are allowed in the name.\n",
    "Comments follow `#` characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "a = 3 # valid\n",
    "variable3 = 3 # valid\n",
    "_variable_name = \"this variable name starts with an underscore\" \n",
    "# though you might want to for module variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-2-ae347ee45edb>, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-2-ae347ee45edb>\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    3variable = 'this will not work' # SyntaxError\u001b[0m\n\u001b[0m            ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "3variable = 'this will not work' # SyntaxError"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Strings\n",
    "Strings are everywhere in Python. Values enclosed in quotes `\" or '` will be taken as strings, or the `str` method can be used to construct them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "string_variable = \"this is a string\"\n",
    "another = 'so is this'\n",
    "and_another = str(35823) # this is too"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are also `raw` strings and `bytes` objects but we will ignore these for now..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('raw \\\\n\\\\rstring', b'bytes', 'not a raw \\nstring')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r\"raw \\n\\rstring\", b'bytes', \"not a raw \\nstring\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tuples\n",
    "Tuples are collections of objects, you can create them with the `,` operator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tup = 1, 2, 3, 4, (5, 6)\n",
    "tup[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5, 6)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tup[4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'tuple' object does not support item assignment",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-7-81e277b7f4d2>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m3\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m6\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: 'tuple' object does not support item assignment"
     ]
    }
   ],
   "source": [
    "x = 3, 6\n",
    "x[1] = 4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 'pythonic' python idioms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lists, dicts\n",
    "Lists and dictionaries are basically the go-to simple data structures in python.\n",
    "Lists are (as you'd think) collections of objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "animals = ['kangaroo', 'bald eagle', 'hedgehog', 'dodo']\n",
    "places = ['Australa', 'USA', 'UK', 'Madagascar']\n",
    "fibs = [1, 1, 2, 3, 5, 8, 13]\n",
    "evens = [x for x in range(1, 100) if x % 2 == 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'hedgehog'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "animals[2] # zero based indexing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KANGAROO\n",
      "BALD EAGLE\n",
      "HEDGEHOG\n",
      "DODO\n",
      "Madagascar\n",
      "UK\n",
      "USA\n",
      "Australa\n"
     ]
    }
   ],
   "source": [
    "for animal in animals:\n",
    "    print(animal.upper())\n",
    "\n",
    "for place in reversed(places):\n",
    "    print(place)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A `dict` is an associations between objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "fruits = {'banana':'mushy', 'apple': 'crisp', 'lemon': 'sour'}\n",
    "is_prime = {2: True, 3: True, 4: False, 5: True, 6: False}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "is_prime[5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mushy\n",
      "crisp\n",
      "sour\n"
     ]
    }
   ],
   "source": [
    "for fruit, adjective in fruits.items():\n",
    "    print(adjective)\n",
    "\n",
    "# this will do nothing in this case\n",
    "for key in fruits.keys():\n",
    "    fruits[str(key)] = fruits[key] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `in`, `is`, `==`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'kangaroo' in animals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'k' in 'kangaroo'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'banana' in fruits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = 3\n",
    "b = 500\n",
    "a is 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b is 500 # Don't use `is` for equality, use it for *identity*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b == 500"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `if` statements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "value = True\n",
    "\n",
    "# Do this:\n",
    "if value:\n",
    "    print(value)\n",
    "\n",
    "# Not this:\n",
    "if value == True:\n",
    "    print(value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`0`, `[]`, 'None' and several other values evaluate to `False` boolean condiations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if [] or None or 0:\n",
    "    print(\"this won't happen\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because of this, if you want to check if an object is None,\n",
    "generally you should check `x is None` in functions rather than checking its boolean evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if x is None:\n",
    "    print('None')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Strings\n",
    "join strings using their builtin `join` method as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kangaroo, bald eagle, hedgehog, dodo\n",
      "Options are: kangaroo, bald eagle, hedgehog or dodo\n"
     ]
    }
   ],
   "source": [
    "print(', '.join(animals))\n",
    "print('Options are:', ', '.join(animals[:-1]), 'or', animals[-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['one', 'two', 'three', 'four-five']"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'one two three four-five'.split()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['one two three four', 'five']"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'one two three four-five'.split('-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Formatting strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'this is a float formatted to 3 decimal places 0.333'"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_string = \"this is a {word} formatted to 3 decimal places {s:.3f}\"\n",
    "example_string.format(word=\"float\", s=(1/3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For details of the formatting check [this](https://mkaz.tech/python-string-format.html) out."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Opening a file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test file please ignore!\n",
      "\n",
      "A new lineTest file please ignore!\\nThis will not have a new line\n"
     ]
    }
   ],
   "source": [
    "# Open a file for writing\n",
    "with open('/tmp/file.txt', 'w') as file_object: \n",
    "    file_object.write('Test file please ignore!\\nA new line')\n",
    "    file_object.write(r'Test file please ignore!\\nThis will not have a new line')\n",
    "# Now let's read the contents back, line by line\n",
    "with open('/tmp/file.txt') as file_object:\n",
    "    for line in file_object:\n",
    "        print(line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using pathlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/prs/OneDrive/notebooks\n",
      "Learning Python\n"
     ]
    }
   ],
   "source": [
    "this_directory = Path('.').absolute() # Convert to absolute path\n",
    "print(this_directory.parent)\n",
    "print(this_directory.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also `glob` paths just like the shell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/prs/OneDrive/notebooks/Learning Python/Idioms.ipynb True\n",
      "/Users/prs/OneDrive/notebooks/Learning Python/FFI.ipynb True\n"
     ]
    }
   ],
   "source": [
    "for d in this_directory.glob('*.ipynb'): # glob is a generator, not a list\n",
    "    print(d, d.is_file())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using generators"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def my_range(x):\n",
    "    i = 0\n",
    "    while i < x:\n",
    "        i = i + 1\n",
    "        yield i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<generator object my_range at 0x1140e02b0>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_range(10) # generators are not lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(my_range(10)) # They can be converted to lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n",
      "6\n",
      "7\n",
      "8\n",
      "9\n",
      "10\n"
     ]
    }
   ],
   "source": [
    "for value in my_range(10): # And used in for loops\n",
    "    print(value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generator expressions\n",
    "Rather than doing this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2, 4, 6, 8, 10, 12, 14, 16, 18, 20]\n"
     ]
    }
   ],
   "source": [
    "twos = []\n",
    "for i in range(1, 100):\n",
    "    if i % 2 == 0:\n",
    "        twos.append(i)\n",
    "print(twos[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "twos_generator = (i for i in range(1, 10000000) if i % 2 == 0)\n",
    "# this will NOT generate the whole object/list! it is a generator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can make a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "l = list(twos_generator)[:30] # this will take a long time, don't do it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lambdas\n",
    "we could also have done it using lambdas (anonymous functions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "twos_func = lambda x: (i for i in range(1, x) if i % 2 == 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2,\n",
       " 4,\n",
       " 6,\n",
       " 8,\n",
       " 10,\n",
       " 12,\n",
       " 14,\n",
       " 16,\n",
       " 18,\n",
       " 20,\n",
       " 22,\n",
       " 24,\n",
       " 26,\n",
       " 28,\n",
       " 30,\n",
       " 32,\n",
       " 34,\n",
       " 36,\n",
       " 38,\n",
       " 40,\n",
       " 42,\n",
       " 44,\n",
       " 46,\n",
       " 48]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(twos_func(50))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using *args\n",
    "This is basically how you do 'variadic' arguments in python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "blahblah\n",
      "6\n",
      "11.4\n",
      "200\n",
      "applesapples\n",
      "blah 3 5.7 100 apples\n"
     ]
    }
   ],
   "source": [
    "def func_using_args(*the_arguments):\n",
    "    # Have a guess how 'print' works...\n",
    "    for arg in the_arguments:\n",
    "        print(arg*2)\n",
    "    # convert each arg to a string then concatenate them\n",
    "    print(' '.join(map(str, the_arguments)))\n",
    "        \n",
    "func_using_args('blah', 3, 5.7, 100, 'apples')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using **kwargs\n",
    "`**kwargs` might look a bit intimidating but it can be extremely useful. it's basically like args, but we want some semantic information about what the intent of the argument was..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def number_of_animals(**kwargs):\n",
    "    for animal in animals:\n",
    "        value = kwargs.get(animal, 0)\n",
    "        print('We have {} {}{}'.format(value, animal, '' if value == 1 else 's'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We have 0 kangaroos\n",
      "We have 0 bald eagles\n",
      "We have 3 hedgehogs\n",
      "We have 0 dodos\n"
     ]
    }
   ],
   "source": [
    "number_of_animals(hedgehog=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Basically, it comes from so called 'dict unpacking'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "my_dict = {'hedgehog': 5, 'bananas':3}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We have 0 kangaroos\n",
      "We have 0 bald eagles\n",
      "We have 5 hedgehogs\n",
      "We have 0 dodos\n"
     ]
    }
   ],
   "source": [
    "number_of_animals(**my_dict) # Unpack the dictionary as args"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### zip!\n",
    "Zip does what it says, it pushes together two (or more) lists/generators..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['kangaroo', 'bald eagle', 'hedgehog', 'dodo']"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "animals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Australa', 'USA', 'UK', 'Madagascar']"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "places"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('kangaroo', 'Australa')\n",
      "('bald eagle', 'USA')\n",
      "('hedgehog', 'UK')\n",
      "('dodo', 'Madagascar')\n"
     ]
    }
   ],
   "source": [
    "for x in zip(animals, places): # length \n",
    "    print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'bald eagle': 'USA',\n",
       " 'dodo': 'Madagascar',\n",
       " 'hedgehog': 'UK',\n",
       " 'kangaroo': 'Australa'}"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dict(zip(animals,places)) # We can also directly construct things like dicts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### namedtuple\n",
    "For quick and dirty classes use `namedtuple` (can also be `pickle`d):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from collections import namedtuple"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "Coordinates = namedtuple('Coordinates', 'x y z units')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Coordinates(x=3, y=1.5, z=0.3333333333333333, units='cartesian')"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Coordinates(3, 1.5, 1/3, 'cartesian')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Coordinates(x=1.0, y=1.707, z=0.3, units='spherical')"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Coordinates(1.0, 1.707, 0.3, 'spherical')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Miscellaneous and advanced stuff\n",
    "Swapping values at the *same* time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7 3\n"
     ]
    }
   ],
   "source": [
    "a = 3\n",
    "b = 7\n",
    "a, b = b, a\n",
    "print(a, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Partial application \n",
    "Iterating until a sentinel value is reached"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Test file please', ' ignore!\\nA new l', 'ineTest file ple', 'ase ignore!\\\\nThi', 's will not have ', 'a new line']\n"
     ]
    }
   ],
   "source": [
    "from functools import partial\n",
    "# e.g. read a file blocks of 16 characters at a time\n",
    "with open('/tmp/file.txt') as f:\n",
    "    blocks = [block for block in iter(partial(f.read, 16), '')]\n",
    "print(blocks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[16, 16, 16, 16, 16, 10]"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[len(x) for x in blocks]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `for` ... `else`\n",
    "An `else` block after a `for` loop is executed only when the loop runs to completion (i.e. no `break`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def find(seq, target):\n",
    "    for i, value in enumerate(seq):\n",
    "        if value == target:\n",
    "            break\n",
    "    else:\n",
    "        return -1\n",
    "    return i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "find(range(10), 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-1"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "find(range(10), 13)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `defaultdict`\n",
    "We often want to create a `dict` object with the same initial default type/value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "# group a dictionary by animal name length\n",
    "\n",
    "d = defaultdict(list) # if a key doesn't exist, it's created as an empty list\n",
    "for name in animals:\n",
    "    key = len(name)\n",
    "    d[key].append(name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defaultdict(list,\n",
       "            {4: ['dodo'], 8: ['kangaroo', 'hedgehog'], 10: ['bald eagle']})"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d[3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `chainmap`\n",
    "Use chainmap to chain lookups, e.g. if we have a program with command line argumeents, preferred over environment variables preferred over default values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import argparse\n",
    "from collections import ChainMap\n",
    "import os\n",
    "\n",
    "defaults = {'color': 'red', 'user': 'guest'}\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('-u', '--user')\n",
    "parser.add_argument('-c', '--color')\n",
    "namespace = parser.parse_args([])\n",
    "command_line_args = {k:v for k, v in vars(namespace).items() if v}\n",
    "\n",
    "# getting values will first try command line, then environment, then default values\n",
    "d = ChainMap(command_line_args, os.environ, defaults)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/Users/prs/notebooks'"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d['PWD']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'guest'"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d['user']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Caching\n",
    "Cache expensive lookups like web calls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from functools import lru_cache as cache # least recently used cache\n",
    "from urllib import request\n",
    "\n",
    "@cache(10) # Cache up to 10 calls of 'web_lookup'\n",
    "def web_lookup(url):\n",
    "    return request.urlopen(url).read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 60.9 ms, sys: 7.84 ms, total: 68.8 ms\n",
      "Wall time: 918 ms\n"
     ]
    }
   ],
   "source": [
    "%time x = web_lookup('https://google.com')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 8 µs, sys: 1 µs, total: 9 µs\n",
      "Wall time: 15.7 µs\n"
     ]
    }
   ],
   "source": [
    "%time x = web_lookup('https://google.com')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FFI.ipynb                      \u001b[34mmod.cpython-36m-darwin.so.dSYM\u001b[m\u001b[m\n",
      "Idioms.ipynb                   mod.f90\n",
      "_example.c                     test.a\n",
      "\u001b[31m_example.cpython-36m-darwin.so\u001b[m\u001b[m test.f90\n",
      "_example.o                     test.mod\n",
      "f2py.log                       test.o\n",
      "\u001b[31mmod.cpython-36m-darwin.so\u001b[m\u001b[m      test.pyx\n",
      "CPU times: user 6.41 ms, sys: 13 ms, total: 19.4 ms\n",
      "Wall time: 135 ms\n"
     ]
    }
   ],
   "source": [
    "# subprocess calls have overhead!\n",
    "%time !/bin/ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mod.f90\n",
      "test.mod\n",
      "test.f90\n",
      "_example.cpython-36m-darwin.so\n",
      "mod.cpython-36m-darwin.so\n",
      "test.a\n",
      "Idioms.ipynb\n",
      "test.o\n",
      "_example.c\n",
      "FFI.ipynb\n",
      "test.pyx\n",
      ".ipynb_checkpoints\n",
      "mod.cpython-36m-darwin.so.dSYM\n",
      "f2py.log\n",
      "_example.o\n",
      "CPU times: user 708 µs, sys: 759 µs, total: 1.47 ms\n",
      "Wall time: 830 µs\n"
     ]
    }
   ],
   "source": [
    "%time print('\\n'.join(str(x) for x in Path('.').iterdir()))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}