GRPUG: Black Magic Python. http://nbviewer.ipython.org/gist/jacebrowning/110d33cc39d24c746f6f

magic.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Black Magic Python\n",
    "\n",
    "[@JaceBrowning](https://twitter.com/jacebrowning)\n",
    "\n",
    "### Format\n",
    "\n",
    "* Demonstration\n",
    "* Explanation\n",
    "* Repeat!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Even Modules are Objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.09\n"
     ]
    }
   ],
   "source": [
    "! pip3 install sh==1.09 --quiet\n",
    "import sh\n",
    "print(sh.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Demonstration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Hello, world!"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sh import echo\n",
    "\n",
    "echo(\"Hello, world!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "git version 2.1.4"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sh import git\n",
    "\n",
    "git.version()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "68.43.180.239"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sh import curl\n",
    "\n",
    "curl(\"icanhazip.com\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "total 3608\n",
       "-rw-------  1 Browning  staff      624 Dec 15 19:32 AAL.md\n",
       "-rw-------  1 Browning  staff      254 Oct 19 22:18 Black Magic.gdoc\n",
       "-rw-------  1 Browning  staff      258 Oct 21 16:17 Black Magic: Metaclasses.gslides\n",
       "-rw-------  1 Browning  staff  1631707 Dec 15 19:38 Continuous Integration.key\n",
       "-rw-------  1 Browning  staff      258 Nov 17 17:04 GR GiveCamp 2014.gslides\n",
       "-rw-------  1 Browning  staff      258 Oct 19 20:25 PyPI Distribution.gslides\n",
       "-rw-------  1 Browning  staff      258 Jan 20  2014 Python 3 in Doorstop.gslides\n",
       "-rw-------  1 Browning  staff      258 Aug 19  2013 Python Toolbox.gslides\n",
       "drwxr-xr-x  8 Browning  staff      272 Dec 22 19:50 \u001b[1m\u001b[36mVirtualenv and All His Friends.key\u001b[m\u001b[m\n",
       "-rw-------  1 Browning  staff   182818 Aug 18  2014 Virtualenv and All His Friends.pdf\n",
       "drwxr-xr-x  7 Browning  staff      238 Mar 10 01:48 \u001b[1m\u001b[36mblack_magic\u001b[m\u001b[m\n",
       "drwxr-xr-x  5 Browning  staff      170 Dec 22 19:50 \u001b[1m\u001b[36mmock_integration\u001b[m\u001b[m"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sh import ls\n",
    "\n",
    "ll = ls.bake(\"-l\")\n",
    "\n",
    "ll(\"..\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Explanation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a wrapper for the current module:\n",
    "\n",
    "https://github.com/amoffat/sh/blob/80af5726d8aa42017ced548abbd39b489068922a/sh.py#L1695-1700"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Replace the module object with the wrapper object:\n",
    "\n",
    "https://github.com/amoffat/sh/blob/80af5726d8aa42017ced548abbd39b489068922a/sh.py#L1770"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Maintaining Object Identity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8.3\n"
     ]
    }
   ],
   "source": [
    "! pip3 install doorstop==0.8.3 --quiet\n",
    "import doorstop\n",
    "print(doorstop.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Demonstration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TEST_CASE\n",
      "42\n"
     ]
    }
   ],
   "source": [
    "from doorstop.core.types import UID\n",
    "\n",
    "value = \"TEST_CASE-00042\"\n",
    "\n",
    "uid = UID(value)  # sometimes the user provides a string\n",
    "\n",
    "print(uid.prefix)\n",
    "print(uid.number)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "uid2 = UID(uid)  # other times they provide an object\n",
    "\n",
    "uid is uid2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Explanation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Override `__new__` to return the original object:\n",
    "\n",
    "https://github.com/jacebrowning/doorstop/blob/master/doorstop/core/types.py#L72-76"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modifying a Class Definition with a Decorator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.3\n"
     ]
    }
   ],
   "source": [
    "! pip3 install yorm==0.3 --quiet\n",
    "import yorm\n",
    "print(yorm.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Demonstration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class Student:\n",
    "    def __init__(self, name, school, number, year=2009):\n",
    "        self.name = name\n",
    "        self.school = school\n",
    "        self.number = number\n",
    "        self.year = year\n",
    "        self.gpa = 0.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hasattr(Student, 'yorm_attrs')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import yorm\n",
    "from yorm.standard import String, Integer, Float\n",
    "\n",
    "@yorm.attr(name=String, year=Integer, gpa=Float)\n",
    "@yorm.sync(\"students/{self.school}/{self.number}.yml\")\n",
    "class MappedStudent:\n",
    "    def __init__(self, name, school, number, year=2009):\n",
    "        self.name = name\n",
    "        self.school = school\n",
    "        self.number = number\n",
    "        self.year = year\n",
    "        self.gpa = 0.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'gpa': yorm.standard.Float,\n",
       " 'name': yorm.standard.String,\n",
       " 'year': yorm.standard.Integer}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MappedStudent.yorm_attrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Explanation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set a new base class:\n",
    "\n",
    "https://github.com/jacebrowning/yorm/blob/24f4c3b1e25d4a5ab8cfff6d8f95cc3bb57e0cff/yorm/utilities.py#L67"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dynamically Modifying an Object's Class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Demonstration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class Student:\n",
    "    def __init__(self, name, school, number, year=2009):\n",
    "        self.name = name\n",
    "        self.school = school\n",
    "        self.number = number\n",
    "        self.year = year\n",
    "        self.gpa = 0.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "student = Student(\"John Doe\", \"GVSU\", 123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hasattr(Student, 'yorm_attrs')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from yorm import sync\n",
    "\n",
    "path = \"students/GVSU/123.yml\"\n",
    "attrs = {\n",
    "    'year': yorm.standard.Integer,\n",
    "    'name': yorm.standard.String,\n",
    "    'gpa': yorm.standard.Float,\n",
    "}\n",
    "student = sync(student, path, attrs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'gpa': yorm.standard.Float,\n",
       " 'name': yorm.standard.String,\n",
       " 'year': yorm.standard.Integer}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "student.yorm_attrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Explanation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the object's class at runtime:\n",
    "\n",
    "https://github.com/jacebrowning/yorm/blob/24f4c3b1e25d4a5ab8cfff6d8f95cc3bb57e0cff/yorm/utilities.py#L30"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Controlling All Attribute Access"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Demonstration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "john = MappedStudent(\"John Doe\", \"GVSU\", 123)\n",
    "jane = MappedStudent(\"Jane Doe\", \"GVSU\", 456, year=2014)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "total 0\n",
       "drwxr-xr-x  4 Browning  staff  136 Mar 10 01:48 \u001b[1m\u001b[36mGVSU\u001b[m\u001b[m"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ll(\"students\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "total 16\n",
       "-rw-r--r--  1 Browning  staff  35 Mar 10 01:48 123.yml\n",
       "-rw-r--r--  1 Browning  staff  35 Mar 10 01:48 456.yml"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ll(\"students/GVSU\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "! open students/GVSU/123.yml  # observe file created"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "john.gpa = 3.7  # change reflected in the file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "! open students/GVSU/123.yml  # observe file updated, change year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2009"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "john.year  # change reflected in the object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Explanation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Override `__setattr__` to control how attribues are set:\n",
    "\n",
    "https://github.com/jacebrowning/yorm/blob/24f4c3b1e25d4a5ab8cfff6d8f95cc3bb57e0cff/yorm/base.py#L30"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Override `__getattribute__` to control how **ALL** attributes are read:\n",
    "\n",
    "https://github.com/jacebrowning/yorm/blob/24f4c3b1e25d4a5ab8cfff6d8f95cc3bb57e0cff/yorm/base.py#L14"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note:  `__getattribute__` is the complement of `__setattr__`, `__getattr__` is used more often."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dymanically Looking Up Subclasses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Demonstration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "! mkdir students/GRCC\n",
    "! touch students/GRCC/789.yml\n",
    "! open students/GRCC/789.yml  # change name to \"GRPUG\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "student = MappedStudent(None, \"GRCC\", 789)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'GRPUG'"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "student.name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "student.gpa = 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "! open students/GRCC/789.yml  # observe value converted to `float`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hasattr(student, 'major')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "! open students/GRCC/789.yml  # add a major \"Python\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Python'"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "student.major"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hasattr(student, 'graduated')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "! open students/GRCC/789.yml  # add a graduation status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "student.graduated"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "student.graduated = \"yes\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "! open students/GRCC/789.yml  # observe value converted to `bool`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Explanation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `__subclasses__` attribute lists all known subclasses:\n",
    "\n",
    "https://github.com/jacebrowning/yorm/blob/24f4c3b1e25d4a5ab8cfff6d8f95cc3bb57e0cff/yorm/standard.py#L104"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}