Skip to content

Instantly share code, notes, and snippets.

@hltbra

hltbra/dynamic-python.ipynb

Created August 1, 2018 22:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hltbra/90794c4b275401d8a0d4a4ebe344170e to your computer and use it in GitHub Desktop.
Save hltbra/90794c4b275401d8a0d4a4ebe344170e to your computer and use it in GitHub Desktop.
Download ZIP
Tech talk at Yipit about Python dynamic features - 2018/08/01
Display the source blob
Display the rendered blob
Raw
dynamic-python.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Dynamic Python & Metaprogramming\n",
"\n",
"## Agenda\n",
"\n",
"* `__init__()`\n",
"* `*args` & `**kwargs`\n",
"* protected, private, magic: conventions\n",
"* getattr (`__getattr__`), hasattr (`__getattr__`), setattr (`__setattr__`)\n",
"* `__dict__`\n",
"* `__call__`\n",
"* decorators\n",
"* `property`\n",
"* `staticmethod`\n",
"* `classmethod`\n",
"* context manager\n",
"* magic methods\n",
" - numeric types\n",
" - reverse numeric\n",
" - container types\n",
" - comparison\n",
"* type class\n",
"* metaclass\n",
"\n",
"## References\n",
"\n",
"* https://www.youtube.com/watch?v=Bg_cEIkOS9g\n",
"* http://python-3-patterns-idioms-test.readthedocs.io/en/latest/Metaprogramming.html\n",
"* https://realpython.com/python-metaclasses/\n",
"* https://stackoverflow.com/a/17330273/565999\n",
"* https://gist.github.com/hltbra/9c0983584e40bd690b36da1a37303950\n",
"* https://github.com/Yipit/elasticfun\n",
"* https://github.com/Yipit/restaurant_chain/blob/master/restaurant_chain/factory.py\n",
"* https://github.com/Yipit/pipeapp/blob/master/pipeapp/pipe_db/__init__.py#L90-L109\n",
"* https://github.com/spulec/freezegun/blob/d99329008fe2abe439b108068c026ade511f6ea3/freezegun/api.py#L238-L240\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Why would I ever do this?\n",
"\n",
"* syntax sugar\n",
"* simpler (to use) APIs\n",
"* DSLs[[1]](https://martinfowler.com/bliki/DomainSpecificLanguage.html)[[2]](https://martinfowler.com/bliki/DslBoundary.html)\n",
" - [web frameworks](http://flask.pocoo.org/docs/1.0/quickstart/)\n",
" - [ORM](https://docs.sqlalchemy.org/en/devel/orm/tutorial.html#declare-a-mapping)\n",
" - testing\n",
" - spreadsheet example at the end\n",
"* control context behavior\n",
"* aspect-oriented programming (i.e., extend behavior of existing code)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"f = open('hdkflahl.txt')\n",
"try:\n",
" content = f.read()\n",
"finally:\n",
" f.close()\n",
" \n",
"with open('fdjhsaklfdaskl.txt') as f:\n",
" content = f.read()\n",
"\n",
"with clean_redis() as redis:\n",
" redis.set(\"foo\", \"bar\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## `__init__()`"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"ename": "TypeError",
"evalue": "__init__() takes exactly 1 argument (2 given)",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-14-b06ba6e02151>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0msam\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mPerson\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Sam Larson'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0mwes\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mSibling\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Wes'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mTypeError\u001b[0m: __init__() takes exactly 1 argument (2 given)"
]
}
],
"source": [
"class Person(object):\n",
"\n",
" def __init__(self, name):\n",
" self.name = name\n",
"\n",
" \n",
"\n",
"class Sibling(Person):\n",
" def __init__(self):\n",
" pass\n",
" \n",
"sam = Person('Sam Larson')\n",
"wes = Sibling('Wes')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## `*args` & `**kwargs`"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"params = ('hugo', 'lopes')\n",
"namedargs = {}\n",
"Hello hugo lopes\n",
"params = ('hugo', 'lopes', 'tavares')\n",
"namedargs = {'my-name': 'hugo'}\n",
"Hello hugo lopes tavares\n"
]
}
],
"source": [
"def greet(*params, **namedargs):\n",
" print('params = {}'.format(repr(params)))\n",
" print(\"namedargs = {}\".format(repr(namedargs)))\n",
" print(\"Hello {}\".format(' '.join(params)))\n",
" \n",
"greet('hugo', 'lopes')\n",
"\n",
"def mysum(*args): return sum(args)\n",
"\n",
"mysum(1, 2, 3)\n",
"\n",
"d = {\n",
" 'my-name': 'hugo'\n",
"}\n",
"\n",
"greet(\"hugo\", \"lopes\", \"tavares\", **d)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## conventions for method names (protected, private, magic)"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"('Hugo',\n",
" ['_Person__get_name',\n",
" '__class__',\n",
" '__delattr__',\n",
" '__dict__',\n",
" '__doc__',\n",
" '__format__',\n",
" '__getattribute__',\n",
" '__hash__',\n",
" '__init__',\n",
" '__module__',\n",
" '__new__',\n",
" '__reduce__',\n",
" '__reduce_ex__',\n",
" '__repr__',\n",
" '__setattr__',\n",
" '__sizeof__',\n",
" '__str__',\n",
" '__subclasshook__',\n",
" '__weakref__'])"
]
},
"execution_count": 41,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class Person(object):\n",
" def __get_name(self): return 'Hugo'\n",
" \n",
"p = Person()\n",
"#p.__get_name()\n",
"p._Person__get_name(), dir(p)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## getattr, hasattr, setattr"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 52,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# p.name -> getattr(p, 'name')\n",
"# hasattr(p, 'name')\n",
"# p.name = 'Hugo' -> setattr(p, 'name', 'Hugo')\n",
"\n",
"class Person2(object):\n",
" def __getattr__(self, attr):\n",
"# raise 1/0\n",
" return 'called with {}'.format(attr)\n",
"p = Person2()\n",
"hasattr(p,'foobar')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## `__dict__`"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'John Doe'"
]
},
"execution_count": 63,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class Employee(object):\n",
" name = 'John'\n",
"\n",
" def __init__(self, name): self.name = name\n",
" def get_salary(self): return float(\"+inf\")\n",
"\n",
" \n",
"e = Employee(\"John Doe\")\n",
"e.name"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## `__call__`"
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"extra msg = 'boooo'\n",
"Hello World\n"
]
}
],
"source": [
"class Callable(object):\n",
" def __init__(self, fn): self.fn = fn\n",
"\n",
" def __call__(self, extra_msg):\n",
" print(\"extra msg = {}\".format(repr(extra_msg)))\n",
" return self.fn()\n",
"\n",
"def show_hello_world():\n",
" print(\"Hello World\")\n",
"\n",
"c = Callable(show_hello_world)\n",
"c(\"boooo\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## decorators"
]
},
{
"cell_type": "code",
"execution_count": 81,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<function __main__.hellofolks2>"
]
},
"execution_count": 81,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def greet(): return 'hugo'\n",
"\n",
"def nicer_greet(fn):\n",
" @wraps(fn)\n",
" def newfn():\n",
" return 'Dear {}'.format(fn())\n",
" return newfn\n",
"\n",
"greet()\n",
"nice = nicer_greet(greet)\n",
"nice()\n",
"\n",
"@nicer_greet\n",
"def hellofolks():\n",
" return 'folks'\n",
"\n",
"def hellofolks2():\n",
" return 'folks'\n",
"\n",
"hellofolks = nicer_greet(hellofolks2)\n",
"\n",
"hellofolks"
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Oh my Dear hugo'"
]
},
"execution_count": 83,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def even_nicer_greet(greet_msg):\n",
" def decorator(fn):\n",
" @wraps(fn)\n",
" def newfn():\n",
" return '{} Dear {}'.format(greet_msg, fn())\n",
" return newfn\n",
" return decorator\n",
"\n",
"@even_nicer_greet(\"Oh my\")\n",
"def hello():\n",
" return 'hugo'\n",
"\n",
"def hello():\n",
" return 'hugo'\n",
"\n",
"hello = even_nicer_greet(\"Oh my\")(hello)\n",
"hello = even_nicer_greet(hello)\n",
"\n",
"\n",
"hello()"
]
},
{
"cell_type": "code",
"execution_count": 77,
"metadata": {},
"outputs": [],
"source": [
"from functools import wraps"
]
},
{
"cell_type": "code",
"execution_count": 78,
"metadata": {},
"outputs": [],
"source": [
"wraps?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## `property`"
]
},
{
"cell_type": "code",
"execution_count": 111,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\"Employee(name='foo')\""
]
},
"execution_count": 111,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class Person(object):\n",
" def __init__(self, name):\n",
" self._name = name\n",
" \n",
" def getter(self): return self._name\n",
" \n",
" def setter(self, value): self._name = value\n",
" \n",
" name = property(getter, setter)\n",
"\n",
"p = Person(\"Hugo\")\n",
"p.name = 'Joe'\n",
"p.name\n",
"\n",
"class MyProp(object):\n",
" def __get__(self, instance, owner): return 'prop'\n",
"\n",
"class Employee2(object):\n",
" name = MyProp()\n",
" \n",
" def __repr__(self):\n",
" return 'Employee(name={})'.format(repr('foo'))\n",
" def __str__(self): return 'fooled you'\n",
"\n",
"e = Employee2()\n",
"type(e.name)\n",
"repr(e)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## `staticmethod`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## `classmethod`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## context managers"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## magic methods"
]
},
{
"cell_type": "code",
"execution_count": 148,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(....., True)"
]
},
"execution_count": 148,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class Number(object):\n",
" def __init__(self, value): self.value = value\n",
" def __add__(self, x):\n",
" return Number(\"{}{}\".format(self.value, x.value))\n",
" def __repr__(self):\n",
" return self.value\n",
" def __eq__(self, other):\n",
" return self.value == other.value\n",
" def __radd__(self, left):\n",
" return self + left\n",
"\n",
"zero = Number(\"\")\n",
"one = Number(\".\")\n",
"two = one + one\n",
"three = two + one\n",
"four = two + two\n",
"five = two + three\n",
"five, (four == (two + two))\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Spreadsheet example\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"\"\"\"\n",
"s = Spreadsheet()\n",
"\n",
"# version 1\n",
"s.set('a1', 1)\n",
"s.set('a2', s.get('a1') + 2)\n",
"s.get('a2') == 3\n",
"s.set('a1', s.get('a1') + 3)\n",
"s.get('a2') == 4\n",
"str(s.get('a1')) == '1'\n",
"str(s.get('a2')) == \"(a1) + 3\"\n",
"\n",
"\n",
"# version 2\n",
"s.a1 = 1\n",
"s.a2 = s.a1 + 2\n",
"s.a2 == 3\n",
"s.a1 = s.a1 + 3\n",
"s.a2 == 4\n",
"str(s.a1) == '1'\n",
"str(s.a2) == \"(a1) + 3\"\n",
"\"\"\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## the `type` class"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## metaclasses\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.15"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment