Last active
March 12, 2018 21:59
-
-
Save asford/bbdd3a8f09e643883d4b545bc94a73db to your computer and use it in GitHub Desktop.
reactive_properties.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "cells": [ | |
| { | |
| "metadata": {}, | |
| "cell_type": "markdown", | |
| "source": "# Reactive Properties\n\nA prototype implementation of \"reactive\" properties in python.\n\nA containing class stores state as properties divided into \"standard\" props, which represent source data, and \"reactive\" props, which are derived from other props.\n\nReactive props are calculated dynamically via a \"pull\" model, accessing a property invokes its definition function to calculate the property value. The value is cached within the containing object and is returned, if available, on the next property access. The definition function may access other properties within the object, which *may* be themselves reactive, resulting in a backward traversal through reactive properties to calculate the required value. DAG-ness is not mandated, but infinite recursion will occur in the case of mutually interdependent property values. Caveat usor.\n\nReative props are invalidated via a \"push\" model. Deleting or changing a property value will invalidate the store value for all declared dependent properties. These dependents may themselves have dependents, resulting in a forward traversal through reactive properties to invalidate all derived property values. This allows recalcuation when the property is next requested.\n\nInter-property dependencies must be *explictly* declared to support forward invalidation. Explict is better than implict, eh?" | |
| }, | |
| { | |
| "metadata": { | |
| "trusted": true | |
| }, | |
| "cell_type": "code", | |
| "source": "from collections import Counter\nimport re\nimport decorator\nimport six\nimport properties", | |
| "execution_count": 1, | |
| "outputs": [] | |
| }, | |
| { | |
| "metadata": { | |
| "trusted": true | |
| }, | |
| "cell_type": "code", | |
| "source": "from toolz import get_in", | |
| "execution_count": 2, | |
| "outputs": [] | |
| }, | |
| { | |
| "metadata": { | |
| "trusted": true | |
| }, | |
| "cell_type": "code", | |
| "source": "class CachedProperty(properties.basic.DynamicProperty):\n def get_property(self):\n scope = self\n\n def fget(self):\n if not scope.name in self._backend:\n value = scope.func(self)\n if value is properties.undefined:\n return None\n value = scope.validate(self, value)\n \n self._set(scope.name, value)\n \n return self._get(scope.name) \n\n def fset(self, value):\n raise AttributeError(\"cannot set attribute\")\n\n def fdel(self):\n self._set(scope.name, properties.undefined)\n\n return property(fget=fget, fset=fset, fdel=fdel, doc=scope.sphinx())\n\ndef cached(prop):\n def _wrap(f):\n return CachedProperty(doc=prop.doc, prop=prop, func=f)\n \n return _wrap\n\nclass DerivedProperty(properties.basic.DynamicProperty):\n @property\n def dependencies(self):\n return getattr(self, \"_dependencies\")\n \n @dependencies.setter\n def dependencies(self, value):\n if not isinstance(value, (tuple, list, set)):\n value = [value]\n for val in value:\n if not isinstance(val, six.string_types):\n raise TypeError('Observed names must be strings')\n self._dependencies = value\n \n def __init__(self, dependencies, doc, func, prop, **kwargs):\n self.dependencies = dependencies\n self.obs = properties.handlers.Observer(self.dependencies, \"observe_set\")(self.clear)\n \n properties.basic.DynamicProperty.__init__(self, doc, func, prop, **kwargs)\n \n def clear(self, instance, _):\n instance.__delattr__(self.name)\n \n def get_property(self):\n scope = self\n \n def fget(self):\n if not scope.name in self._backend:\n value = scope.func(self)\n if value is properties.undefined:\n return None\n value = scope.validate(self, value)\n \n self._set(scope.name, value)\n \n has_observers = all(\n any(\n scope.obs is o \n for o in get_in([name, scope.obs.mode], self._listeners, ())\n )\n for name in scope.obs.names\n )\n \n if not has_observers:\n properties.handlers._set_listener(self, scope.obs)\n \n return self._get(scope.name) \n\n def fset(self, value):\n raise AttributeError(\"cannot set attribute\")\n\n def fdel(self):\n self._set(scope.name, properties.undefined)\n\n return property(fget=fget, fset=fset, fdel=fdel, doc=scope.sphinx())\n \ndef derived_from(dependencies, prop):\n def _wrap(f):\n return DerivedProperty(dependencies, doc=prop.doc, prop=prop, func=f)\n \n return _wrap", | |
| "execution_count": 3, | |
| "outputs": [] | |
| }, | |
| { | |
| "metadata": {}, | |
| "cell_type": "markdown", | |
| "source": "# A Simple Demo" | |
| }, | |
| { | |
| "metadata": { | |
| "trusted": true, | |
| "scrolled": false | |
| }, | |
| "cell_type": "code", | |
| "source": "class TextAnalysis(properties.HasProperties):\n raw_text = properties.String(\"Raw source text\")\n \n @derived_from(\n \"raw_text\",\n properties.String(\"Source text, lowercase normalized\"))\n def norm_text(self):\n print(\"Normalizing text.\")\n return re.sub(\"\\s+\", \" \", re.sub(r'([^\\s\\w]|_)+', '', self.raw_text)).lower()\n \n @derived_from(\n \"norm_text\",\n properties.Dictionary(\"Normalized word counts\"))\n def word_counts(self):\n return Counter(self.norm_text.split())\n \n @derived_from(\n \"norm_text\",\n properties.Dictionary(\"Normalized character counts\"))\n def char_counts(self):\n return Counter(re.sub(\"\\s\", \"\", self.norm_text))\n\nlorem_analysis = TextAnalysis(raw_text=\"\"\"\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc vel suscipit velit, et accumsan diam. Pellentesque felis nulla, maximus ac feugiat eget, consectetur ac urna. Vestibulum sed feugiat neque. Vestibulum molestie dolor non purus sodales facilisis. Nam at tortor vel tortor sodales auctor. Nullam vel facilisis turpis. Quisque vel turpis nec ligula vestibulum euismod. Nam fermentum maximus risus id scelerisque. Maecenas vehicula dolor id tincidunt auctor. Nullam molestie, mauris non aliquet convallis, dui nunc euismod magna, nec tincidunt arcu leo ut eros. In hac habitasse platea dictumst. Praesent vitae dignissim nisl. Proin consequat, est sed commodo ornare, justo ipsum hendrerit turpis, a laoreet diam augue sed neque. Ut tincidunt facilisis justo vitae elementum. In at imperdiet elit, nec pellentesque leo.\"\"\")\n\ndisplay(lorem_analysis.word_counts)\ndisplay(lorem_analysis.char_counts)", | |
| "execution_count": 4, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": "Normalizing text.\n", | |
| "name": "stdout" | |
| }, | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "text/plain": "Counter({'a': 1,\n 'ac': 2,\n 'accumsan': 1,\n 'adipiscing': 1,\n 'aliquet': 1,\n 'amet': 1,\n 'arcu': 1,\n 'at': 2,\n 'auctor': 2,\n 'augue': 1,\n 'commodo': 1,\n 'consectetur': 2,\n 'consequat': 1,\n 'convallis': 1,\n 'diam': 2,\n 'dictumst': 1,\n 'dignissim': 1,\n 'dolor': 3,\n 'dui': 1,\n 'eget': 1,\n 'elementum': 1,\n 'elit': 2,\n 'eros': 1,\n 'est': 1,\n 'et': 1,\n 'euismod': 2,\n 'facilisis': 3,\n 'felis': 1,\n 'fermentum': 1,\n 'feugiat': 2,\n 'habitasse': 1,\n 'hac': 1,\n 'hendrerit': 1,\n 'id': 2,\n 'imperdiet': 1,\n 'in': 2,\n 'ipsum': 2,\n 'justo': 2,\n 'laoreet': 1,\n 'leo': 2,\n 'ligula': 1,\n 'lorem': 1,\n 'maecenas': 1,\n 'magna': 1,\n 'mauris': 1,\n 'maximus': 2,\n 'molestie': 2,\n 'nam': 2,\n 'nec': 3,\n 'neque': 2,\n 'nisl': 1,\n 'non': 2,\n 'nulla': 1,\n 'nullam': 2,\n 'nunc': 2,\n 'ornare': 1,\n 'pellentesque': 2,\n 'platea': 1,\n 'praesent': 1,\n 'proin': 1,\n 'purus': 1,\n 'quisque': 1,\n 'risus': 1,\n 'scelerisque': 1,\n 'sed': 3,\n 'sit': 1,\n 'sodales': 2,\n 'suscipit': 1,\n 'tincidunt': 3,\n 'tortor': 2,\n 'turpis': 3,\n 'urna': 1,\n 'ut': 2,\n 'vehicula': 1,\n 'vel': 4,\n 'velit': 1,\n 'vestibulum': 3,\n 'vitae': 2})" | |
| }, | |
| "metadata": {} | |
| }, | |
| { | |
| "output_type": "display_data", | |
| "data": { | |
| "text/plain": "Counter({'a': 50,\n 'b': 4,\n 'c': 32,\n 'd': 24,\n 'e': 80,\n 'f': 7,\n 'g': 8,\n 'h': 4,\n 'i': 68,\n 'j': 2,\n 'l': 45,\n 'm': 34,\n 'n': 45,\n 'o': 36,\n 'p': 14,\n 'q': 9,\n 'r': 31,\n 's': 58,\n 't': 57,\n 'u': 58,\n 'v': 12,\n 'x': 2})" | |
| }, | |
| "metadata": {} | |
| } | |
| ] | |
| }, | |
| { | |
| "metadata": { | |
| "trusted": true | |
| }, | |
| "cell_type": "code", | |
| "source": "import unittest\n\nclass TestReactiveProperties(unittest.TestCase):\n def test_cached(self):\n class TProp(properties.HasProperties):\n raw_text = properties.String(\"Raw source text\")\n\n @cached(properties.String(\"Source text, lowercase normalized\"))\n def norm_text(self):\n return re.sub(r'([^\\s\\w]|_)+', '', self.raw_text).lower()\n\n @cached(properties.Dictionary(\"Normalized word counts\"))\n def word_counts(self):\n return Counter(self.norm_text.split())\n \n tp = TProp(raw_text=\"foo bar. Bat bazz. Foo bar.\")\n\n self.assertCountEqual(tp._backend.keys(), [\"raw_text\"])\n \n self.assertDictEqual(dict(tp.word_counts), {\"foo\" : 2, \"bar\" : 2, \"bat\" : 1, \"bazz\" : 1})\n self.assertCountEqual(tp._backend.keys(), [\"raw_text\", \"norm_text\", \"word_counts\"])\n \n tp.raw_text = \"Lorum ipsum\"\n self.assertDictEqual(dict(tp.word_counts), {\"foo\" : 2, \"bar\" : 2, \"bat\" : 1, \"bazz\" : 1})\n self.assertCountEqual(tp._backend.keys(), [\"raw_text\", \"norm_text\", \"word_counts\"])\n \n del tp.norm_text\n self.assertDictEqual(dict(tp.word_counts), {\"foo\" : 2, \"bar\" : 2, \"bat\" : 1, \"bazz\" : 1})\n self.assertCountEqual(tp._backend.keys(), [\"raw_text\", \"word_counts\"])\n \n \n def test_derived(self):\n \n class TProp(properties.HasProperties):\n raw_text = properties.String(\"Raw source text\")\n\n @derived_from(\n \"raw_text\",\n properties.String(\"Source text, lowercase normalized\"))\n def norm_text(self):\n return re.sub(r'([^\\s\\w]|_)+', '', self.raw_text).lower()\n\n @derived_from(\n \"norm_text\",\n properties.Dictionary(\"Normalized word counts\"))\n def word_counts(self):\n return Counter(self.norm_text.split())\n\n tp = TProp(raw_text=\"foo bar. Bat bazz. Foo bar.\")\n\n self.assertCountEqual(tp._backend.keys(), [\"raw_text\"])\n self.assertEqual(tp._listeners, {})\n\n self.assertDictEqual(dict(tp.word_counts), {\"foo\" : 2, \"bar\" : 2, \"bat\" : 1, \"bazz\" : 1})\n\n self.assertCountEqual(tp._backend.keys(), [\"raw_text\", \"norm_text\", \"word_counts\"])\n self.assertEqual(len(get_in([\"raw_text\", \"observe_set\"], tp._listeners)), 1)\n self.assertEqual(len(get_in([\"norm_text\", \"observe_set\"], tp._listeners)), 1)\n\n tp.raw_text = \"Lorum ipsum\"\n self.assertCountEqual(tp._backend.keys(), [\"raw_text\"])\n self.assertEqual(len(get_in([\"raw_text\", \"observe_set\"], tp._listeners)), 1)\n self.assertEqual(len(get_in([\"norm_text\", \"observe_set\"], tp._listeners)), 1)\n\n self.assertDictEqual(dict(tp.word_counts), {\"lorum\" : 1, \"ipsum\" : 1})\n\n self.assertCountEqual(tp._backend.keys(), [\"raw_text\", \"norm_text\", \"word_counts\"])\n self.assertEqual(len(get_in([\"raw_text\", \"observe_set\"], tp._listeners)), 1)\n self.assertEqual(len(get_in([\"norm_text\", \"observe_set\"], tp._listeners)), 1)", | |
| "execution_count": 5, | |
| "outputs": [] | |
| }, | |
| { | |
| "metadata": { | |
| "trusted": true | |
| }, | |
| "cell_type": "code", | |
| "source": "TestReactiveProperties(\"test_cached\").debug()\nTestReactiveProperties(\"test_derived\").debug()", | |
| "execution_count": 6, | |
| "outputs": [] | |
| } | |
| ], | |
| "metadata": { | |
| "_draft": { | |
| "nbviewer_url": "https://gist.github.com/bbdd3a8f09e643883d4b545bc94a73db" | |
| }, | |
| "gist": { | |
| "id": "bbdd3a8f09e643883d4b545bc94a73db", | |
| "data": { | |
| "description": "reactive_properties.ipynb", | |
| "public": true | |
| } | |
| }, | |
| "kernelspec": { | |
| "name": "conda-env-tmol-py", | |
| "display_name": "Python [conda env:tmol]", | |
| "language": "python" | |
| }, | |
| "language_info": { | |
| "name": "python", | |
| "mimetype": "text/x-python", | |
| "codemirror_mode": { | |
| "name": "ipython", | |
| "version": 3 | |
| }, | |
| "nbconvert_exporter": "python", | |
| "version": "3.5.4", | |
| "file_extension": ".py", | |
| "pygments_lexer": "ipython3" | |
| }, | |
| "toc": { | |
| "nav_menu": {}, | |
| "number_sections": true, | |
| "sideBar": true, | |
| "skip_h1_title": false, | |
| "title_cell": "Table of Contents", | |
| "title_sidebar": "Contents", | |
| "toc_cell": false, | |
| "toc_position": {}, | |
| "toc_section_display": true, | |
| "toc_window_display": false | |
| } | |
| }, | |
| "nbformat": 4, | |
| "nbformat_minor": 2 | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment