Created
June 3, 2020 22:51
-
-
Save Ben-Epstein/1ad1861af051be3a433971b1db2bc99a to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"cells":[{"metadata":{},"cell_type":"markdown","source":"<style>\n#s {\n}\nh1, h2, h3, h4, h5, h6, table, button, a, p, blockquote {\nfont-family:Geneva;\n}\n\n.log {\ntransition: all .2s ease-in-out;\n}\n\n.log:hover {a\ntransform: scale(1.05);\n}\n</style>\n<div id='s' style='width:90%'>\n<center><img class='log' src='https://splicemachine.com/wp-content/uploads/splice-logo-1.png' width='20%' style='z-index:5'></center>\n<center><h1 class='log' style='font-size:40px; color:black;'>Welcome to Splice Machine MLManager</h1></center>\n<center><h2 class = 'log' style='font-size:25px; color:grey;'>The data platform for intelligent applications</center>\n<center><img class='log' src='https://splice-demo.s3.amazonaws.com/splice-machine-data-science-process-h2o.png' width='40%' style='z-index:5'></center>\n</div>\n \n "},{"metadata":{},"cell_type":"markdown","source":"# In this notebook, we're going to take a look at using MLManager with [H2O](https://www.h2o.ai/) + [Spark](https://spark.apache.org/)\n<h2 style='font-size:25px; font-weight:bold'>What is <a href=http://docs.h2o.ai/sparkling-water/2.1/latest-stable/doc/pysparkling.html>PySparkling Water?</a> What is <a href=https://splicemachine.com/product/ml-manager/>MLManager?</a></h2>\n<style>\nblockquote{\n font-size: 15px;\n background: #f9f9f9;\n border-left: 10px solid #ccc;\n margin: .5em 10px;\n padding: 30em, 10px;\n quotes: \"\\201C\"\"\\201D\"\"\\2018\"\"\\2019\";\n padding: 10px 20px;\n line-height: 1.4;\n}\n\nblockquote:before {\n content: open-quote;\n display: inline;\n height: 0;\n line-height: 0;\n left: -10px;\n position: relative;\n top: 30px;\n bottom:30px;\n color: #ccc;\n font-size: 3em;\n display:none;\n\n}\n\np{\n margin: 0;\n}\n\nfooter{\n margin:0;\n text-align: right;\n font-size: 1em;\n font-style: italic;\n}\n</style>\n<blockquote><p class='quotation'><b><br><span style='font-size:25px'>PySparkling</span></b> <br><br>PySparkling Water is an awesome H2O extension that allows you to run H2O clusters on top of existing Spark clusters. With Splice Machine, this integration is taken care of for you, so it's simple to start modeling with your new favorite library</i></br><footer>Splice Machine</footer></blockquote><br>\n<blockquote><p class='quotation'><b><br><span style='font-size:25px'>MLManager (+MLFlow)</span></b><br><br>As a data scientist constantly creating new models and testing new features, it is necessary to effectively track and manage those different ML runs. MLManager + MLFlow allows you to track entire <code>experiments</code> and individual <code>run</code> parameters and metrics. The way you organize your flow is unique to you, and the intuitive Python API allows you to organize your development process and run with it.<br>\n <center><img class='log' src='https://s3.amazonaws.com/splice-demo/mlflow+ui.png' width='40%' style='z-index:5'></center>"},{"metadata":{},"cell_type":"markdown","source":"# Let's get started\n## In this notebook, we will see how to use Spark, H2O and MLManager to predict sentiment analysis of Amazon reviews, tracking everything in the [MLFlow UI](/mlflow) and deploy our models to production\nThis is an adaptation of the original [H2O Demo](http://docs.h2o.ai/h2o-tutorials/latest-stable/h2o-world-2017/nlp/index.html)"},{"metadata":{},"cell_type":"markdown","source":"## Important imports and setup\n* Create our Spark Session\n* Create our Native Spark Data Source\n* Create our PySparkling Water cluster\n* Import our MLManager functionality"},{"metadata":{"scrolled":true,"trusted":false},"cell_type":"code","source":"pip install --upgrade git+https://www.github.com/splicemachine/pysplice@DBAAS-3990","execution_count":3,"outputs":[{"name":"stdout","output_type":"stream","text":"Collecting git+https://www.github.com/splicemachine/pysplice@DBAAS-3990\n Cloning https://www.github.com/splicemachine/pysplice (to revision DBAAS-3990) to /tmp/pip-req-build-fbn95kw6\n Running command git clone -q https://www.github.com/splicemachine/pysplice /tmp/pip-req-build-fbn95kw6\n Running command git checkout -b DBAAS-3990 --track origin/DBAAS-3990\n Switched to a new branch 'DBAAS-3990'\n Branch 'DBAAS-3990' set up to track remote branch 'DBAAS-3990' from 'origin'.\nRequirement already satisfied, skipping upgrade: py4j==0.10.8.1 in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (0.10.8.1)\nCollecting pytest==5.1.3\n Downloading pytest-5.1.3-py3-none-any.whl (224 kB)\n\u001b[K |████████████████████████████████| 224 kB 9.3 MB/s eta 0:00:01\n\u001b[?25hRequirement already satisfied, skipping upgrade: mlflow==1.6.0 in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (1.6.0)\nRequirement already satisfied, skipping upgrade: mleap==0.15.0 in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (0.15.0)\nRequirement already satisfied, skipping upgrade: graphviz==0.13 in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (0.13)\nRequirement already satisfied, skipping upgrade: requests in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (2.23.0)\nRequirement already satisfied, skipping upgrade: gorilla==0.3.0 in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (0.3.0)\nRequirement already satisfied, skipping upgrade: tqdm==4.43.0 in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (4.43.0)\nRequirement already satisfied, skipping upgrade: pyspark-dist-explore==0.1.8 in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (0.1.8)\nRequirement already satisfied, skipping upgrade: numpy==1.18.2 in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (1.18.2)\nRequirement already satisfied, skipping upgrade: pandas==1.0.3 in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (1.0.3)\nRequirement already satisfied, skipping upgrade: scipy==1.4.1 in /opt/conda/lib/python3.7/site-packages (from splicemachine==2.0.0) (1.4.1)\nRequirement already satisfied, skipping upgrade: wcwidth in /opt/conda/lib/python3.7/site-packages (from pytest==5.1.3->splicemachine==2.0.0) (0.2.3)\nRequirement already satisfied, skipping upgrade: py>=1.5.0 in /opt/conda/lib/python3.7/site-packages (from pytest==5.1.3->splicemachine==2.0.0) (1.8.1)\nRequirement already satisfied, skipping upgrade: more-itertools>=4.0.0 in /opt/conda/lib/python3.7/site-packages (from pytest==5.1.3->splicemachine==2.0.0) (8.3.0)\nRequirement already satisfied, skipping upgrade: attrs>=17.4.0 in /opt/conda/lib/python3.7/site-packages (from pytest==5.1.3->splicemachine==2.0.0) (19.3.0)\nRequirement already satisfied, skipping upgrade: packaging in /opt/conda/lib/python3.7/site-packages (from pytest==5.1.3->splicemachine==2.0.0) (20.4)\nRequirement already satisfied, skipping upgrade: atomicwrites>=1.0 in /opt/conda/lib/python3.7/site-packages (from pytest==5.1.3->splicemachine==2.0.0) (1.4.0)\nRequirement already satisfied, skipping upgrade: pluggy<1.0,>=0.12 in /opt/conda/lib/python3.7/site-packages (from pytest==5.1.3->splicemachine==2.0.0) (0.13.1)\nRequirement already satisfied, skipping upgrade: importlib-metadata>=0.12; python_version < \"3.8\" in /opt/conda/lib/python3.7/site-packages (from pytest==5.1.3->splicemachine==2.0.0) (1.6.0)\nRequirement already satisfied, skipping upgrade: sqlalchemy in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (1.3.8)\nRequirement already satisfied, skipping upgrade: python-dateutil in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (2.8.1)\nRequirement already satisfied, skipping upgrade: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (7.1.2)\nRequirement already satisfied, skipping upgrade: gitpython>=2.1.0 in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (3.1.3)\nRequirement already satisfied, skipping upgrade: sqlparse in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (0.3.1)\nRequirement already satisfied, skipping upgrade: cloudpickle in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (1.2.2)\nRequirement already satisfied, skipping upgrade: alembic in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (1.4.2)\nRequirement already satisfied, skipping upgrade: querystring-parser in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (1.2.4)\nRequirement already satisfied, skipping upgrade: Flask in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (1.1.2)\nRequirement already satisfied, skipping upgrade: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (3.9.1)\nRequirement already satisfied, skipping upgrade: databricks-cli>=0.8.7 in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (0.11.0)\nRequirement already satisfied, skipping upgrade: entrypoints in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (0.3)\nRequirement already satisfied, skipping upgrade: six>=1.10.0 in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (1.15.0)\nRequirement already satisfied, skipping upgrade: pyyaml in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (5.3.1)\nRequirement already satisfied, skipping upgrade: docker>=4.0.0 in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (4.2.1)\nRequirement already satisfied, skipping upgrade: prometheus-flask-exporter in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (0.13.0)\nRequirement already satisfied, skipping upgrade: simplejson in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (3.17.0)\nRequirement already satisfied, skipping upgrade: gunicorn; platform_system != \"Windows\" in /opt/conda/lib/python3.7/site-packages (from mlflow==1.6.0->splicemachine==2.0.0) (20.0.4)\nRequirement already satisfied, skipping upgrade: scikit-learn>=0.18.dev0 in /opt/conda/lib/python3.7/site-packages (from mleap==0.15.0->splicemachine==2.0.0) (0.21.3)\nRequirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->splicemachine==2.0.0) (3.0.4)\nRequirement already satisfied, skipping upgrade: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->splicemachine==2.0.0) (2.9)\nRequirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests->splicemachine==2.0.0) (2020.4.5.1)\nRequirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->splicemachine==2.0.0) (1.25.9)\nRequirement already satisfied, skipping upgrade: matplotlib in /opt/conda/lib/python3.7/site-packages (from pyspark-dist-explore==0.1.8->splicemachine==2.0.0) (3.1.1)\nRequirement already satisfied, skipping upgrade: pytz>=2017.2 in /opt/conda/lib/python3.7/site-packages (from pandas==1.0.3->splicemachine==2.0.0) (2020.1)\nRequirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->pytest==5.1.3->splicemachine==2.0.0) (2.4.7)\nRequirement already satisfied, skipping upgrade: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata>=0.12; python_version < \"3.8\"->pytest==5.1.3->splicemachine==2.0.0) (3.1.0)\nRequirement already satisfied, skipping upgrade: gitdb<5,>=4.0.1 in /opt/conda/lib/python3.7/site-packages (from gitpython>=2.1.0->mlflow==1.6.0->splicemachine==2.0.0) (4.0.5)\nRequirement already satisfied, skipping upgrade: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->mlflow==1.6.0->splicemachine==2.0.0) (1.0.4)\nRequirement already satisfied, skipping upgrade: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->mlflow==1.6.0->splicemachine==2.0.0) (1.1.0)\nRequirement already satisfied, skipping upgrade: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from Flask->mlflow==1.6.0->splicemachine==2.0.0) (2.11.2)\nRequirement already satisfied, skipping upgrade: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from Flask->mlflow==1.6.0->splicemachine==2.0.0) (1.0.1)\nRequirement already satisfied, skipping upgrade: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from Flask->mlflow==1.6.0->splicemachine==2.0.0) (1.1.0)\nRequirement already satisfied, skipping upgrade: setuptools in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.0->mlflow==1.6.0->splicemachine==2.0.0) (47.1.1.post20200529)\n"},{"name":"stdout","output_type":"stream","text":"Requirement already satisfied, skipping upgrade: tabulate>=0.7.7 in /opt/conda/lib/python3.7/site-packages (from databricks-cli>=0.8.7->mlflow==1.6.0->splicemachine==2.0.0) (0.8.7)\nRequirement already satisfied, skipping upgrade: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker>=4.0.0->mlflow==1.6.0->splicemachine==2.0.0) (0.57.0)\nRequirement already satisfied, skipping upgrade: prometheus-client in /opt/conda/lib/python3.7/site-packages (from prometheus-flask-exporter->mlflow==1.6.0->splicemachine==2.0.0) (0.8.0)\nRequirement already satisfied, skipping upgrade: joblib>=0.11 in /opt/conda/lib/python3.7/site-packages (from scikit-learn>=0.18.dev0->mleap==0.15.0->splicemachine==2.0.0) (0.15.1)\nRequirement already satisfied, skipping upgrade: cycler>=0.10 in /opt/conda/lib/python3.7/site-packages (from matplotlib->pyspark-dist-explore==0.1.8->splicemachine==2.0.0) (0.10.0)\nRequirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib->pyspark-dist-explore==0.1.8->splicemachine==2.0.0) (1.2.0)\nRequirement already satisfied, skipping upgrade: smmap<4,>=3.0.1 in /opt/conda/lib/python3.7/site-packages (from gitdb<5,>=4.0.1->gitpython>=2.1.0->mlflow==1.6.0->splicemachine==2.0.0) (3.0.4)\nRequirement already satisfied, skipping upgrade: MarkupSafe>=0.9.2 in /opt/conda/lib/python3.7/site-packages (from Mako->alembic->mlflow==1.6.0->splicemachine==2.0.0) (1.1.1)\nBuilding wheels for collected packages: splicemachine\n Building wheel for splicemachine (setup.py) ... \u001b[?25ldone\n\u001b[?25h Created wheel for splicemachine: filename=splicemachine-2.0.0-py3-none-any.whl size=54439 sha256=e7604f8e3bc139361bd06316074976078f66e39aa4b0f4949235929a21d1a00e\n Stored in directory: /tmp/pip-ephem-wheel-cache-4vcdsiuk/wheels/0c/58/ec/d4dd5d3e14310dcd82c132198196b6973a4139393dee4ddd79\nSuccessfully built splicemachine\nInstalling collected packages: pytest, splicemachine\n Attempting uninstall: pytest\n Found existing installation: pytest 5.4.3\n Uninstalling pytest-5.4.3:\n Successfully uninstalled pytest-5.4.3\n Attempting uninstall: splicemachine\n Found existing installation: splicemachine 2.0.0\n Uninstalling splicemachine-2.0.0:\n Successfully uninstalled splicemachine-2.0.0\nSuccessfully installed pytest-5.1.3 splicemachine-2.0.0\nNote: you may need to restart the kernel to use updated packages.\n"}]},{"metadata":{"scrolled":true,"trusted":false},"cell_type":"code","source":"from pyspark.sql import SparkSession\nfrom splicemachine.spark import PySpliceContext\nfrom splicemachine.mlflow_support import *\nfrom pysparkling import *\nimport h2o\n\n# Spark Session\nspark = SparkSession.builder.config('spark.driver.memoryOverhead',1000).config('spark.driver.memory','2g').getOrCreate()\n#spark.scheduler.minRegisteredResourcesRatio=1\n# Native Spark Data Source\nsplice = PySpliceContext(spark)\n# Register Splice so we can access database functions\nmlflow.register_splice_context(splice)\n# Create H2O Cluster\nconf = H2OConf().setInternalClusterMode()\nhc = H2OContext.getOrCreate(conf)","execution_count":1,"outputs":[{"name":"stdout","output_type":"stream","text":"Connecting to H2O server at http://10.128.24.161:54321 ... successful.\n"},{"data":{"text/html":"<div style=\"overflow:auto\"><table style=\"width:50%\"><tr><td>H2O cluster uptime:</td>\n<td>09 secs</td></tr>\n<tr><td>H2O cluster timezone:</td>\n<td>UTC</td></tr>\n<tr><td>H2O data parsing timezone:</td>\n<td>UTC</td></tr>\n<tr><td>H2O cluster version:</td>\n<td>3.28.1.2</td></tr>\n<tr><td>H2O cluster version age:</td>\n<td>2 months and 17 days </td></tr>\n<tr><td>H2O cluster name:</td>\n<td>sparkling-water-jovyan_spark-application-1591213257609</td></tr>\n<tr><td>H2O cluster total nodes:</td>\n<td>1</td></tr>\n<tr><td>H2O cluster free memory:</td>\n<td>3.833 Gb</td></tr>\n<tr><td>H2O cluster total cores:</td>\n<td>16</td></tr>\n<tr><td>H2O cluster allowed cores:</td>\n<td>5</td></tr>\n<tr><td>H2O cluster status:</td>\n<td>locked, healthy</td></tr>\n<tr><td>H2O connection url:</td>\n<td>http://10.128.24.161:54321</td></tr>\n<tr><td>H2O connection proxy:</td>\n<td>None</td></tr>\n<tr><td>H2O internal security:</td>\n<td>False</td></tr>\n<tr><td>H2O API Extensions:</td>\n<td>XGBoost, Algos, Amazon S3, Sparkling Water REST API Extensions, AutoML, Core V3, TargetEncoder, Core V4</td></tr>\n<tr><td>Python version:</td>\n<td>3.7.6 final</td></tr></table></div>","text/plain":"-------------------------- -------------------------------------------------------------------------------------------------------\nH2O cluster uptime: 09 secs\nH2O cluster timezone: UTC\nH2O data parsing timezone: UTC\nH2O cluster version: 3.28.1.2\nH2O cluster version age: 2 months and 17 days\nH2O cluster name: sparkling-water-jovyan_spark-application-1591213257609\nH2O cluster total nodes: 1\nH2O cluster free memory: 3.833 Gb\nH2O cluster total cores: 16\nH2O cluster allowed cores: 5\nH2O cluster status: locked, healthy\nH2O connection url: http://10.128.24.161:54321\nH2O connection proxy:\nH2O internal security: False\nH2O API Extensions: XGBoost, Algos, Amazon S3, Sparkling Water REST API Extensions, AutoML, Core V3, TargetEncoder, Core V4\nPython version: 3.7.6 final\n-------------------------- -------------------------------------------------------------------------------------------------------"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nSparkling Water Context:\n * Sparkling Water Version: 3.28.1.2-1-2.4\n * H2O name: sparkling-water-jovyan_spark-application-1591213257609\n * cluster size: 1\n * list of used nodes:\n (executorId, host, port)\n ------------------------\n (1,10.128.26.72,54321)\n ------------------------\n\n Open H2O Flow in browser: http://10.128.24.161:54321 (CMD + click in Mac OSX)\n\n \n"}]},{"metadata":{},"cell_type":"markdown","source":"# Great! Now let's import our data\n<style>\nblockquote{\n font-size: 15px;\n background: #f9f9f9;\n border-left: 10px solid #ccc;\n margin: .5em 10px;\n padding: 30em, 10px;\n quotes: \"\\201C\"\"\\201D\"\"\\2018\"\"\\2019\";\n padding: 10px 20px;\n line-height: 1.4;\n}\n\nblockquote:before {\n content: open-quote;\n display: inline;\n height: 0;\n line-height: 0;\n left: -10px;\n position: relative;\n top: 30px;\n bottom:30px;\n color: #ccc;\n font-size: 3em;\n display:none;\n\n}\n\np{\n margin: 0;\n}\n\nfooter{\n margin:0;\n text-align: right;\n font-size: 1em;\n font-style: italic;\n}\n</style>\n<blockquote><p class='quotation'><b><br><span style='font-size:25px'>Importing Data</span></b> <br><br>There are a few easy ways to get data into Splice Machine, and we'll demonstrate 2 of them here. You can use the built-in <code>%%sql</code> magic to import data directly from external sources, such as S3, or you can use H2O to directly read the data from S3, create a table from that dataframe, and insert the data directly using the <code>PySpliceContext</code> you created in the cell above. </i></br><footer>Splice Machine</footer></blockquote><br>"},{"metadata":{},"cell_type":"markdown","source":"## Option 1: Direct Import from SQL\n<style>\nblockquote{\n font-size: 15px;\n background: #f9f9f9;\n border-left: 10px solid #ccc;\n margin: .5em 10px;\n padding: 30em, 10px;\n quotes: \"\\201C\"\"\\201D\"\"\\2018\"\"\\2019\";\n padding: 10px 20px;\n line-height: 1.4;\n}\n\nblockquote:before {\n content: open-quote;\n display: inline;\n height: 0;\n line-height: 0;\n left: -10px;\n position: relative;\n top: 30px;\n bottom:30px;\n color: #ccc;\n font-size: 3em;\n display:none;\n\n}\n\np{\n margin: 0;\n}\n\nfooter{\n margin:0;\n text-align: right;\n font-size: 1em;\n font-style: italic;\n}\n</style>\n<blockquote><p class='quotation'><b><br><span style='font-size:25px'>SQL Import</span></b> <br><br>This method is simple: Create your table, point it to a an S3 location, and run the import command</i></br><footer>Splice Machine</footer></blockquote><br>"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%sql\nDROP TABLE IF EXISTS AMAZON_REVIEWS;\nCREATE TABLE AMAZON_REVIEWS(\n PRODUCTID VARCHAR(250),\n USERID VARCHAR(250),\n SUMMARY VARCHAR(500),\n SCORE INT,\n HELPFULNESSDENOMINATOR BIGINT,\n ID INT,\n PROFILENAME VARCHAR(500),\n HELPFULNESSNUMERATOR BIGINT,\n REVIEW_TIME BIGINT,\n REVIEW VARCHAR(15000),\n PRIMARY KEY(ID)\n);\n\n\n-- Import the data\ncall SYSCS_UTIL.IMPORT_DATA (\n null,\n 'AMAZON_REVIEWS',\n null,\n 's3a://splice-demo/AmazonReviews.csv',\n ',',\n null,\n null,\n null,\n null,\n -1,\n 's3a://splice-demo/bad',\n null, \n null);","execution_count":87,"outputs":[{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"ac47520b-4fe1-40ee-89ca-60d7bcce515d","version_major":2}},"metadata":{}},{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"815f20e0-d3f9-4685-969b-06c0aa5220ed","version_major":2}},"metadata":{}},{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"f72ac661-b8cd-4649-bee8-7b5d82ade17e","version_major":2}},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%sql\nselect top 10 * from AMAZON_REVIEWS","execution_count":81,"outputs":[{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"d6e9ab38-cf68-4a11-9ed7-e055ba959df5","version_major":2}},"metadata":{}}]},{"metadata":{"scrolled":true,"trusted":false},"cell_type":"code","source":"# Get data from table into Spark Dataframe\ndf2 = splice.df('select * from REPLACE_ME_DBSCHEMA.amazon_reviews')\nhdf = hc.asH2OFrame(df2)\nhdf.describe()","execution_count":5,"outputs":[{"name":"stdout","output_type":"stream","text":"Rows:99999\nCols:10\n\n\n"},{"data":{"text/html":"<table>\n<thead>\n<tr><th> </th><th>PRODUCTID </th><th>USERID </th><th>SUMMARY </th><th>SCORE </th><th>HELPFULNESSDENOMINATOR </th><th>ID </th><th>PROFILENAME </th><th>HELPFULNESSNUMERATOR </th><th>REVIEW_TIME </th><th>REVIEW </th></tr>\n</thead>\n<tbody>\n<tr><td>type </td><td>string </td><td>string </td><td>string </td><td>int </td><td>int </td><td>int </td><td>string </td><td>int </td><td>int </td><td>string </td></tr>\n<tr><td>mins </td><td>NaN </td><td>NaN </td><td>NaN </td><td>1.0 </td><td>0.0 </td><td>3.0 </td><td>NaN </td><td>0.0 </td><td>940809600.0 </td><td>NaN </td></tr>\n<tr><td>mean </td><td>NaN </td><td>NaN </td><td>NaN </td><td>4.1861018610186 </td><td>2.2364723647236353 </td><td>284618.29231292347</td><td>NaN </td><td>1.7454674546745361 </td><td>1296171870.326685</td><td>NaN </td></tr>\n<tr><td>maxs </td><td>NaN </td><td>NaN </td><td>NaN </td><td>5.0 </td><td>878.0 </td><td>568436.0 </td><td>NaN </td><td>866.0 </td><td>1351209600.0 </td><td>NaN </td></tr>\n<tr><td>sigma </td><td>NaN </td><td>NaN </td><td>NaN </td><td>1.309542187819563</td><td>8.805400733729053 </td><td>164159.3591659324 </td><td>NaN </td><td>8.171450255812838 </td><td>48107386.40835964</td><td>NaN </td></tr>\n<tr><td>zeros </td><td>0 </td><td>0 </td><td>0 </td><td>0 </td><td>47593 </td><td>0 </td><td>0 </td><td>53553 </td><td>0 </td><td>0 </td></tr>\n<tr><td>missing</td><td>0 </td><td>0 </td><td>0 </td><td>0 </td><td>0 </td><td>0 </td><td>0 </td><td>0 </td><td>0 </td><td>0 </td></tr>\n<tr><td>0 </td><td>B000TO7U64 </td><td>A3CM2BMS75UZ4U</td><td>walden pancake syrup review </td><td>5.0 </td><td>0.0 </td><td>275757.0 </td><td>Ronnie Stewart </td><td>0.0 </td><td>1249171200.0 </td><td>for a sugar free syrup i found this to be very good. let some frindes at work try it they liked it to. </td></tr>\n<tr><td>1 </td><td>B002ANCCK6 </td><td>APM0IV2TBRW1A </td><td>Pet Food </td><td>5.0 </td><td>1.0 </td><td>5717.0 </td><td>PetFood </td><td>1.0 </td><td>1306454400.0 </td><td>My finicky calico just LOVES this food. One of the few wet foods she will actually eat. It is loved by our other three cats as well. Will definitely be buying this again. </td></tr>\n<tr><td>2 </td><td>B0014ATRV8 </td><td>A1WZY9GOEE7IAZ</td><td>Very Good! </td><td>5.0 </td><td>0.0 </td><td>530678.0 </td><td>alidep </td><td>0.0 </td><td>1309219200.0 </td><td>The tea taste is very good, I've taken it yesterday for the first time. I'm taking it because of my pregnancy, many friends told me about its precious toning for the uterus. </td></tr>\n<tr><td>3 </td><td>B00141QYSQ </td><td>A1YS02UZZGRDCT</td><td>Do Not Buy </td><td>1.0 </td><td>2.0 </td><td>41471.0 </td><td>Evan Eberhardt </td><td>2.0 </td><td>1348358400.0 </td><td>These are made in China (do not buy ANY pet food from China). Dogswell has been using propylene glycol to soften their treats (what are they thinkng?). Do not purchase or support this company in any way until they clean up their act. And for whatever reason Amazon doesn't allow returns of this item, so I had to toss mine out. Bad business all around on this one. </td></tr>\n<tr><td>4 </td><td>B002LVA88U </td><td>A1OGO9DYD7D6PN</td><td>Tasty Little Jelly Beans </td><td>4.0 </td><td>3.0 </td><td>299328.0 </td><td>Erin </td><td>3.0 </td><td>1318723200.0 </td><td>I used these Jelly Beans to help me train for my first marathon. They taste good and seem to work although I did not seem to get as much of a boost as some of my friends say they have. I would only eat 2-4 at a time so maybe I need to eat a few more to get the full energy boost.<br /><br />I agree with some of the other reviewers in that, although the little bag is resealable, it is still sometimes hard to open while you are jogging/running/walking.<br /><br />Nice product. </td></tr>\n<tr><td>5 </td><td>B001E5DYU8 </td><td>AS3T5DITRJFLS </td><td>Some of the best espresso out there! </td><td>5.0 </td><td>2.0 </td><td>172934.0 </td><td>James Petree "Paladin"</td><td>2.0 </td><td>1186617600.0 </td><td>I bought this to come with my Capresso 114 machine, since I hadn't used pods before, and I was very pleased! The aroma is wonderful, these pods make excellent crema and the flavor is outstanding! I'm just glad I bought 2 tins because one is already empty. </td></tr>\n<tr><td>6 </td><td>B000E63L6U </td><td>A2SCW8P8HFJFKU</td><td>this is not the old Celestial Seasonings Red Zinger! </td><td>1.0 </td><td>11.0 </td><td>460241.0 </td><td>Ann Cardinal </td><td>11.0 </td><td>1237680000.0 </td><td>I was so excited to find my old favorite tea on amazon. I have been looking for it in supermarkets for months, but thought it had been discontinued. However I wish I had listened to the earlier review. This is not the same formula as it was, the peppermint overpowers all the other flavors in this tea. It used to have a citrusy flavor, tangy and lovely, now all you can smell or taste is peppermint and I don't like peppermint tea. </td></tr>\n<tr><td>7 </td><td>B001EO69SS </td><td>A19Q6EHPUDOMBE</td><td>Nuitrition not included. </td><td>1.0 </td><td>2.0 </td><td>335387.0 </td><td>Voltaire </td><td>2.0 </td><td>1321747200.0 </td><td>I purchased this thinking "the Italians are good with food, this should be delicious." If you know anything about food, it looses its nutrition the more it's cooked. This product comes pre-cooked, par-boiled. Black rice is a rich, nutty, food that is so filling it's like a whole meal. Foodies love it because it balance out the usual plate of protein, vegetables, and starch with flavorful starch. It can be eaten alone, with milk as a breakfast, with beans in the forest or as a centerpiece of exotic meal.<br />However, the reason we love this stuff is because it's high in vitamins. Not this stuff. Par-boiling then drying it out destroys much of the vitamins and flavor.<br /><br />Stay far, far away if you want vitamins and flavor.</td></tr>\n<tr><td>8 </td><td>B002QWHJOU </td><td>ARWGHMIEH0IRP </td><td>My puppies love them and it helps keep their teeth clean!</td><td>5.0 </td><td>0.0 </td><td>329647.0 </td><td>EddieT </td><td>0.0 </td><td>1329350400.0 </td><td>Got these large Greenies originally when we noticed our 60+lb Boxer mix puppies' teeth were getting a little dirty and brushing didn't too much. We give them each one Greenie in the morning when we leave for work, they love them. The boy gobbles it down, but his sister savors every bit. They also keep the buildup on their teeth down quite a bit, they're not going to make them shine, but they'll keep them pretty clean! </td></tr>\n<tr><td>9 </td><td>B0007A0AQM </td><td>A29OX3XJ0QDI02</td><td>Doesn't like </td><td>1.0 </td><td>6.0 </td><td>143590.0 </td><td>Mocha272 </td><td>4.0 </td><td>1316044800.0 </td><td>My dog refuses to eat it and it smells pretty bad. I can't recommend it as treat when my dog refuses to touch it. </td></tr>\n</tbody>\n</table>"},"metadata":{},"output_type":"display_data"}]},{"metadata":{},"cell_type":"markdown","source":"## Option 2: Import from H2O\n<style>\nblockquote{\n font-size: 15px;\n background: #f9f9f9;\n border-left: 10px solid #ccc;\n margin: .5em 10px;\n padding: 30em, 10px;\n quotes: \"\\201C\"\"\\201D\"\"\\2018\"\"\\2019\";\n padding: 10px 20px;\n line-height: 1.4;\n}\n\nblockquote:before {\n content: open-quote;\n display: inline;\n height: 0;\n line-height: 0;\n left: -10px;\n position: relative;\n top: 30px;\n bottom:30px;\n color: #ccc;\n font-size: 3em;\n display:none;\n\n}\n\np{\n margin: 0;\n}\n\nfooter{\n margin:0;\n text-align: right;\n font-size: 1em;\n font-style: italic;\n}\n</style>\n<blockquote><p class='quotation'><b><br><span style='font-size:25px'>H2O Import</span></b> <br><br>This method is also straightforward, and may be preferable to Data Scientists: Import your data using H2O, and then use the <code>PySpliceContext</code> to create the table from the dataframe and insert the data directly. You'll notice that this method doesn't directly involve any SQL.</i></br><footer>Splice Machine</footer></blockquote><br>"},{"metadata":{"scrolled":true,"trusted":false},"cell_type":"code","source":"data_path = \"https://splice-demo.s3.amazonaws.com/AmazonReviews.csv\"\n# Load data into H2O\nreviews = h2o.import_file(data_path)\nreviews.head()","execution_count":6,"outputs":[{"name":"stdout","output_type":"stream","text":"Parse progress: |█████████████████████████████████████████████████████████| 100%\n"},{"data":{"text/html":"<table>\n<thead>\n<tr><th>ProductId </th><th>UserId </th><th>Summary </th><th style=\"text-align: right;\"> Score</th><th style=\"text-align: right;\"> HelpfulnessDenominator</th><th style=\"text-align: right;\"> Id</th><th>ProfileName </th><th style=\"text-align: right;\"> HelpfulnessNumerator</th><th style=\"text-align: right;\"> Time</th><th>Text </th></tr>\n</thead>\n<tbody>\n<tr><td>B00141QYSQ </td><td>A1YS02UZZGRDCT</td><td>Do Not Buy </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 41471</td><td>Evan Eberhardt </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\">1.34836e+09</td><td>These are made in China (do not buy ANY pet food from China). Dogswell has been using propylene glycol to soften their treats (what are they thinkng?). Do not purchase or support this company in any way until they clean up their act. And for whatever reason Amazon doesn't allow returns of this item, so I had to toss mine out. Bad business all around on this one. </td></tr>\n<tr><td>B0089SPEO2 </td><td>A3JOYNYL458QHP</td><td>Less lemon and less zing </td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 28582</td><td>coleridge </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.32391e+09</td><td>Everything is ok, except it just isn't as good as it is in the bags. Just considerably more bland -- less lemon and less zing. Boring. </td></tr>\n<tr><td>B001PMCDK2 </td><td>A14TTMM0Z03Y2W</td><td>my cat goes crazy for these! </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">389965</td><td>Lindsay S. Bradford </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.3106e+09 </td><td>Best cat treat ever. There isn't anything comparable to the love my cat has for these treats, he snubs away any other kind now.<br />I know he likes to manipulate me with his cattiness but these treats are my way of manipulating him to come sit on my lap and have some chill time. :) </td></tr>\n<tr><td>B002Q8JOSI </td><td>A17UQD2RSSQH5X</td><td>My dogs tell me these treats are YUMMY</td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">212536</td><td>in the dark </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.31613e+09</td><td>My two Corgis were thoroughly spoiled by my late husband (I spent a year and a half dieting them down a combined total of 25 pounds!)<br /><br />They are accustomed to the finest of fare, and they absolutely love the Wellness brand of treats. </td></tr>\n<tr><td>B00176G870 </td><td>A2F2MZW8EOGH5J</td><td>Yummy to the tummy </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">115971</td><td>daemoncycler "When you arrive at a fork in th...</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.33479e+09</td><td>We used to have drive down to the specialty pet food store for this product. So glad we discovered Amazon. As far as I can tell it is no more expensive & in some cases less - Prime membership is awesome. Loving Pets treats are some of the best according to my dog. They do not develop that nasty smell like some dog treats do. </td></tr>\n<tr><td>B001CHFUGY </td><td>A2M8VROSDPU4JT</td><td>Very good coffee </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">434484</td><td>Officefan "Officefankt" </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.27725e+09</td><td>I really liked this coffee, it was just as good as everyone claimed it was. Strong, bold and flavorful! I would recommend! </td></tr>\n<tr><td>B0041CIR62 </td><td>A16I6WJUEBJ1C3</td><td>okay but not as healthy as it appears </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">138997</td><td>doctorsirena "doctorsirena" </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.34369e+09</td><td>I am always looking for healthier, whole grain versions of foods I enjoy. Unfortunately, these Peacock brand noodles are yet another food masquerading as healthy. The product title in big letters on the front says "Brown Rice Vermicelli", making the consumer think "this is made with brown rice, so it should be a healthy choice". But the first indication that it is not is when looking at the fiber content on the nutrition facts - only 0.6g per 2oz serving. Then onto the ingredients list to see why so low... contains brown rice, sago starch and water. The sago starch comes from palms and must not have much (if any) fiber.<br /><br />The Annie Chun's Maifun Brown Rice Noodles (sold on Amazon and in my local healthy grocer) has become one of my staples and is my frame of reference when comparing to the Peacock brand. The Annie Chun's product is made with 100% whole grain, with ingredients brown rice flour and water. Per 2oz serving, it has 4g fiber and pretty much the same calories and other nutrients as the Peacock brand.<br /><br />If you do try this Peacock brand noodles and have not used rice noodles before, you will need to seek guidance elsewhere on preparation. As others have pointed out, the Peacock package gives almost no directions on how to prepare the product, aside from a brief mention in the recipes (in the header text it does say that they are "easy-to-cook" but does not say how). It also contains a very strange recipe for rice noodles: Aglio Olio style - this is an Italian recipe for noodles with olive oil/garlic/sprinkled with grated cheese that I think would not be very tasty. The second recipe appears to be for a soup with veggie strips. Neither recipe gives amounts or much direction. In comparison, the Annie Chun's package gives clear, specific directions on rice noodle preparation and two recipes.<br /><br />I use rice noodles = maifun = rice sticks = sometimes called vermicelli for making the Vietnamese salad "bun tofu", to serve with stir-fried veggies or in lettuce rolls. They can also be used in spring rolls/egg rolls. When cooking with thin rice noodles, be careful not to oversoak/overcook/overmix or they tend to disintegrate. Asian rice noodle vermicelli (maifun) are not the same as Italian vermicelli and are not readily interchangeable. If making an Italian recipe, the best results would be expected from Italian pasta and not maifun.<br /><br />A few final notes... Both Peacock and Annie Chun's brown rice noodles are gluten free. The Peacock is made in Singapore and the Annie Chun's in Thailand. The Peacock noodles do taste fine (kind of bland), but so do the Annie Chun's. At this time, they are both approximately the same price. Peacock come in an plastic bag with some noodle crushage upon shipping; Annie Chun's are perfect upon removal from their cellophane bag in a box. Overall, I highly recommend the Annie Chun's Maifun as a healthier option over the Peacock brand. On a related note, the Annie Chun's soba and brown rice pad thai noodles are also excellent.<br /><br />Rating for this product: 2.5 stars rounded down to 2 stars.</td></tr>\n<tr><td>B001R3BQFW </td><td>AM50E42AFUVNL </td><td>Taste great. </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">126555</td><td>T. Higley "Tina" </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.32356e+09</td><td>I have tried many different drink mix, this is the best tasting by far. It does not have the after taste of the sweetener and I really like it, it is pretty strong, so I use a big water bottle (20 oz) for one tube, it still a little stronger than I like, but it is just my taste. </td></tr>\n<tr><td>B005HGAV8I </td><td>A2I5KDNOESGJ1H</td><td>variety galore </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">438837</td><td>TJ </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.33402e+09</td><td>This is my favorite item to order for my Keurig. There are so many flavors, my finicky palate never gets bored! The only downside is there are probably 5-6 decaf varieties. I don't drink decaf (I REQUIRE copious amounts of caffeine), so they sit on the shelf... </td></tr>\n<tr><td>B000GFYRHQ </td><td>A3A7YUR6FS6ZCI</td><td>Bigelow Earl Grey Green Tea </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">245379</td><td>Tea Lover </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.17841e+09</td><td>Tastes like Earl Grey, but it's green tea so it's healthier. </td></tr>\n</tbody>\n</table>"},"metadata":{},"output_type":"display_data"},{"data":{"text/plain":""},"execution_count":6,"metadata":{},"output_type":"execute_result"}]},{"metadata":{},"cell_type":"markdown","source":"## H2O offers great functions to convert H2OFrames into Pandas and Spark DataFrames"},{"metadata":{"trusted":false},"cell_type":"code","source":"# Spark DataFrame\ndf = hc.asSparkFrame(reviews, copyMetadata=False)\ndf.limit(100).show()\ndel df._h2o_frame\nprint(type(df))\n# Pandas DataFrame\npdf = reviews.head().as_data_frame()\ndisplay(pdf)\nprint(type(pdf))","execution_count":7,"outputs":[{"name":"stdout","output_type":"stream","text":"+----------+--------------+--------------------+-----+----------------------+------+--------------------+--------------------+----------+--------------------+\n| ProductId| UserId| Summary|Score|HelpfulnessDenominator| Id| ProfileName|HelpfulnessNumerator| Time| Text|\n+----------+--------------+--------------------+-----+----------------------+------+--------------------+--------------------+----------+--------------------+\n|B00141QYSQ|A1YS02UZZGRDCT| Do Not Buy| 1| 2| 41471| Evan Eberhardt| 2|1348358400|These are made in...|\n|B0089SPEO2|A3JOYNYL458QHP|Less lemon and le...| 3| 0| 28582| coleridge| 0|1323907200|Everything is ok,...|\n|B001PMCDK2|A14TTMM0Z03Y2W|my cat goes crazy...| 5| 0|389965| Lindsay S. Bradford| 0|1310601600|Best cat treat ev...|\n|B002Q8JOSI|A17UQD2RSSQH5X|My dogs tell me t...| 5| 1|212536| in the dark| 1|1316131200|My two Corgis wer...|\n|B00176G870|A2F2MZW8EOGH5J| Yummy to the tummy| 5| 0|115971|daemoncycler \"Whe...| 0|1334793600|We used to have d...|\n|B001CHFUGY|A2M8VROSDPU4JT| Very good coffee| 5| 1|434484|Officefan \"Office...| 1|1277251200|I really liked th...|\n|B0041CIR62|A16I6WJUEBJ1C3|okay but not as h...| 2| 1|138997|doctorsirena \"doc...| 1|1343692800|I am always looki...|\n|B001R3BQFW| AM50E42AFUVNL| Taste great.| 5| 0|126555| T. Higley \"Tina\"| 0|1323561600|I have tried many...|\n|B005HGAV8I|A2I5KDNOESGJ1H| variety galore| 5| 1|438837| TJ| 1|1334016000|This is my favori...|\n|B000GFYRHQ|A3A7YUR6FS6ZCI|Bigelow Earl Grey...| 5| 0|245379| Tea Lover| 0|1178409600|Tastes like Earl ...|\n|B001KUUNHE|A35R32TA60XD57|Best simple cat food| 5| 0| 64358| M. Torma| 0|1338940800|My cat that can e...|\n|B000RHXLDO|A1P8ZVG6MJX9O1|Organic is Better...| 5| 5| 75530| Mark Emdee| 4|1277164800|UPDATE: I now no ...|\n|B00013UQOG|A2AJNKK1S0W8F0| Lots of Roses!| 4| 1|312517| James R. Kelley| 1|1331337600|Much more roses t...|\n|B003E6COLK|A1S4RN19B0G9TQ|The best Gummy Be...| 5| 0|444328|S. Moon \"Moons Girl\"| 0|1313971200|I recommend these...|\n|B009PFJUF2|A16HJRHRHNSUZ6|Looks very pretty...| 5| 1|286942| Danielle Tietz| 1|1350432000|My neighborhood r...|\n|B000VK08OC|A15DZG6P6YB869|Fresh and delicious!| 5| 3|292099| P. Banks| 3|1313107200|We purchased this...|\n|B005K4Q4KG|A13L66J35SMYE5| Not good| 1| 2|243065| Elizabeth Ramsoram| 1|1330646400|This does not tas...|\n|B000LKZD4W|A2MQ5M9IJAXX6C|Great high-protei...| 5| 0|152409| Carol Mathis| 0|1238025600|This is a great p...|\n|B00213ERI0|A20GEEXSF3DULO|Gluten Free yummi...| 5| 0| 11177| Vicki \"Noah's Mom\"| 0|1274832000|to all of you glu...|\n|B0000D9MXL|A16OEO4JDUIP48|A fantastic Cheddar!| 5| 2|213880|D. Kennedy \"zibed...| 2|1234569600|The epitome of an...|\n+----------+--------------+--------------------+-----+----------------------+------+--------------------+--------------------+----------+--------------------+\nonly showing top 20 rows\n\n<class 'pyspark.sql.dataframe.DataFrame'>\n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>ProductId</th>\n <th>UserId</th>\n <th>Summary</th>\n <th>Score</th>\n <th>HelpfulnessDenominator</th>\n <th>Id</th>\n <th>ProfileName</th>\n <th>HelpfulnessNumerator</th>\n <th>Time</th>\n <th>Text</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>B00141QYSQ</td>\n <td>A1YS02UZZGRDCT</td>\n <td>Do Not Buy</td>\n <td>1</td>\n <td>2</td>\n <td>41471</td>\n <td>Evan Eberhardt</td>\n <td>2</td>\n <td>1348358400</td>\n <td>These are made in China (do not buy ANY pet fo...</td>\n </tr>\n <tr>\n <th>1</th>\n <td>B0089SPEO2</td>\n <td>A3JOYNYL458QHP</td>\n <td>Less lemon and less zing</td>\n <td>3</td>\n <td>0</td>\n <td>28582</td>\n <td>coleridge</td>\n <td>0</td>\n <td>1323907200</td>\n <td>Everything is ok, except it just isn't as good...</td>\n </tr>\n <tr>\n <th>2</th>\n <td>B001PMCDK2</td>\n <td>A14TTMM0Z03Y2W</td>\n <td>my cat goes crazy for these!</td>\n <td>5</td>\n <td>0</td>\n <td>389965</td>\n <td>Lindsay S. Bradford</td>\n <td>0</td>\n <td>1310601600</td>\n <td>Best cat treat ever. There isn't anything comp...</td>\n </tr>\n <tr>\n <th>3</th>\n <td>B002Q8JOSI</td>\n <td>A17UQD2RSSQH5X</td>\n <td>My dogs tell me these treats are YUMMY</td>\n <td>5</td>\n <td>1</td>\n <td>212536</td>\n <td>in the dark</td>\n <td>1</td>\n <td>1316131200</td>\n <td>My two Corgis were thoroughly spoiled by my la...</td>\n </tr>\n <tr>\n <th>4</th>\n <td>B00176G870</td>\n <td>A2F2MZW8EOGH5J</td>\n <td>Yummy to the tummy</td>\n <td>5</td>\n <td>0</td>\n <td>115971</td>\n <td>daemoncycler \"When you arrive at a fork in th...</td>\n <td>0</td>\n <td>1334793600</td>\n <td>We used to have drive down to the specialty pe...</td>\n </tr>\n <tr>\n <th>5</th>\n <td>B001CHFUGY</td>\n <td>A2M8VROSDPU4JT</td>\n <td>Very good coffee</td>\n <td>5</td>\n <td>1</td>\n <td>434484</td>\n <td>Officefan \"Officefankt\"</td>\n <td>1</td>\n <td>1277251200</td>\n <td>I really liked this coffee, it was just as goo...</td>\n </tr>\n <tr>\n <th>6</th>\n <td>B0041CIR62</td>\n <td>A16I6WJUEBJ1C3</td>\n <td>okay but not as healthy as it appears</td>\n <td>2</td>\n <td>1</td>\n <td>138997</td>\n <td>doctorsirena \"doctorsirena\"</td>\n <td>1</td>\n <td>1343692800</td>\n <td>I am always looking for healthier, whole grain...</td>\n </tr>\n <tr>\n <th>7</th>\n <td>B001R3BQFW</td>\n <td>AM50E42AFUVNL</td>\n <td>Taste great.</td>\n <td>5</td>\n <td>0</td>\n <td>126555</td>\n <td>T. Higley \"Tina\"</td>\n <td>0</td>\n <td>1323561600</td>\n <td>I have tried many different drink mix, this is...</td>\n </tr>\n <tr>\n <th>8</th>\n <td>B005HGAV8I</td>\n <td>A2I5KDNOESGJ1H</td>\n <td>variety galore</td>\n <td>5</td>\n <td>1</td>\n <td>438837</td>\n <td>TJ</td>\n <td>1</td>\n <td>1334016000</td>\n <td>This is my favorite item to order for my Keuri...</td>\n </tr>\n <tr>\n <th>9</th>\n <td>B000GFYRHQ</td>\n <td>A3A7YUR6FS6ZCI</td>\n <td>Bigelow Earl Grey Green Tea</td>\n <td>5</td>\n <td>0</td>\n <td>245379</td>\n <td>Tea Lover</td>\n <td>0</td>\n <td>1178409600</td>\n <td>Tastes like Earl Grey, but it's green tea so i...</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" ProductId UserId Summary Score \\\n0 B00141QYSQ A1YS02UZZGRDCT Do Not Buy 1 \n1 B0089SPEO2 A3JOYNYL458QHP Less lemon and less zing 3 \n2 B001PMCDK2 A14TTMM0Z03Y2W my cat goes crazy for these! 5 \n3 B002Q8JOSI A17UQD2RSSQH5X My dogs tell me these treats are YUMMY 5 \n4 B00176G870 A2F2MZW8EOGH5J Yummy to the tummy 5 \n5 B001CHFUGY A2M8VROSDPU4JT Very good coffee 5 \n6 B0041CIR62 A16I6WJUEBJ1C3 okay but not as healthy as it appears 2 \n7 B001R3BQFW AM50E42AFUVNL Taste great. 5 \n8 B005HGAV8I A2I5KDNOESGJ1H variety galore 5 \n9 B000GFYRHQ A3A7YUR6FS6ZCI Bigelow Earl Grey Green Tea 5 \n\n HelpfulnessDenominator Id \\\n0 2 41471 \n1 0 28582 \n2 0 389965 \n3 1 212536 \n4 0 115971 \n5 1 434484 \n6 1 138997 \n7 0 126555 \n8 1 438837 \n9 0 245379 \n\n ProfileName HelpfulnessNumerator \\\n0 Evan Eberhardt 2 \n1 coleridge 0 \n2 Lindsay S. Bradford 0 \n3 in the dark 1 \n4 daemoncycler \"When you arrive at a fork in th... 0 \n5 Officefan \"Officefankt\" 1 \n6 doctorsirena \"doctorsirena\" 1 \n7 T. Higley \"Tina\" 0 \n8 TJ 1 \n9 Tea Lover 0 \n\n Time Text \n0 1348358400 These are made in China (do not buy ANY pet fo... \n1 1323907200 Everything is ok, except it just isn't as good... \n2 1310601600 Best cat treat ever. There isn't anything comp... \n3 1316131200 My two Corgis were thoroughly spoiled by my la... \n4 1334793600 We used to have drive down to the specialty pe... \n5 1277251200 I really liked this coffee, it was just as goo... \n6 1343692800 I am always looking for healthier, whole grain... \n7 1323561600 I have tried many different drink mix, this is... \n8 1334016000 This is my favorite item to order for my Keuri... \n9 1178409600 Tastes like Earl Grey, but it's green tea so i... "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"<class 'pandas.core.frame.DataFrame'>\n"}]},{"metadata":{},"cell_type":"markdown","source":"## Nice!\n<style>\nblockquote{\n font-size: 15px;\n background: #f9f9f9;\n border-left: 10px solid #ccc;\n margin: .5em 10px;\n padding: 30em, 10px;\n quotes: \"\\201C\"\"\\201D\"\"\\2018\"\"\\2019\";\n padding: 10px 20px;\n line-height: 1.4;\n}\n\nblockquote:before {\n content: open-quote;\n display: inline;\n height: 0;\n line-height: 0;\n left: -10px;\n position: relative;\n top: 30px;\n bottom:30px;\n color: #ccc;\n font-size: 3em;\n display:none;\n\n}\n\np{\n margin: 0;\n}\n\nfooter{\n margin:0;\n text-align: right;\n font-size: 1em;\n font-style: italic;\n}\n</style>\n<blockquote><p class='quotation'><b><br><span style='font-size:25px'>Create Table and Insert Data</span></b> <br><br>Now that we have our Spark DataFrame, we can create a table and insert data using <code>splice.createTable</code> and <code>splice.insert</code><br><b>Note: </b>If your code is hanging on the <code>insert</code> your cluser may be out of memory. Try configuring your Spark or H2O cluster with more memory. Read about that <a href=https://docs.h2o.ai/sparkling-water/2.1/latest-stable/doc/configuration/configuration_properties.html>here</a> and <a href=https://spark.apache.org/docs/latest/configuration.html#available-properties>here</a></footer></blockquote><br>\n"},{"metadata":{"trusted":false},"cell_type":"code","source":"help(splice.createTable)\nprint('----------------------------------------------------------------------------------------------------------------')\nhelp(splice.insert)","execution_count":8,"outputs":[{"name":"stdout","output_type":"stream","text":"Help on method createTable in module splicemachine.spark.context:\n\ncreateTable(dataframe, schema_table_name, primary_keys=None, create_table_options=None, to_upper=False, drop_table=False) method of splicemachine.spark.context.PySpliceContext instance\n Creates a schema.table from a dataframe\n :param dataframe: The Spark DataFrame to base the table off\n :param schema_table_name: str The schema.table to create\n :param primary_keys: List[str] the primary keys. Default None\n :param create_table_options: str The additional table-level SQL options default None\n :param to_upper: bool If the dataframe columns should be converted to uppercase before table creation\n If False, the table will be created with lower case columns. Default False\n :param drop_table: bool whether to drop the table if it exists. Default False. If False and the table exists,\n the function will throw an exception.\n\n----------------------------------------------------------------------------------------------------------------\nHelp on method insert in module splicemachine.spark.context:\n\ninsert(dataframe, schema_table_name, to_upper=False) method of splicemachine.spark.context.PySpliceContext instance\n Insert a dataframe into a table (schema.table).\n \n :param dataframe: (DF) The dataframe you would like to insert\n :param schema_table_name: (string) The table in which you would like to insert the DF\n :param to_upper: bool If the dataframe columns should be converted to uppercase before table creation\n If False, the table will be created with lower case columns. Default False\n\n"}]},{"metadata":{"scrolled":true,"trusted":false},"cell_type":"code","source":"# Create the table\nschema = 'splice'\ndf = df.withColumnRenamed('Time', 'Review_Time')\ndf = df.withColumnRenamed('Text', 'Review')\nsplice.createTable(df, f'{schema}.AMAZON_REVIEWS_H2O', to_upper=True, drop_table=True)","execution_count":9,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Now we can insert our dataframe"},{"metadata":{"trusted":false},"cell_type":"code","source":"print('Inserting... ', end='')\nsplice.insert(df, f'{schema}.AMAZON_REVIEWS_H2O',to_upper=True) # Use to_upper to give the SQL table uppercase columns\nprint('Done.')","execution_count":11,"outputs":[{"name":"stdout","output_type":"stream","text":"Inserting... Done.\n"}]},{"metadata":{"trusted":false},"cell_type":"code","source":"%%sql\nselect top 10 varchar(Summary) Summary, Score, HelpfulnessDenominator, Id from AMAZON_REVIEWS_H2O;","execution_count":12,"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"85df984e-6b47-4232-894a-6e64552e15ba","version_major":2,"version_minor":0},"method":"display_data"},"metadata":{},"output_type":"display_data"}]},{"metadata":{},"cell_type":"markdown","source":"# Awesome! Let's get modeling\n<style>\nblockquote{\n font-size: 15px;\n background: #f9f9f9;\n border-left: 10px solid #ccc;\n margin: .5em 10px;\n padding: 30em, 10px;\n quotes: \"\\201C\"\"\\201D\"\"\\2018\"\"\\2019\";\n padding: 10px 20px;\n line-height: 1.4;\n}\n\nblockquote:before {\n content: open-quote;\n display: inline;\n height: 0;\n line-height: 0;\n left: -10px;\n position: relative;\n top: 30px;\n bottom:30px;\n color: #ccc;\n font-size: 3em;\n display:none;\n\n}\n\np{\n margin: 0;\n}\n\nfooter{\n margin:0;\n text-align: right;\n font-size: 1em;\n font-style: italic;\n}\n</style>\n<blockquote><p class='quotation'><b><br><span style='font-size:25px'>Modeling</span></b> <br><br>We're going to try three different ways to approach this problem, and track it all with MLManager. \n <ol>\n <li>No Text Model: We will try to predict the customer reviews without using the text from the review. Just the Numeric Columns</li>\n <li>Using the reviews: We will use Word2Vec to create vectors from the text of the reviews. We will then train a model on that word embedding feature-vector</li>\n <li>Using the review summaries: We will use Word2Vec to create vectors from the text of the review summaries</li>\n </ol>\n Which do you think will perform the best?\n </i></br><footer>Splice Machine</footer></blockquote><br>"},{"metadata":{},"cell_type":"markdown","source":"## First Attempt\nLet's create a simple model using the non-review columns\n<br>\n<blockquote>\n First, let's start our mlflow experiment! We can start a run and log import parameters, tags, and metrics as they come<br>\nNext, we can turn this into a binary-classification problem by turning score into a positive or negative review. We will say that 4 and 5 start reviews are positive, but you can change this and try other things!\n</br><footer>Splice Machine</footer></blockquote><br>"},{"metadata":{"trusted":false},"cell_type":"code","source":"# Set our mlflow experiment\nmlflow.set_experiment('Sentiment Analysis')\n# Look at our dataframe\nreviews[\"PositiveReview\"] = (reviews[\"Score\"] >= 4).ifelse(\"1\", \"0\")\nreviews[\"PositiveReview\"].table()\n","execution_count":13,"outputs":[{"name":"stdout","output_type":"stream","text":"INFO: 'Sentiment Analysis' does not exist. Creating a new experiment\n"},{"data":{"text/html":"<table>\n<thead>\n<tr><th style=\"text-align: right;\"> PositiveReview</th><th style=\"text-align: right;\"> Count</th></tr>\n</thead>\n<tbody>\n<tr><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 21791</td></tr>\n<tr><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\"> 78209</td></tr>\n</tbody>\n</table>"},"metadata":{},"output_type":"display_data"},{"data":{"text/plain":""},"execution_count":13,"metadata":{},"output_type":"execute_result"}]},{"metadata":{},"cell_type":"markdown","source":"## Let's see our Data Correlation"},{"metadata":{"trusted":false},"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport seaborn as sns\nfrom pyspark.sql.types import FloatType, IntegerType\n\n%matplotlib inline\n\npdf = reviews[['ProductId', 'UserId', 'HelpfulnessNumerator', 'HelpfulnessDenominator', 'Time','PositiveReview']].as_data_frame()\ncorr = pdf.corr()\n\nticks = [i for i in range(len(corr.columns))]\n# Generate a mask for the upper triangle\n\n# Set up the matplotlib figure\nf, ax = plt.subplots(figsize=(11, 9))\n\n# Color Scheme\ncmap = \"coolwarm\"\n\n# Draw the heatmap with the mask and correct aspect ratio\nsns.heatmap(corr, cmap=cmap, vmax=.3, center=0,\n square=False, linewidths=.5, cbar_kws={\"shrink\": .5})\n\nplt.xticks(ticks, corr.columns)\nplt.yticks(ticks, corr.columns)\nplt.title('Sentiment Data correlation heatmap')\nplt.show()\n","execution_count":14,"outputs":[{"name":"stderr","output_type":"stream","text":"PILLOW_VERSION is deprecated and will be removed in a future release. Use __version__ instead.\npandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAoIAAAJOCAYAAAAj2mbaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd5xldX3/8dd7FxBFikZEQ5EiYkVFBLHE3sCCFQ0aWyT4U1ATTbCiJhpj1Ag2XCtGBbuioGADNDaaoIgIIrooRVEBKVL28/vjnIHLeGd2Znd2z5x7Xs/H4zzmnv4599yZ+dxvO6kqJEmSNDxLug5AkiRJ3TARlCRJGigTQUmSpIEyEZQkSRooE0FJkqSBMhGUJEkaKBNBaRFIckiS13Qdh/5aknOTPGwV931AkjPXQExbJ6kk6yz0sSUNi4mgNIMk90/y3SSXJPlDkv9Lcu8FOO6zk3xndFlV7VtV/766x16FWF6X5GMr2ebcJFcmuSzJn9r3ZN8kc/r7MaSkpb3O20/NV9W3q2qHLmOaryQfSfIfXcchae0wEZTGSLIR8GXgncAtgc2B1wN/6TKuDj22qjYEbge8Gfg34IPdhjQ/4xLRISSnkjQbE0FpvDsAVNVhVXVdVV1ZVcdU1WlTGyR5bpIzkvwxydFJbjeyrtpSs7Pa9e9O407AIcBuSf6c5E/t9teXwiR5UJLzkvxrkouSnJ9kzyS7J/l5Wzr5ypFzLUlyQJJfJLk4yaeS3LJdN1Ua96wkv07y+ySvatc9CnglsFcby6kre1Oq6pKqOgLYC3hWkru2x9ojySlJLk2yPMnrRnY7vv35p/Y8uyXZLsk323h/n+TjSTaZ6bxJ7pLka+21Xzh1/UlukuQdSX7bTu9IcpNp7+O/JbkA+PC4Ze22j0nyo5ESzx1niGOXJN9rtzs/ybuSrNeum7rOU9vr3GvqfCP73ynJse3+pyd53Mi6j7SfkyPb0tcfJNluJbdk7+n3tT3WjJ+Jdv2nk1yQprT7+CR3aZfvA+wN/Gt7DV9ql5+b5OVJTktyeZIPJtksyVfaWL+e5BYrO/7IdR7S3s/LkhyXkd8dSWtZVTk5OU2bgI2Ai4FDgUcDt5i2fk/gbOBOwDrAq4HvjqwvmhLFTYCtgN8Bj2rXPRv4zrTjfQT4j/b1g4BrgdcC6wLPb/f/BLAhcBfgKmDbdvuXAN8HtgBuArwPOKxdt3Uby/uBmwJ3pynVvFO7/nXAx1byXpwLPGzM8l8DLxiJ+W40Xy53BC4E9pwWwzoj+94eeHgb76Y0yeI7Zjj/hsD5wL8A67fzu7br3tBe+63b43wX+Pdp7+N/tee56QzLdgIuAnYFlgLPaq/5JtOvH7gXcJ/2nm8NnAG8ZNp9v/3I/IOA89rX69J8Zl4JrAc8BLgM2GHkM/AHYJf2+B8HDp/hPVnZfZ3xM9Guf277Pt4EeAfwo3GfxWmfge8Dm9GUjl8EnAzcsz3GN4ED53H8y4C/a9cfxLTfBycnp7U3dR6Ak9NinWiSvI8A57XJwxHAZu26rwDPG9l2CXAFcLt2voD7j6z/FHBA+/rZ0//x8deJ4JXA0nZ+w/Z4u45sfxI3JFpnAA8dWXdb4JqRZKWALUbW/xB4Wvv6dax6Ivh94FUz7PMO4H/a11MxrDPLOfYETplh3dNnWfcLYPeR+UcC5468j1cD64+sH7fsvbTJ48iyM4EHznb97bqXAJ8fmZ8tEXwAcAGwZGT9YcDrRj4DHxhZtzvwsxnOu7L7OuNnYsyxNmmPtfH0z+K0z8DeI/OfBd47Mr8f8IUZYh13/MNH1t8cuA7YciF+b52cnOY3WTUszaCqzqiqZ1fVFsBdgb+lSXCgaSt3UFvF9yeakpzQlJZMuWDk9RU0//Dm6uKquq59fWX788KR9VeOHO92wOdHYjmD5h/rZgsUy0w2p7lukuya5FtJfpfkEmBf4FYz7Zjk1kkOT/KbJJcCH5tl+y1pEr5x/hb41cj8r9plU35XVVdN22f6stsB/zL1/rXv4ZbTjjMV9x2SfLmt9rwUeNNs1zkm1uVVtWJavKvzmZlp+xk/E0mWJnlzW218KU2SxxyuY/rnb+zncY7HXz71oqr+TPM5+qv3W9KaZyIozUFV/YymJOOu7aLlwD9V1SYj002r6rtzOdwCh7ccePS0WNavqt+sqVjS9J7eHJjq/fwJmhLTLatqY5p2kJnlHP/ZLt+xqjYCnjGy/XTLgZnayv2WJumZslW7bMq4c09fthx447T372ZVddiYfd8L/AzYvo37lbPEPS7WLXPj3tZbAXO5T/M122fi74HHAw8DNqYpXYTZ79d8rOz40CTazcLk5jQdskbvm6S1xERQGiPJHZP8S5It2vktaaoov99ucgjwipFG9hsnecocD38hsMVUJ4MFcAjwxqkG90k2TfL4ecSydeY+FMxGSR4DHE5TpfzjdtWGwB+q6qoku9AkA1N+B6wAth1ZtiHwZ5oOJJsDL5/ltF8GbpPkJWk6h2yYZNd23WHAq9trvhVNu8pZh8MZ4/3Avm2pZpJskKbzy4Zjtt0QuBT4c5I7Ai+Ytv7Cadc56gfA5TQdMdZN8iDgsTTv5UKb7TOxIU17wouBm9GUao6a7RrmYmXHB9g9zfBM6wH/DvygqpaP2U7SGmYiKI13GU3ngR8kuZwmAfwJTYcFqurzNB0ODm+rv35C06lkLr4JnA5ckOT3CxDrQTSlccckuayNddfZd7nep9ufFyc5eZbtvtQeeznwKuDtwHNG1v8/4A3tNq+laRMJQFVdAbwR+L+2qvI+NEPx7ARcAhwJfG6mE1fVZTQdSx5LUxV6FvDgdvV/ACcCpwE/punAMK8x8KrqRJoOOe8C/kjToePZM2z+Mpok9zKaBPKT09a/Dji0vc6nTjvP1cDjaD4nvwfeA/xDW9q80Gb7THyUpkr6N8BPueHLzZQPAndur+ELq3DulR0fmhLkA2mqhO9F01NZUgdStdC1VJIkjZfkIzQdaF7ddSySLBGUJEkaLBNBSZKkgbJqWJIkaaB6XSLYjlf10q7jkCRJ6qNeJ4LtgLtzHSZDkiRJI3pfNZzkjTSDln6SZowuAKpq7FAY7UPV9wF43/ved6/NX/i2tRGmOrTHNWcCcOS6O3Qcida0qXv9ziP7/XdNK7ffHuHHZ1+48g3Ve3e7/WZzHbRdq2CdrgNYAPdtf75hZFnRPND9r1TVMmDZ1OyRJoKSJGmgep8IVtWDV76VJEmSput1G0G4/tFeb09yYju9LcnGXcclSZK02PU+EQQ+RPO4p6e206XAhzuNSJIkqQd6XzUMbFdVTxqZf32SH3UWjSRJUk9MQonglUnuPzWT5H7AlR3GI0mS1AuTUCK4L/DRkXaBfwSe1WE8kiRJvTAJieClVXX3JBsBVNWlSbbpOihJkqTFbhKqhj8LTQJYVZe2yz7TYTySJEm90NsSwSR3BO4CbJzkiSOrNgLW7yYqSZKk/uhtIgjsADwG2AR47Mjyy4DndxKRJElSj/Q2EayqLwJfTLJbVX2v63gkSZL6preJ4IhTkryQppr4+irhqnpudyFJkiQtfpPQWeR/gdsAjwSOA7agqR6WJEnSLCYhEbx9Vb0GuLyqDgX2AO7WcUySJEmL3iQkgte0P/+U5K7AxsDW3YUjSZLUD5PQRnBZklsArwaOAG4OvKbbkCRJkha/XieCSZbQPFnkj8DxwLYdhyRJktQbva4arqoVwIu6jkOSJKmPep0Itr6W5GVJtkxyy6mp66AkSZIWu15XDbemxgt84ciywmpiSZKkWfU+EayqbbqOQZIkqY96XzWc5GZJXp1kWTu/fZLHdB2XJEnSYtf7RBD4MHA1cN92/jzgP7oLR5IkqR8mIRHcrqreQjuwdFVdCaTbkCRJkha/SUgEr05yU5oOIiTZDvhLtyFJkiQtfr3vLAIcCHwV2DLJx4H7Ac/uNCJJkqQe6H0iWFVfS3IycB+aKuEXV9XvOw5LkiRp0ZuEqmGAzYGlwHrA3yV5YsfxSJIkLXq9LxFM8iFgR+B0YEW7uIDPdRaUJElSD/Q+EQTuU1V37joISZKkvpmEquHvJTERlCRJmqdJKBE8lCYZvIBm2JgAVVU7dhuWJEnS4jYJieCHgGcCP+aGNoKSJElaiUlIBH9dVUd0HYQkSVLfTEIi+LMknwC+xMgTRarKXsOSJEmzmIRE8KY0CeAjRpY5fIwkSdJK9D4RrKrndB2DJElSH/U+EUzyYZoSwBupqud2EI4kSVJv9D4RBL488np94AnAbzuKRZIkqTd6nwhW1WdH55McBny9o3AkSZJ6YxKeLDLd9sBWXQchSZK02PW+RDDJZdy4jeAFwL91FI4kSVJv9D4RrKoNu45BkiSpj3qbCCaZtfq3qn69tmKRJEnqo94mgsCRNFXCGVlWwKbArYGlXQQlSZLUF71NBKvqbqPzSbamaRv4MOBNHYQkSZLUK73vNZxk+yQfAb4CnATcuare2W1UkiRJi19vSwST3BV4FXAX4C3A86rqum6jkiRJ6o/eJoLAqcBymraCuwC7JDc0F6yq/TuKS5IkqRf6nAj6LGFJkqTV0NtEsKoOnb4syRLg5lV1aQchSZIk9cokdBb5RJKNkmwA/BQ4M8nLu45LkiRpset9IkjTS/hSYE/gKJrnDD+z25AkSZIWv0lIBNdNsi5NIvjFqrqGGz97WJIkSWNMQiL4PuBcYAPg+CS3A2wjKEmStBK97SwypaoOBg4eWfSrJA/uKh5JkqS+6H2JYJIXt51FkuSDSU4GHtJ1XJIkSYtd7xNB4LltZ5FHAJsCzwHe3G1IkiRJi98kJIJTjxPZHfhwVZ06skySJEkzmIRE8KQkx9Akgkcn2RBY0XFMkiRJi17vO4sAzwPuAZxTVVck+Rua6mFJkiTNYhJKBAu4M7B/O78BsH534UiSJPXDJCSC7wF2A57ezl8GvLu7cCRJkvphEqqGd62qnZKcAlBVf0yyXtdBSZIkLXaTUCJ4TZKltI+VS7IpdhaRJElaqUlIBA8GPg/cOskbge8Ab+o2JEmSpMWv91XDVfXxJCcBD6UZP3DPqjqj47AkSZIWvd4ngq2zgEtpryfJVlX1625DkiRJWtx6nwgm2Q84ELgQuI6mVLCAHbuMS5IkabHrfSIIvBjYoaou7joQSZKkPpmEziLLgUu6DkKSJKlvJqFE8Bzg2CRHAn+ZWlhVb+8uJEmSpMVvEhLBX7fTeu0kSZKkOeh9IlhVr+86BkmSpD7qbSKY5Eu0TxMZp6oetxbDkSRJ6p3eJoLAW7sOQJIkqc96mwhW1XFdxyBJktRnvU0EpyT5JWOqiKtq2w7CkSRJ6o3eJ4LAziOv1weeAtyyo1gkSZJ6o/cDSlfVxSPTb6rqHcBDuo5LkiRpset9iWCSnUZml9CUEG7YUTiSJEm90ftEEHjbyOtrgV8CT+0oFkmSpN7obSKY5MVVdRDwmqr6TtfxSJIk9U2f2wg+p/15cKdRSJIk9VRvSwSBM5KcC2ya5LSR5QGqqnbsJixJkqR+6G0iWFVPT3Ib4GjAx8lJkiTNU28TQYCqugC4e5L1gDvSDCx9ZlVd3W1kkiRJi1+vE0GAJLsD7wN+QVMtvE2Sf6qqr3QbmSRJ0uLW+0QQeDvw4Ko6GyDJdsCRgImgJEnSLPrca3jKRVNJYOsc4KKugpEkSeqLSSgRPD3JUcCnaNoIPgU4IckTAarqc10GJ0mStFhNQiK4PnAh8MB2/nfALYHH0iSGJoKSJElj9D4RrKrnrHwrSZIkTdfbRDDJO2lK/Maqqv3XYjiSJEm909tEEDix6wAkSZL6rLeJYFUdOjqfZIOquryreCRJkvqm98PHJNktyU+BM9r5uyd5T8dhSZIkLXq9TwSBdwCPBC4GqKpTgb/rNCJJkqQemIREkKpaPm3RdZ0EIkmS1CO9bSM4YnmS+wKVZD1gf9pqYkmSJM1sEkoE9wVeCGwOnAfco52XJEnSLHpfIlhVvwf27joOSZKkvultIuiA0pIkSaunt4kgNx5Q+vXAgV0FIkmS1Ee9TQRHB5RO8pLpA0xLkiRpdpPQWQRmqSKWJEnSeJOSCEqSJGmeels1nOQybigJvFmSS6dWAVVVG3UTmSRJUj/0NhGsqg27jkGSJKnPrBqWJEkaKBNBSZKkgTIRlCRJGigTQUmSpIEyEZQkSRooE0FJkqSBMhGUJEkaKBNBSZKkgTIRlCRJGigTQUmSpIEyEZQkSRqo3j5rWJIkaU06ct0daj7b73HNmVlTsawplghKkiQNlCWCkiRJY2Td3hXwzZuJoCRJ0hhL1jERlCRJGqSsO/kt6Cb/CiVJkjSWJYKSJEljWDUsSZI0UHYWkSRJGqghlAjaRlCSJGmgLBGUJEkaw6phSZKkgRpC1bCJoCRJ0hhZaiIoSZI0SEsGkAjaWUSSJGmgLBGUJEkaI0smv0TQRFCSJGmMLJ38ilMTQUmSpDFsIyhJkqSJZYmgJEnSGLYRlCRJGqghVA2bCEqSJI3hgNKSJEkDlSWT35XCRFCSJGkM2whKkiQNlG0EJUmSBmoIJYKTX/ktSZKksSwRlCRJGmMInUUm/wolSZJWQZZkXtOcjpk8KsmZSc5OcsCY9XsnOa2dvpvk7tPWL01ySpIvL8Q1WiIoSZI0xkJ3FkmyFHg38HDgPOCEJEdU1U9HNvsl8MCq+mOSRwPLgF1H1r8YOAPYaCFiskRQkiRp7dgFOLuqzqmqq4HDgcePblBV362qP7az3we2mFqXZAtgD+ADCxWQJYKSJEljrIFew5sDy0fmz+PGpX3TPQ/4ysj8O4B/BTZcqIBMBCVJksaYb2eRJPsA+4wsWlZVy0Y3GbNbzXCsB9Mkgvdv5x8DXFRVJyV50LwCm4WJoCRJ0hjzLRFsk75ls2xyHrDlyPwWwG//6rzJjjTVv4+uqovbxfcDHpdkd2B9YKMkH6uqZ8wryGlsIyhJkjTGGug1fAKwfZJtkqwHPA044kbnTLYCPgc8s6p+PrW8ql5RVVtU1dbtft9c3SQQLBGUJElaK6rq2iQvAo4GlgIfqqrTk+zbrj8EeC3wN8B7kgBcW1U7r6mYTAQlSZLGWBOPmKuqo4Cjpi07ZOT1PwL/uJJjHAscuxDxmAhKkiSNMYQni5gISpIkjbHQA0ovRr1OddvHrHy96zgkSZL6qNclglV1XZIrkmxcVZesyjH2uObMhQ5Li5T3ejj222Pyv8UL7nb7zboOQRNuTbQRXGx6XSLYugr4cZIPJjl4appp4yT7JDkxyYnLls021I8kSRqyLFkyr6mPel0i2DqyneZk2mCPdeS6O6yRoLR4TJUEeq8n39S9ftMnr+s4Eq1pr9xrKV8++dquw9Ba8JiduktVhlAi2PtEsKoObQdlvEO76MyquqbLmCRJUv+ZCPZA+7y9Q4FzaZ7ht2WSZ1XV8V3GJUmS+q2v1b3z0ftEEHgb8IiqOhMgyR2Aw4B7dRqVJEnqNUsE+2HdqSQQoKp+nmTdLgOSJEn9Z4lgP5yY5IPA/7bzewMndRiPJElSL0xCIvgC4IXA/jRtBI8H3t1pRJIkqf9i1XAf7FtVbwfePrUgyYuBg7oLSZIk9d0Q2ghOQuX3s8Yse/baDkKSJE0WB5RexJI8Hfh7YJskR4ys2hC4uJuoJEmS+qO3iSDwXeB84FY0Q8hMuQw4rZOIJEnSxBhC1XBvE8Gq+hXwK2C3rmORJEmTp6/VvfPR+ytMcp8kJyT5c5Krk1yX5NKu45IkSf2WJZnX1Ee9LREc8S7gacCngZ2BfwBu32lEkiSp9/qa3M3HJCSCVNXZSZZW1XXAh5N8t+uYJEmSFrtJSASvSLIe8KMkb6HpQLJBxzFJkqS+s41gLzyT5jpeBFwObAk8qdOIJElS7yWZ19RHvS8RbHsPA1wFvL7LWCRJ0uQYQq/h3ieCSe4HvA64HSPXU1XbdhWTJElSH/Q+EQQ+CLwUOAm4ruNYJEnShLDXcD9cUlVf6ToISZI0Yawa7oVvJflv4HPAX6YWVtXJ3YUkSZL6zhLBfti1/bnzyLICHtJBLJIkaUIklgguelX14K5jkCRJE8gSwcUryTOq6mNJ/nnc+qp6+9qOSZIkTQ6Hj1ncpp4esmGnUUiSpIlkG8FFrKre1/50EGlJkqRV0NtEcEqSbYD9gK258YDSj+sqJkmSNAHsLNILX6AZVPpLwIqOY5EkSRNiCFXDk5DqXlVVB1fVt6rquKmp66AkSVLPLVkyv2kOkjwqyZlJzk5ywJj1d0zyvSR/SfKyaes2SfKZJD9LckaS3Vb3EiehRPCgJAcCx+CA0pIkaYEkC1simGQp8G7g4cB5wAlJjqiqn45s9gdgf2DPMYc4CPhqVT05yXrAzVY3pklIBO8GPJNmAOmpqmEHlJYkSYvNLsDZVXUOQJLDgccD1yeCVXURcFGSPUZ3TLIR8HfAs9vtrgauXt2AJiERfAKwbfuGSJIkLYx5jiOYZB9gn5FFy6pq2cj85sDykfnzuOEJaSuzLfA74MNJ7g6cBLy4qi6fV5DTTEIbwVOBTboOQpIkTZYsybymqlpWVTuPTMumH3LMaWqO4awD7AS8t6ruCVwO/FUbw/mahBLBzYCfJTmBG7cRdPgYSZK06hZ++JjzgC1H5rcAfjuPfc+rqh+085/BRBCAA7sOQJIkaQ5OALZvx0D+DfA04O/nsmNVXZBkeZIdqupM4KGMtC1cVb1PBKvquCSbAfduF/2wbWgpSZK06hZ4HMGqujbJi4CjgaXAh6rq9CT7tusPSXIb4ERgI2BFkpcAd66qS2keoPHxtsfwOcBzVjem3ieCSZ4K/DdwLE3d+zuTvLyqPtNpYJIkqdeyBp4sUlVHAUdNW3bIyOsLaKqMx+37I2DnhYyn94kg8Crg3lOlgEk2Bb5OU3cuSZK0anyySC8smVYVfDGTcV2SJElr1CSUCH41ydHAYe38XkwrcpUkSZqvzHMcwT7qfSJYVS9P8iTgfjRtBJdV1ec7DkuSJPXdAj9ibjHqfSIIUFWfBT7bdRySJGmCDKBEsPdXmOSJSc5KckmSS5NcluTSruOSJEk9l8xv6qFJKBF8C/DYqjqj60AkSdLksI1gP1xoEihJkhbcGhhHcLGZhETwxCSfBL7AjZ81/LnuQpIkSb03gHEEJyER3Ai4AnjEyLICTAQlSZJm0ftEsKpW+zl7kiRJ062JR8wtNr2/wiR3SPKNJD9p53dM8uqu45IkST23JPObeqj3iSDwfuAVwDUAVXUa8LROI5IkSf2XJfObeqj3VcPAzarqh7nx+D3XdhWMJEmaED0dG3A++pm+3tjvk2xH00GEJE8Gzu82JEmSpMVvEkoEXwgsA+6Y5DfAL4G9uw1JkiT1ngNKL35VdQ7wsCQbAEuq6rKuY5IkSROgp+3+5qPXiWCSHYB9gDu2i85Isqyqft5hWJIkaRL0tCfwfPQ21U2yG3AscBlN1fD7gcuBY5Pcp8PQJEmSeqHPJYKvBZ5eVceOLPtCkm8CBwKP7iQqSZI0GQZQNdznK9xuWhIIQFUdB2y79sORJEkTJZnf1EN9LhGcrVPI5WstCkmSNJnsNbyobZnk4DHLA2y+toORJEnqmz4ngi+fZd2Jay0KSZI0mXpa3TsfvU0Eq+rQ6cuSLAFuXlWXdhCSJEmaJHYWWfySfCLJRu2A0j8FzkwyW2mhJEnSyi1ZMr+ph/oZ9Y3duS0B3BM4CtgKeGa3IUmSpN4bQK/hSUgE102yLk0i+MWqugaojmOSJEla9HrbRnDE+4BzgVOB45PcDrCNoCRJWj1LlnYdwRrX+0Swqg4GRoeR+VWSB3cVjyRJmhA9bfc3H72/wiQvbjuLJMkHk5wMPKTruCRJkha73ieCwHPbziKPADYFngO8uduQJElS31Uyr6mPel81TPMkEYDdgQ9X1alJT++GJElaPBxHsBdOSnIMTSJ4dJINgRUdxyRJkvouS+Y3zeWQyaOSnJnk7CQHjFmfJAe3609LstPIupcmOT3JT5IclmT91b3ESUgEnwccANy7qq4A1qOpHpYkSVplC101nGQp8G7g0cCdgacnufO0zR4NbN9O+wDvbffdHNgf2Lmq7gosBZ62utc4CYlg0byZ+7fzGwCrnSFLkiQtsF2As6vqnKq6GjgcePy0bR4PfLQa3wc2SXLbdt06wE2TrAPcDPjt6gY0CYnge4DdgKe385fRZNuSJEmrbp5Vw0n2SXLiyLTPtCNuDiwfmT+vXbbSbarqN8BbgV8D5wOXVNUxq3uJk9BZZNeq2inJKQBV9cck63UdlCRJ6rl59j2tqmXAstmOOG63uWyT5BY0pYXbAH8CPp3kGVX1sXkFOc0klAhe09a5F0CSTbGziCRJWl1LlsxvWrnzgC1H5rfgr6t3Z9rmYcAvq+p37eN0Pwfcd5WvrTUJieDBwOeBWyd5I/Ad4E3dhiRJkvRXTgC2T7JNW3v5NOCIadscAfxD23v4PjRVwOfTVAnfJ8nN2mHyHgqcsboB9b5quKo+nuQkmjckwJ5VtdpvjCRJGraFHiS6qq5N8iLgaJpevx+qqtOT7NuuPwQ4imZIvLOBK2hHQqmqHyT5DHAycC1wCrNXQ89J7xPB1lnApbTXk2Srqvp1tyFJkqReWwMDSlfVUTTJ3uiyQ0ZeF/DCGfY9EDhwIePpfSKYZD+aN+VC4DqaUsECduwyLkmS1G81gCeL9D4RBF4M7FBVF3cdiCRJmiADeGLtJKS6y4FLug5CkiSpbyahRPAc4NgkRwJ/mVpYVW/vLiRJktR3Vg33w6/bab12kiRJWn0DqBrufSJYVa/vOgZJkjSBLBFcvJJ8ib9+LMv1qupxazEcSZKk3ultIkjz4GVJkqQ1opYs7TqENa63iWBVHdd1DJIkaXIVthFc9JL8kjFVxFW1bQfhSJIk9UbvE0Fg55HX6wNPAW7ZUSySJGlCDGH4mN5fYVVdPDL9pqreATyk67gkSVLPZcn8ph7qfYlgkp1GZpfQlBBu2FE4kiRpQpTjCPbC20ZeXwv8EnhqR7FIkqQJMYSq4d4mgkleXFUHAa+pqu90HY8kSVLf9DnVfU778+BOo5AkSZMpmd/UQ70tEQTOSGAC+pAAAB1QSURBVHIusGmS00aWB6iq2rGbsCRJ0iSwangRq6qnJ7kNcDTg4+QkSdKCckDpRa6qLgDunmQ94I40A0ufWVVXdxuZJEnS4tfrRBAgye7A+4Bf0FQLb5Pkn6rqK91GJkmS+syq4X54O/DgqjobIMl2wJGAiaAkSVp1Pe0AMh+TkAheNJUEts4BLuoqGEmSNBmq14OrzM0kJIKnJzkK+BRNG8GnACckeSJAVX2uy+AkSVI/+WSRflgfuBB4YDv/O+CWwGNpEkMTQUmSpDF6nwhW1XNWvpUkSdL82FlkEUvyTpoSv7Gqav+1GI4kSZowjiO4uJ3YdQCSJGlyWSK4iFXVoaPzSTaoqsu7ikeSJKlvep/qJtktyU+BM9r5uyd5T8dhSZKknqtkXlMf9bZEcMQ7gEcCRwBU1alJ/q7bkCRJUt+tyNKuQ1jjel8iCFBVy6ctuq6TQCRJknpkEkoElye5L1BJ1gP2p60mliRJWlX2Gu6HfYGDgM2B84BjgBd2GpEkSeq9IfQa7v0VVtXvq2rvqtqsqm5dVc+oqou7jkuSJPVbkXlNc5HkUUnOTHJ2kgPGrE+Sg9v1pyXZaa77rorelgg6oLQkSVqTFrpEMMlS4N3Aw2lqMU9IckRV/XRks0cD27fTrsB7gV3nuO+89TYR5MYDSr8eOLCrQCRJkuZgF+DsqjoHIMnhwOOB0WTu8cBHq6qA7yfZJMltga3nsO+89TYRHB1QOslLpg8wLUmStDrWQGeRzYHRkU7Ooyn1W9k2m89x33nrfRvB1oxVxJIkSativgNKJ9knyYkj0z7TDjkus5yew8y0zVz2nbfelghKkiStSVXzKxGsqmXAslk2OQ/YcmR+C+C3c9xmvTnsO2+9LRFMclmSS5NcCuw49XpqedfxSZIkTXMCsH2Sbdqxj59G+2S0EUcA/9D2Hr4PcElVnT/HfeettyWCVbVh1zFIkqTJVQtcXlZV1yZ5EXA0sBT4UFWdnmTfdv0hwFHA7sDZwBXAc2bbd3Vj6m0iKEmStCatiSeLVNVRNMne6LJDRl4XMzwYY9y+q8tEUJIkaQwfMSdJkjRQQ0gEe9tZRJIkSavHEkFJkqQxhlAiaCIoSZI0xnzHEewjE0FJkqQxhlAiaBtBSZKkgbJEUJIkaYwhlAiaCEqSJI2xoia/4nTyr1CSJEljWSIoSZI0xgqrhiVJkobJNoKSJEkD5TiCkiRJAzWEEkE7i0iSJA2UJYKSJEljWDUsSZI0UEOoGjYRlCRJGmMIJYK2EZQkSRooSwQlSZLGWNF1AGuBiaAkSdIYQ6gaNhGUJEkaw84ikiRJAzWEEkE7i0iSJA2UJYKSJEljWDUsSZI0UCuq6wjWPBNBSZKkMYZQImgbQUmSpIGyRFCSJGmMIfQaNhGUJEkao2wjKEmSNEzX1eS3oJv8K5QkSdJYlghKkiSNYdWwJEnSQDl8jCRJ0kCtqPlNqyvJLZN8LclZ7c9bzLDdo5KcmeTsJAeMLL9Hku8n+VGSE5PssrJzmghKkiQtDgcA36iq7YFvtPM3kmQp8G7g0cCdgacnuXO7+i3A66vqHsBr2/lZmQhKkiSNUZV5TQvg8cCh7etDgT3HbLMLcHZVnVNVVwOHt/sBFLBR+3pj4LcrO+FEtBFMskFVXd51HJIkaXLMt7NIkn2AfUYWLauqZfM4xGZVdX5z7jo/ya3HbLM5sHxk/jxg1/b1S4Cjk7yVprDvvis7Ya8TwST3BT4A3BzYKsndgX+qqv/XbWSSJKnvVsyzs0ib9M2a+CX5OnCbMateNcfTjAtqKmV9AfDSqvpskqcCHwQeNuvBqsd9o5P8AHgycERV3bNd9pOquuscD9Hfi5ckaRg667r75ZOvnVee8Jid1lmtWJOcCTyoLQ28LXBsVe0wbZvdgNdV1SPb+VcAVNV/JrkE2KSqKkmAS6pqI2bR+zaCVbV82qLrZts+yT5tT5oTly2bT2mtJEkakqr5TQvgCOBZ7etnAV8cs80JwPZJtkmyHvC0dj9o2gQ+sH39EOCslZ2w11XDwPK2erjaN2N/4IzZdphWbFvvPNJCwUm33x7NF7Q3fXLW7wiaAK/caykAR667w0q2VN/tcc2ZHHf6FV2HobXggXe5WWfnXqAOIPPxZuBTSZ4H/Bp4CkCSvwU+UFW7V9W1SV4EHA0sBT5UVae3+z8fOCjJOsBV3Li94lh9TwT3BQ6iaTh5HnAM8MJOI5IkSRNhIcYGnI+quhh46JjlvwV2H5k/CjhqzHbfAe41n3P2OhGsqt8De3cdhyRJmjw97kYxZ71OBJNsA+wHbM3ItVTV47qKSZIkqS96nQgCX6DpGv0lYEXHsUiSpAkyhGcN9z0RvKqqDu46CEmSNHnWdhvBLvQ9ETwoyYE0nUT+MrWwqk7uLiRJkjQJbCO4+N0NeCbNWDlTVcPVzkuSJGkWfU8EnwBs2z50WZIkacFYIrj4nQpsAlzUdSCSJGmyrFj7A0qvdX1PBDcDfpbkBG7cRtDhYyRJ0mpZMYDxSPqeCB7YdQCSJEl91etEsKqO6zoGSZI0mRw+ZpFK8p2qun+Sy2h6CV+/Cqiq2qij0CRJ0oQo2wguWhsAVNWGXQciSZIm0xB6DS/pOoBVNIBbI0mStGb1tUTw1kn+eaaVVfX2tRmMJEmaPLYRXLyWAjeHATwNWpIkdWIIVcN9TQTPr6o3dB2EJEmaXENIBPvaRtCSQEmSpNXU1xLBh3YdgCRJmmy2EVykquoPXccgSZIm2xCqhnuZCEqSJK1pPmtYkiRpoIZQItjXziKSJElaTZYISpIkjTGEEkETQUmSpDHsNSxJkjRQNe8iwf4Nc2wbQUmSpIGyRFCSJGkM2whKkiQNlOMISpIkDZQlgpIkSQNlr2FJkqSBskRQkiRpoGreRYIOHyNJkjQRVtT8ptWV5JZJvpbkrPbnLWbY7kNJLkrykzHr9ktyZpLTk7xlZec0EZQkSVocDgC+UVXbA99o58f5CPCo6QuTPBh4PLBjVd0FeOvKTmgiKEmSNEbV/KYF8Hjg0Pb1ocCe4+Oq44E/jFn1AuDNVfWXdruLVnZCE0FJkqQxVqyoeU1J9kly4si0zzxPuVlVnQ/Q/rz1PPe/A/CAJD9IclySe69sBzuLSJIkjTHfUr6qWgYsm22bJF8HbjNm1avmd7ax1gFuAdwHuDfwqSTb1iwPTTYRlCRJGuOVey1d8G7AVfWwmdYluTDJbavq/CS3BVZatTvNecDn2sTvh0lWALcCfjfTDlYNS5IkLQ5HAM9qXz8L+OI89/8C8BCAJHcA1gN+P9sOJoKSJEmLw5uBhyc5C3h4O0+Sv01y1NRGSQ4DvgfskOS8JM9rV30I2LYdVuZw4FmzVQuDVcOSJEmLQlVdDDx0zPLfAruPzD99hv2vBp4xn3NaIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkDZSIoSZI0UCaCkiRJA2UiKEmSNFAmgpIkSQNlIihJkjRQJoKSJEkD1ftEMMlzk2zfdRySJEl9k6rqOobVkuQNwP2B2wEnAd8Gvl1VP5rD7v2+eEmSJl+6DmCS9T4RnJLkpsDzgZcBm1fV0hm22wfYp509o6qeuZZCVIeS7FNVy7qOQ2ue93o4vNfS6ut9Ipjk1cD9gJsDpwDfoSkRPH8O+55YVTuv4RC1CHivh8N7PRzea2n1rdN1AAvgicC1wJHAccD3q+qqbkOSJEla/HrfWaSqdgIeCvwQeDjw4yTf6TYqSZKkxa/3JYJJ7go8AHggsDOwnKbDyFzYtmQ4vNfD4b0eDu+1tJomoY3gkcDxNMnfCVV1TcchSZIk9ULvE0G4vsfwVlV1ZtexSJIk9cVK2wgm+fO0+WcneddK9lnpNu12hyU5LclLZ9nmQUm+PMv6xwI/Ar7azt8jyRErO/cMMa9IsuPIsp8k2Xq+x1oTkuyZ5M4LdKxFfU8XShvz75KckuSsJEcnue+aPu8s8eyb5B9Wcd+tk/z9QsfUB0n+JsmP2umCJL8Zmf9u1/ENRZLr2vf8J0k+neRmq3CMD0z9HUvyymnrVuletr8bV7ax/TTJR5Osu4rH2jnJwauyr9RXnXUWSXIb4L5VtWNV/c9qHOp1wC7AnwDagaS3XsVjnQe8ajViWS1Jxo592NoTmFcimGSttgFdwHu6kD5ZVfesqu2BNwOfS3KnLgKpqkOq6qOruPvWwLwSwbV9/9eUqrq4qu5RVfcADgH+Z2q+qjpL7AfoyvY9vytwNbDvfA9QVf9YVT9tZ185bd3q3MtftJ+PuwFbAE9dlYNU1YlVtf9qxCH1zmolgkk2TfLZJCe00/3GbPORJIck+XaSnyd5TLvqGODW7be4ByQ5NsnO7T63SnLumGO9LsmH2m3PSbI/cG1VXQLcIskPk/wI2DzJ0nb6SPsN9sdTpVRJ9m+/OZ6W5PCRU3wZuEuSHcac+88jr5+c5CMj1/feJN9qY3pgG+MZU9u02z0iyfeSnNx+m755u/zcJK9tezo/Jcnz2/fy1Pa9vVlbivU44L/b92u7tuTz++01fD7JLdrjHZvkTUmOA14813s5EudiuKdT654xdU+TvG8V7+n1qupbNI3L92n32S7JV5Oc1F7LHUeu7+Ak321jenK7PEn+e+Tce7XLH5TkuCSfat+PNyfZu439x0m2G7nWl43cp/9qt/l5kge0y7duYzm5nab+Ob4ZeED7Xrw0yfpJPtwe/5QkD273f3b7+fpSez8m2tTv5TzuwUo/35qTbwO3B0jyz+3vxE+SvKRdtkGSI9u/Yz8Z+V05Nk2p25uBm7af54+366bu5SeT7D51ovb38Unt7/5/t/fttCT/ND2oqrqOZgSJzdt9x+4zyzmur61or+FD7b6nJHl8u/yotDVH7fLXtq//Pck/LuzbLK15cykxuGma5GrKLYGpqteDaL6dfyfJVsDRwLjSlq1pevVuB3wrye1pEpsvt9/iSOb8BJk7Ag8GNgTOBL7Y/nO9JfBo4P+16/cGTqd5yshd23Ns0h7jAGCbqvrLyDKAFcBbaL6pPmuuAQG3AB7SXtOXaAa4/kfghCT3oClpfDXwsKq6PMm/Af8MvKHd/6qqun8b499U1fvb1/8BPK+q3pmmuvvLVfWZdt1pwH5VdVyax+wdCLykPd4mVfXAWeJd1Pc0yXtp/snsBdyvqq5J8h5W7Z5OdzIw9Q9kGbBvVZ2VZFfgPTT3EeC2NI8uvCPNe/MZmjEr7wHcHbgVzf09vt3+7jTv0x+Ac4APVNUuSV4M7McN92bUOu02u9Pcv4cBFwEPr6qr0jxD+zCa3vAHAC+rqse01/0vAFV1tzQJ7DFJ7tAedzdgx6r6wyzvwySayz2Y6+dbM0hT0vxo4KtJ7gU8B9iV5jFgP0jzJXRb4LdVtUe7z8ajx6iqA5K8aOpvxTSH0/zuH5VkPZrhwV4APA+4pKruneQmwP8lOYaRR4UmWb+NZepL8Ez7zHSOXUfieBXwzap6bvs35YdJvk7TOfEBab7YXkvz9x6avxcfm/s7KS0Oc0kErxz9ZU3ybJp/TND847rzyD/8jZJsOOYYn6qqFcBZSc6h+ef6p1WM+ciq+gvwlyQXAW8C3gWsT/Os4UvbaVuapGzbJO+kGXB6qoTkNODjSb4AfGHa8T8BvCrJNvOI6UtVVUl+DFxYVT8GSHI6TcK0BU217v+179V6wPdG9v/kyOu7tgngJjRPSzl6+snaP6qbVNVx7aJDgU/PcLxxFvs93YzmD/O9aJItgJvSJEmrck9HBSBNiex9gU+PXOtNRrb7Qnt9P02yWbvs/sBhbanDhe0/vHvTfN5OmHqaTZJfjMT1Y5okd5zPtT9P4obmDOsC72q/QFwH3GHMflOxvBOgqn6W5Fcj235tgEkgzO0ejP18V9VlazXSfhr9Avlt4IM0ydPnq+pygCSfoxnO66vAW5P8F82Xw7kO6QXwFeDgNnF7FHB8VV2Z5BHAjmlL6IGNge2BnwPbtbFtD3ymqk5rt5lpn5nOMRrHI4DHtQUN0PyP2aq99v2BX9L8DXp4mvaSW9thUX20um2IlgC7VdWVowvHlARN75o8rqvytdxQVb3+LOf8y8jr67jhqSKnVNUrpm+c5O7AI4EX0rQbeS6wB/B3NCVYr0lyl+sDq7o2yduAf5sl5unxTcW0Ylp8K2je4+to/jk/fYZrunzk9UeAPavq1DZBe9AM+8zm8pVvMqPFcE/XoUnYDl2IezrNPYEz2rj+NEOJxPSYMu3nyrYf/RxMfQZm2+e6kW1eClxIU7q1BJjpKTmzxbI697/P5nIPxn6+NSdXTv99yQzF/lX187a0cHfgP5McU1VvGLftmH2vSnIsze/4XjSl4tB85verqht9OU7Toe8XVXWPJLcFjk3yuKo6YqZ92v3GneNGmwBPmp7ctSWIO9OUOn+Npnbg+TRf6KTeWd3OIscAL5qaaUsxxnlKkiVp2ulsS1OlO925NCVAAE8es/5GkryjffkB4AnAfmnaex2R5CtJbpfkVsCSqvos8BpgpyRLgC3b9mL/yg0lb6M+QlNysOnIsguT3Knd/wkri2+a7wP3a6tPSdPub6aSng2B89P0ett7ZPll7TraNpF/TNuuDHgmzeP1FkJn93SabwBPTnLrNo5bruY9JckDadoHvr+qLgV+meQp7bq0CeZsjgf2StPuaFOaxPOH87yuldkYOL8tjXwmMNWB6Pr7PxLL3gDtZ2krxt8D3dhcP9+am+OBPdu/aRvQ/G38dpK/Ba6oqo8BbwV2GrPvNZm5d+/hNFXOD+CGWpGjgRdM7ZPkDu05r9eWCB8AvGIO+4w7x6ijaf6vTNUi3LM9x9U0Dy54Ks3f9m8DL2PuDzKQFpXVLRHcH3h3mvZq69D8URjXk+xMmkRlM5o2WVeN+SL5VuBTSZ4JfHMO5/5fmmTt/cAF3NAucAlNSdRtgSuBD7eJAjR/HJYCH2urV0PTXuhPo/FU1dVphhA4aOR8B9B0JlkO/IQxicZMqup3beneYW1VBDRtBn8+ZvPXAD8AfkVTpTX1z/9w4P1pOlM8maYN4yFtlcQ5NH/QFkKX9/R6VfXTJK+mafu2BLiGpgRwvvd0ryT3B25GU5XzpKo6o913b+C97XnWpXmPT50lrM/TtL87laYE9F+r6oK0nUwWyHuAz7YJ6re4oXTvNODaJKfSfFF5D839/zHN5/3ZbfvIBQxlIs318605qKqT03SKm/pC9IGqOiXJI2k6t62g+d19wZjdlwGnJTm5qvaetu4Y4KPAEW3iBc2X/q2Bk9vk7Hc0oylM9wXgde0X5dn2GXeOUf8OvKONMTRfbKc6xn0beGhVXZHk2zTNf0wE1UtrfEDp9o/E9Z0c1sDxnwAc1bYx01qwpu+pJElaOzobR3ABPQ74eZL/TbJHJmTsNEmSpDVtUh4xty7NcAZ70fSm/FpVOZ6TJEnSLCYiEYTrk8FH0Tb+rapNV7KLJEnSoPW+ajjJo9o2a2fTdKL4AE1HEUmSJM2i9yWCaR4ndjjwFTuMSJIkzV3vE0GAJLcDtq+qrye5Kc2ju3xSgCRJ0iwmoWr4+TTPgX1fu2gLZn/EmCRJkpiARJBmkOH70Tzvlao6C7j1/2/vDl6tqsIwjD9v3cpb5CxBKhpkUIQ3tEGIOgk0xYiCsnFYEUTUMKXG/QllKBUWGEYhgmUFYRaRg6J0ENqgBkFBIy9Kavo1uPuWgqBl13XWuc8PDmevcybvnr2stddeTRNJkiR1YByK4Mlz3wo/vEew//VuSZKkOTYORXB/ki3AZJI1wC5gT+NMkiRJI6/7zSLDmbObgLXMnDO7j5nzLvu+MUmSpDnWfRG8kCQrq+rL1jkkSZJGWbfn8ia5GtgI3Ax8VFWHkzwIbAEmgWUt80mSJI26bmcEh9NEbgUOAvcBPwMrgBerytfHSJIkXUTPRfAwMFVVZ5MsAH4HllTVr42jSZIkdaHnXcOnquosQFX9ARyxBEqSJF26nmcETwA/zg6B24dxgKqqqVbZJEmSetDtZhHgrtYBJEmSetbtjOC5ktwG3FFVnyaZBCaqarp1LkmSpFHW8zOCACR5CngP2Dr8dAvgrmFJkqSL6L4IAs8CK4FjAFV1FFjUNJEkSVIHxqEInqyqU7ODJBNA/+vdkiRJc2wciuD+JFuAySRrgF3AnsaZJEmSRl73m0WSXAVsAtYy8+qYfcC26v3GJEmS5tg4FMFHgL1VdbJ1FkmSpJ6Mw9LwQ8CRJDuSbBieEZQkSdJFdD8jCJDkGmA98DiwCvikqp5sm0qSJGm0jUURhL/L4DrgCWB1Vd3UOJIkSdJI635pOMm6JG8yc87wo8A2YHHTUJIkSR3ofkYwyU5gJ/ChG0YkSZIuXfdFUJIkSf9Ntztsk3xRVauSTHP+SSIBqqoWNoomSZLUBWcEJUmS5qlx2Cyy41J+kyRJ0vm6L4LA3ecOhhdK39soiyRJUje6LYJJNg/PB04lOTZ8poHfgN2N40mSJI287p8RTPJKVW1unUOSJKk33RbBJHdW1Q9Jll/o/6r65kpnkiRJ6knPRfD1qno6yWcX+Luq6v4rHkqSJKkj3RZBSZIkXZ5uN4vMSvJYkhuH65eSvJ9kWetckiRJo677Igi8XFXTSVYBDwBvAa81ziRJkjTyxqEInhm+NwCvVtVu4NqGeSRJkrowDkXwlyRbgY3A3iTXMR73JUmSNKe63yyS5HpgHXCoqo4mWQwsraqPG0eTJEkaad0XQYAk9wCrh+GBqvquZR5JkqQedL+EmuR54B1g0fB5O8lzbVNJkiSNvu5nBJN8D6yoquPD+Abgq6qaaptMkiRptHU/IwiEf3YOM1ynURZJkqRuTLQO8D94A/g6yQfD+GFge8M8kiRJXeh+aRggyXJgFTMzgZ9X1beNI0mSJI28botgkgXAM8AS4BCwvar+bJtKkiSpHz0XwXeB08ABYD3wU1W90DaVJElSP3ougoeqaulwPQEcrKrljWNJkiR1o+ddw6dnL1wSliRJ+vd6nhE8AxyfHQKTwInhuqpqYatskiRJPei2CEqSJOny9Lw0LEmSpMtgEZQkSZqnLIKSJEnzlEVQkiRpnrIISpIkzVN/ARgMXACmIaC9AAAAAElFTkSuQmCC\n","text/plain":"<Figure size 792x648 with 2 Axes>"},"metadata":{"needs_background":"light"},"output_type":"display_data"}]},{"metadata":{},"cell_type":"markdown","source":"## We can see some of our features have decent correlation (remember that we aren't using the reviews yet). Let's try a basic model\n### First, let's log some important information in our <code>run</code>\n<blockquote>H2O Has a <a href=https://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science.html>lot</a> of algorithms, so we'll use a Gradient Boosting Estimator<br>We'll log some things like our feature vector, label, train/test/validation split, training time, and even the model and notebook themselves directly to <a href='/mlflow'>mlflow</a></blockquote>"},{"metadata":{"trusted":false},"cell_type":"code","source":"%%time\nfrom h2o.estimators import H2OGradientBoostingEstimator\nfrom splicemachine.mlflow_support.utilities import get_user\n\nRATIOS = [0.7,0.15]\n\n# Start our run to keep track of important information\nmlflow.start_run(run_name='simple_run')\n\npredictors = ['ProductId', 'UserId', 'HelpfulnessNumerator', 'HelpfulnessDenominator', 'Time']\nresponse = 'PositiveReview'\n\n# lp is short for log_param\n# lm is short for log_metric\nmlflow.lp('predictors', predictors)\nmlflow.lp('label', response)\nmlflow.lp('source data table', f'{get_user()}.AMAZON_REVIEWS')\n\n# Train Test Split\ntrain,test,valid = reviews.split_frame(ratios=RATIOS)\n# Log our ratios\nmlflow.lp('ratios',RATIOS)\n\ngbm_baseline = H2OGradientBoostingEstimator(stopping_metric = \"AUC\", stopping_tolerance = 0.001,\n stopping_rounds = 5, score_tree_interval = 10,\n model_id = \"gbm_baseline.hex\"\n )\n\nmlflow.lp('model_type', gbm_baseline.__class__)\n\n# Code block to time training\nwith mlflow.timer('train_time'):\n gbm_baseline.train(x = predictors, y = response, \n training_frame = train, validation_frame = test\n )\n# Log the model params to mlflow\nmlflow.log_params(gbm_baseline.get_params())\n# Log the model to MLFlow\nmlflow.log_model(gbm_baseline, 'baseline_model')\n# Log the training notebook to MLFlow\nmlflow.log_artifact('MLManager H2O Demo.ipynb', 'training_notebook')\ngbm_baseline","execution_count":15,"outputs":[{"name":"stdout","output_type":"stream","text":"Starting Code Block train_time... gbm Model Build progress: |███████████████████████████████████████████████| 100%\nDone.\nCode Block train_time:\nRan in 3.391 secs\nRan in 0.057 mins\nSaving artifact of size: 1579.965 KB to Splice Machine DB\nSaving artifact of size: 777.102 KB to Splice Machine DB\nCPU times: user 577 ms, sys: 74.8 ms, total: 651 ms\nWall time: 9.24 s\nModel Details\n=============\nH2OGradientBoostingEstimator : Gradient Boosting Machine\nModel Key: gbm_baseline.hex\n\n\nModel Summary: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>number_of_trees</th>\n <th>number_of_internal_trees</th>\n <th>model_size_in_bytes</th>\n <th>min_depth</th>\n <th>max_depth</th>\n <th>mean_depth</th>\n <th>min_leaves</th>\n <th>max_leaves</th>\n <th>mean_leaves</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>50.0</td>\n <td>50.0</td>\n <td>24560.0</td>\n <td>5.0</td>\n <td>5.0</td>\n <td>5.0</td>\n <td>20.0</td>\n <td>32.0</td>\n <td>29.62</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" number_of_trees number_of_internal_trees model_size_in_bytes \\\n0 50.0 50.0 24560.0 \n\n min_depth max_depth mean_depth min_leaves max_leaves mean_leaves \n0 5.0 5.0 5.0 20.0 32.0 29.62 "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\n\nModelMetricsBinomial: gbm\n** Reported on train data. **\n\nMSE: 0.1374345189782943\nRMSE: 0.370721619248587\nLogLoss: 0.43951689132292565\nMean Per-Class Error: 0.31278195917903695\nAUC: 0.7540414210595864\nAUCPR: 0.9015791977529423\nGini: 0.5080828421191728\n\nConfusion Matrix (Act/Pred) for max f1 @ threshold = 0.5357817933564483: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>0</th>\n <th>1</th>\n <th>Error</th>\n <th>Rate</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>4288.0</td>\n <td>10939.0</td>\n <td>0.7184</td>\n <td>(10939.0/15227.0)</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>1586.0</td>\n <td>53052.0</td>\n <td>0.029</td>\n <td>(1586.0/54638.0)</td>\n </tr>\n <tr>\n <th>2</th>\n <td>Total</td>\n <td>5874.0</td>\n <td>63991.0</td>\n <td>0.1793</td>\n <td>(12525.0/69865.0)</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" 0 1 Error Rate\n0 0 4288.0 10939.0 0.7184 (10939.0/15227.0)\n1 1 1586.0 53052.0 0.029 (1586.0/54638.0)\n2 Total 5874.0 63991.0 0.1793 (12525.0/69865.0)"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nMaximum Metrics: Maximum metrics at their respective thresholds\n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>metric</th>\n <th>threshold</th>\n <th>value</th>\n <th>idx</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>max f1</td>\n <td>0.535782</td>\n <td>0.894419</td>\n <td>262.0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>max f2</td>\n <td>0.227611</td>\n <td>0.949461</td>\n <td>357.0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>max f0point5</td>\n <td>0.743701</td>\n <td>0.864134</td>\n <td>173.0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>max accuracy</td>\n <td>0.587676</td>\n <td>0.821957</td>\n <td>240.0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>max precision</td>\n <td>0.950400</td>\n <td>1.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>5</th>\n <td>max recall</td>\n <td>0.079913</td>\n <td>1.000000</td>\n <td>398.0</td>\n </tr>\n <tr>\n <th>6</th>\n <td>max specificity</td>\n <td>0.950400</td>\n <td>1.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>7</th>\n <td>max absolute_mcc</td>\n <td>0.627212</td>\n <td>0.391565</td>\n <td>222.0</td>\n </tr>\n <tr>\n <th>8</th>\n <td>max min_per_class_accuracy</td>\n <td>0.815919</td>\n <td>0.680108</td>\n <td>115.0</td>\n </tr>\n <tr>\n <th>9</th>\n <td>max mean_per_class_accuracy</td>\n <td>0.784375</td>\n <td>0.687218</td>\n <td>146.0</td>\n </tr>\n <tr>\n <th>10</th>\n <td>max tns</td>\n <td>0.950400</td>\n <td>15227.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>11</th>\n <td>max fns</td>\n <td>0.950400</td>\n <td>54632.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>12</th>\n <td>max fps</td>\n <td>0.067846</td>\n <td>15227.000000</td>\n <td>399.0</td>\n </tr>\n <tr>\n <th>13</th>\n <td>max tps</td>\n <td>0.079913</td>\n <td>54638.000000</td>\n <td>398.0</td>\n </tr>\n <tr>\n <th>14</th>\n <td>max tnr</td>\n <td>0.950400</td>\n <td>1.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>15</th>\n <td>max fnr</td>\n <td>0.950400</td>\n <td>0.999890</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>16</th>\n <td>max fpr</td>\n <td>0.067846</td>\n <td>1.000000</td>\n <td>399.0</td>\n </tr>\n <tr>\n <th>17</th>\n <td>max tpr</td>\n <td>0.079913</td>\n <td>1.000000</td>\n <td>398.0</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" metric threshold value idx\n0 max f1 0.535782 0.894419 262.0\n1 max f2 0.227611 0.949461 357.0\n2 max f0point5 0.743701 0.864134 173.0\n3 max accuracy 0.587676 0.821957 240.0\n4 max precision 0.950400 1.000000 0.0\n5 max recall 0.079913 1.000000 398.0\n6 max specificity 0.950400 1.000000 0.0\n7 max absolute_mcc 0.627212 0.391565 222.0\n8 max min_per_class_accuracy 0.815919 0.680108 115.0\n9 max mean_per_class_accuracy 0.784375 0.687218 146.0\n10 max tns 0.950400 15227.000000 0.0\n11 max fns 0.950400 54632.000000 0.0\n12 max fps 0.067846 15227.000000 399.0\n13 max tps 0.079913 54638.000000 398.0\n14 max tnr 0.950400 1.000000 0.0\n15 max fnr 0.950400 0.999890 0.0\n16 max fpr 0.067846 1.000000 399.0\n17 max tpr 0.079913 1.000000 398.0"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nGains/Lift Table: Avg response rate: 78.21 %, avg score: 78.19 %\n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>group</th>\n <th>cumulative_data_fraction</th>\n <th>lower_threshold</th>\n <th>lift</th>\n <th>cumulative_lift</th>\n <th>response_rate</th>\n <th>score</th>\n <th>cumulative_response_rate</th>\n <th>cumulative_score</th>\n <th>capture_rate</th>\n <th>cumulative_capture_rate</th>\n <th>gain</th>\n <th>cumulative_gain</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>1</td>\n <td>0.010005</td>\n <td>0.920836</td>\n <td>1.249420</td>\n <td>1.249420</td>\n <td>0.977110</td>\n <td>0.928429</td>\n <td>0.977110</td>\n <td>0.928429</td>\n <td>0.012500</td>\n <td>0.012500</td>\n <td>24.941984</td>\n <td>24.941984</td>\n </tr>\n <tr>\n <th>1</th>\n <td></td>\n <td>2</td>\n <td>0.020010</td>\n <td>0.914269</td>\n <td>1.242103</td>\n <td>1.245761</td>\n <td>0.971388</td>\n <td>0.917213</td>\n <td>0.974249</td>\n <td>0.922821</td>\n <td>0.012427</td>\n <td>0.024928</td>\n <td>24.210259</td>\n <td>24.576122</td>\n </tr>\n <tr>\n <th>2</th>\n <td></td>\n <td>3</td>\n <td>0.030001</td>\n <td>0.911259</td>\n <td>1.232891</td>\n <td>1.241475</td>\n <td>0.964183</td>\n <td>0.912706</td>\n <td>0.970897</td>\n <td>0.919453</td>\n <td>0.012317</td>\n <td>0.037245</td>\n <td>23.289051</td>\n <td>24.147508</td>\n </tr>\n <tr>\n <th>3</th>\n <td></td>\n <td>4</td>\n <td>0.041065</td>\n <td>0.908798</td>\n <td>1.214175</td>\n <td>1.234120</td>\n <td>0.949547</td>\n <td>0.909789</td>\n <td>0.965145</td>\n <td>0.916849</td>\n <td>0.013434</td>\n <td>0.050679</td>\n <td>21.417542</td>\n <td>23.411968</td>\n </tr>\n <tr>\n <th>4</th>\n <td></td>\n <td>5</td>\n <td>0.050025</td>\n <td>0.906579</td>\n <td>1.196983</td>\n <td>1.227468</td>\n <td>0.936102</td>\n <td>0.907585</td>\n <td>0.959943</td>\n <td>0.915190</td>\n <td>0.010725</td>\n <td>0.061404</td>\n <td>19.698347</td>\n <td>22.746810</td>\n </tr>\n <tr>\n <th>5</th>\n <td></td>\n <td>6</td>\n <td>0.100222</td>\n <td>0.899416</td>\n <td>1.189724</td>\n <td>1.208564</td>\n <td>0.930425</td>\n <td>0.903090</td>\n <td>0.945159</td>\n <td>0.909129</td>\n <td>0.059720</td>\n <td>0.121124</td>\n <td>18.972388</td>\n <td>20.856364</td>\n </tr>\n <tr>\n <th>6</th>\n <td></td>\n <td>7</td>\n <td>0.150390</td>\n <td>0.890206</td>\n <td>1.172162</td>\n <td>1.196420</td>\n <td>0.916690</td>\n <td>0.895343</td>\n <td>0.935662</td>\n <td>0.904530</td>\n <td>0.058805</td>\n <td>0.179930</td>\n <td>17.216182</td>\n <td>19.642047</td>\n </tr>\n <tr>\n <th>7</th>\n <td></td>\n <td>8</td>\n <td>0.200029</td>\n <td>0.877669</td>\n <td>1.168813</td>\n <td>1.189569</td>\n <td>0.914072</td>\n <td>0.883597</td>\n <td>0.930304</td>\n <td>0.899336</td>\n <td>0.058018</td>\n <td>0.237948</td>\n <td>16.881303</td>\n <td>18.956947</td>\n </tr>\n <tr>\n <th>8</th>\n <td></td>\n <td>9</td>\n <td>0.301939</td>\n <td>0.855812</td>\n <td>1.136273</td>\n <td>1.171581</td>\n <td>0.888624</td>\n <td>0.865949</td>\n <td>0.916236</td>\n <td>0.888067</td>\n <td>0.115799</td>\n <td>0.353746</td>\n <td>13.627306</td>\n <td>17.158083</td>\n </tr>\n <tr>\n <th>9</th>\n <td></td>\n <td>10</td>\n <td>0.401245</td>\n <td>0.840171</td>\n <td>1.112817</td>\n <td>1.157037</td>\n <td>0.870280</td>\n <td>0.846429</td>\n <td>0.904862</td>\n <td>0.877762</td>\n <td>0.110509</td>\n <td>0.464256</td>\n <td>11.281682</td>\n <td>15.703709</td>\n </tr>\n <tr>\n <th>10</th>\n <td></td>\n <td>11</td>\n <td>0.500265</td>\n <td>0.830372</td>\n <td>1.084427</td>\n <td>1.142665</td>\n <td>0.848077</td>\n <td>0.835059</td>\n <td>0.893623</td>\n <td>0.869309</td>\n <td>0.107379</td>\n <td>0.571635</td>\n <td>8.442719</td>\n <td>14.266510</td>\n </tr>\n <tr>\n <th>11</th>\n <td></td>\n <td>12</td>\n <td>0.601789</td>\n <td>0.815788</td>\n <td>1.072815</td>\n <td>1.130881</td>\n <td>0.838996</td>\n <td>0.823077</td>\n <td>0.884407</td>\n <td>0.861510</td>\n <td>0.108917</td>\n <td>0.680552</td>\n <td>7.281506</td>\n <td>13.088111</td>\n </tr>\n <tr>\n <th>12</th>\n <td></td>\n <td>13</td>\n <td>0.699993</td>\n <td>0.795914</td>\n <td>1.019820</td>\n <td>1.115300</td>\n <td>0.797551</td>\n <td>0.807826</td>\n <td>0.872222</td>\n <td>0.853978</td>\n <td>0.100150</td>\n <td>0.780702</td>\n <td>1.982003</td>\n <td>11.530008</td>\n </tr>\n <tr>\n <th>13</th>\n <td></td>\n <td>14</td>\n <td>0.800000</td>\n <td>0.761463</td>\n <td>0.985873</td>\n <td>1.099121</td>\n <td>0.771003</td>\n <td>0.781952</td>\n <td>0.859568</td>\n <td>0.844974</td>\n <td>0.098594</td>\n <td>0.879296</td>\n <td>-1.412671</td>\n <td>9.912058</td>\n </tr>\n <tr>\n <th>14</th>\n <td></td>\n <td>15</td>\n <td>0.899993</td>\n <td>0.586119</td>\n <td>0.822746</td>\n <td>1.068414</td>\n <td>0.643430</td>\n <td>0.676802</td>\n <td>0.835555</td>\n <td>0.826290</td>\n <td>0.082269</td>\n <td>0.961565</td>\n <td>-17.725361</td>\n <td>6.841429</td>\n </tr>\n <tr>\n <th>15</th>\n <td></td>\n <td>16</td>\n <td>1.000000</td>\n <td>0.060736</td>\n <td>0.384320</td>\n <td>1.000000</td>\n <td>0.300558</td>\n <td>0.382875</td>\n <td>0.782051</td>\n <td>0.781945</td>\n <td>0.038435</td>\n <td>1.000000</td>\n <td>-61.567961</td>\n <td>0.000000</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" group cumulative_data_fraction lower_threshold lift \\\n0 1 0.010005 0.920836 1.249420 \n1 2 0.020010 0.914269 1.242103 \n2 3 0.030001 0.911259 1.232891 \n3 4 0.041065 0.908798 1.214175 \n4 5 0.050025 0.906579 1.196983 \n5 6 0.100222 0.899416 1.189724 \n6 7 0.150390 0.890206 1.172162 \n7 8 0.200029 0.877669 1.168813 \n8 9 0.301939 0.855812 1.136273 \n9 10 0.401245 0.840171 1.112817 \n10 11 0.500265 0.830372 1.084427 \n11 12 0.601789 0.815788 1.072815 \n12 13 0.699993 0.795914 1.019820 \n13 14 0.800000 0.761463 0.985873 \n14 15 0.899993 0.586119 0.822746 \n15 16 1.000000 0.060736 0.384320 \n\n cumulative_lift response_rate score cumulative_response_rate \\\n0 1.249420 0.977110 0.928429 0.977110 \n1 1.245761 0.971388 0.917213 0.974249 \n2 1.241475 0.964183 0.912706 0.970897 \n3 1.234120 0.949547 0.909789 0.965145 \n4 1.227468 0.936102 0.907585 0.959943 \n5 1.208564 0.930425 0.903090 0.945159 \n6 1.196420 0.916690 0.895343 0.935662 \n7 1.189569 0.914072 0.883597 0.930304 \n8 1.171581 0.888624 0.865949 0.916236 \n9 1.157037 0.870280 0.846429 0.904862 \n10 1.142665 0.848077 0.835059 0.893623 \n11 1.130881 0.838996 0.823077 0.884407 \n12 1.115300 0.797551 0.807826 0.872222 \n13 1.099121 0.771003 0.781952 0.859568 \n14 1.068414 0.643430 0.676802 0.835555 \n15 1.000000 0.300558 0.382875 0.782051 \n\n cumulative_score capture_rate cumulative_capture_rate gain \\\n0 0.928429 0.012500 0.012500 24.941984 \n1 0.922821 0.012427 0.024928 24.210259 \n2 0.919453 0.012317 0.037245 23.289051 \n3 0.916849 0.013434 0.050679 21.417542 \n4 0.915190 0.010725 0.061404 19.698347 \n5 0.909129 0.059720 0.121124 18.972388 \n6 0.904530 0.058805 0.179930 17.216182 \n7 0.899336 0.058018 0.237948 16.881303 \n8 0.888067 0.115799 0.353746 13.627306 \n9 0.877762 0.110509 0.464256 11.281682 \n10 0.869309 0.107379 0.571635 8.442719 \n11 0.861510 0.108917 0.680552 7.281506 \n12 0.853978 0.100150 0.780702 1.982003 \n13 0.844974 0.098594 0.879296 -1.412671 \n14 0.826290 0.082269 0.961565 -17.725361 \n15 0.781945 0.038435 1.000000 -61.567961 \n\n cumulative_gain \n0 24.941984 \n1 24.576122 \n2 24.147508 \n3 23.411968 \n4 22.746810 \n5 20.856364 \n6 19.642047 \n7 18.956947 \n8 17.158083 \n9 15.703709 \n10 14.266510 \n11 13.088111 \n12 11.530008 \n13 9.912058 \n14 6.841429 \n15 0.000000 "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\n\nModelMetricsBinomial: gbm\n** Reported on validation data. **\n\nMSE: 0.14223001006483346\nRMSE: 0.3771339418095824\nLogLoss: 0.4525533321024546\nMean Per-Class Error: 0.32591627240012655\nAUC: 0.7316894311747509\nAUCPR: 0.8839714249918257\nGini: 0.4633788623495019\n\nConfusion Matrix (Act/Pred) for max f1 @ threshold = 0.45593713188449025: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>0</th>\n <th>1</th>\n <th>Error</th>\n <th>Rate</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>677.0</td>\n <td>2672.0</td>\n <td>0.7979</td>\n <td>(2672.0/3349.0)</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>203.0</td>\n <td>11587.0</td>\n <td>0.0172</td>\n <td>(203.0/11790.0)</td>\n </tr>\n <tr>\n <th>2</th>\n <td>Total</td>\n <td>880.0</td>\n <td>14259.0</td>\n <td>0.1899</td>\n <td>(2875.0/15139.0)</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" 0 1 Error Rate\n0 0 677.0 2672.0 0.7979 (2672.0/3349.0)\n1 1 203.0 11587.0 0.0172 (203.0/11790.0)\n2 Total 880.0 14259.0 0.1899 (2875.0/15139.0)"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nMaximum Metrics: Maximum metrics at their respective thresholds\n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>metric</th>\n <th>threshold</th>\n <th>value</th>\n <th>idx</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>max f1</td>\n <td>0.455937</td>\n <td>0.889631</td>\n <td>293.0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>max f2</td>\n <td>0.325802</td>\n <td>0.948460</td>\n <td>334.0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>max f0point5</td>\n <td>0.735794</td>\n <td>0.859811</td>\n <td>181.0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>max accuracy</td>\n <td>0.587274</td>\n <td>0.813990</td>\n <td>246.0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>max precision</td>\n <td>0.934944</td>\n <td>0.966667</td>\n <td>4.0</td>\n </tr>\n <tr>\n <th>5</th>\n <td>max recall</td>\n <td>0.094890</td>\n <td>1.000000</td>\n <td>395.0</td>\n </tr>\n <tr>\n <th>6</th>\n <td>max specificity</td>\n <td>0.950525</td>\n <td>0.999701</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>7</th>\n <td>max absolute_mcc</td>\n <td>0.674930</td>\n <td>0.375372</td>\n <td>205.0</td>\n </tr>\n <tr>\n <th>8</th>\n <td>max min_per_class_accuracy</td>\n <td>0.818016</td>\n <td>0.661578</td>\n <td>117.0</td>\n </tr>\n <tr>\n <th>9</th>\n <td>max mean_per_class_accuracy</td>\n <td>0.783111</td>\n <td>0.674084</td>\n <td>151.0</td>\n </tr>\n <tr>\n <th>10</th>\n <td>max tns</td>\n <td>0.950525</td>\n <td>3348.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>11</th>\n <td>max fns</td>\n <td>0.950525</td>\n <td>11789.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>12</th>\n <td>max fps</td>\n <td>0.056394</td>\n <td>3349.000000</td>\n <td>399.0</td>\n </tr>\n <tr>\n <th>13</th>\n <td>max tps</td>\n <td>0.094890</td>\n <td>11790.000000</td>\n <td>395.0</td>\n </tr>\n <tr>\n <th>14</th>\n <td>max tnr</td>\n <td>0.950525</td>\n <td>0.999701</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>15</th>\n <td>max fnr</td>\n <td>0.950525</td>\n <td>0.999915</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>16</th>\n <td>max fpr</td>\n <td>0.056394</td>\n <td>1.000000</td>\n <td>399.0</td>\n </tr>\n <tr>\n <th>17</th>\n <td>max tpr</td>\n <td>0.094890</td>\n <td>1.000000</td>\n <td>395.0</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" metric threshold value idx\n0 max f1 0.455937 0.889631 293.0\n1 max f2 0.325802 0.948460 334.0\n2 max f0point5 0.735794 0.859811 181.0\n3 max accuracy 0.587274 0.813990 246.0\n4 max precision 0.934944 0.966667 4.0\n5 max recall 0.094890 1.000000 395.0\n6 max specificity 0.950525 0.999701 0.0\n7 max absolute_mcc 0.674930 0.375372 205.0\n8 max min_per_class_accuracy 0.818016 0.661578 117.0\n9 max mean_per_class_accuracy 0.783111 0.674084 151.0\n10 max tns 0.950525 3348.000000 0.0\n11 max fns 0.950525 11789.000000 0.0\n12 max fps 0.056394 3349.000000 399.0\n13 max tps 0.094890 11790.000000 395.0\n14 max tnr 0.950525 0.999701 0.0\n15 max fnr 0.950525 0.999915 0.0\n16 max fpr 0.056394 1.000000 399.0\n17 max tpr 0.094890 1.000000 395.0"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nGains/Lift Table: Avg response rate: 77.88 %, avg score: 78.37 %\n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>group</th>\n <th>cumulative_data_fraction</th>\n <th>lower_threshold</th>\n <th>lift</th>\n <th>cumulative_lift</th>\n <th>response_rate</th>\n <th>score</th>\n <th>cumulative_response_rate</th>\n <th>cumulative_score</th>\n <th>capture_rate</th>\n <th>cumulative_capture_rate</th>\n <th>gain</th>\n <th>cumulative_gain</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>1</td>\n <td>0.010040</td>\n <td>0.920851</td>\n <td>1.174234</td>\n <td>1.174234</td>\n <td>0.914474</td>\n <td>0.928367</td>\n <td>0.914474</td>\n <td>0.928367</td>\n <td>0.011790</td>\n <td>0.011790</td>\n <td>17.423385</td>\n <td>17.423385</td>\n </tr>\n <tr>\n <th>1</th>\n <td></td>\n <td>2</td>\n <td>0.020015</td>\n <td>0.913837</td>\n <td>1.207521</td>\n <td>1.190823</td>\n <td>0.940397</td>\n <td>0.917088</td>\n <td>0.927393</td>\n <td>0.922746</td>\n <td>0.012044</td>\n <td>0.023834</td>\n <td>20.752125</td>\n <td>19.082262</td>\n </tr>\n <tr>\n <th>2</th>\n <td></td>\n <td>3</td>\n <td>0.030055</td>\n <td>0.910840</td>\n <td>1.174234</td>\n <td>1.185281</td>\n <td>0.914474</td>\n <td>0.912427</td>\n <td>0.923077</td>\n <td>0.919299</td>\n <td>0.011790</td>\n <td>0.035623</td>\n <td>17.423385</td>\n <td>18.528088</td>\n </tr>\n <tr>\n <th>3</th>\n <td></td>\n <td>4</td>\n <td>0.040756</td>\n <td>0.908798</td>\n <td>1.181013</td>\n <td>1.184160</td>\n <td>0.919753</td>\n <td>0.909681</td>\n <td>0.922204</td>\n <td>0.916774</td>\n <td>0.012638</td>\n <td>0.048261</td>\n <td>18.101289</td>\n <td>18.416027</td>\n </tr>\n <tr>\n <th>4</th>\n <td></td>\n <td>5</td>\n <td>0.050334</td>\n <td>0.906383</td>\n <td>1.142366</td>\n <td>1.176207</td>\n <td>0.889655</td>\n <td>0.907578</td>\n <td>0.916010</td>\n <td>0.915024</td>\n <td>0.010941</td>\n <td>0.059203</td>\n <td>14.236553</td>\n <td>17.620720</td>\n </tr>\n <tr>\n <th>5</th>\n <td></td>\n <td>6</td>\n <td>0.100007</td>\n <td>0.900004</td>\n <td>1.188433</td>\n <td>1.182280</td>\n <td>0.925532</td>\n <td>0.903123</td>\n <td>0.920740</td>\n <td>0.909113</td>\n <td>0.059033</td>\n <td>0.118236</td>\n <td>18.843322</td>\n <td>18.227984</td>\n </tr>\n <tr>\n <th>6</th>\n <td></td>\n <td>7</td>\n <td>0.150010</td>\n <td>0.890719</td>\n <td>1.151748</td>\n <td>1.172102</td>\n <td>0.896962</td>\n <td>0.895875</td>\n <td>0.912814</td>\n <td>0.904700</td>\n <td>0.057591</td>\n <td>0.175827</td>\n <td>15.174750</td>\n <td>17.210239</td>\n </tr>\n <tr>\n <th>7</th>\n <td></td>\n <td>8</td>\n <td>0.200013</td>\n <td>0.878214</td>\n <td>1.136481</td>\n <td>1.163197</td>\n <td>0.885073</td>\n <td>0.884291</td>\n <td>0.905878</td>\n <td>0.899598</td>\n <td>0.056828</td>\n <td>0.232655</td>\n <td>13.648133</td>\n <td>16.319713</td>\n </tr>\n <tr>\n <th>8</th>\n <td></td>\n <td>9</td>\n <td>0.302860</td>\n <td>0.855812</td>\n <td>1.143031</td>\n <td>1.156349</td>\n <td>0.890173</td>\n <td>0.866197</td>\n <td>0.900545</td>\n <td>0.888255</td>\n <td>0.117557</td>\n <td>0.350212</td>\n <td>14.303098</td>\n <td>15.634899</td>\n </tr>\n <tr>\n <th>9</th>\n <td></td>\n <td>10</td>\n <td>0.400555</td>\n <td>0.840171</td>\n <td>1.103471</td>\n <td>1.143452</td>\n <td>0.859364</td>\n <td>0.846380</td>\n <td>0.890501</td>\n <td>0.878042</td>\n <td>0.107803</td>\n <td>0.458015</td>\n <td>10.347058</td>\n <td>14.345203</td>\n </tr>\n <tr>\n <th>10</th>\n <td></td>\n <td>11</td>\n <td>0.500033</td>\n <td>0.830563</td>\n <td>1.082835</td>\n <td>1.131393</td>\n <td>0.843293</td>\n <td>0.835017</td>\n <td>0.881110</td>\n <td>0.869482</td>\n <td>0.107718</td>\n <td>0.565734</td>\n <td>8.283462</td>\n <td>13.139261</td>\n </tr>\n <tr>\n <th>11</th>\n <td></td>\n <td>12</td>\n <td>0.600634</td>\n <td>0.816120</td>\n <td>1.066532</td>\n <td>1.120529</td>\n <td>0.830598</td>\n <td>0.823761</td>\n <td>0.872649</td>\n <td>0.861825</td>\n <td>0.107294</td>\n <td>0.673028</td>\n <td>6.653228</td>\n <td>12.052906</td>\n </tr>\n <tr>\n <th>12</th>\n <td></td>\n <td>13</td>\n <td>0.699980</td>\n <td>0.799444</td>\n <td>1.033049</td>\n <td>1.108113</td>\n <td>0.804521</td>\n <td>0.809260</td>\n <td>0.862980</td>\n <td>0.854364</td>\n <td>0.102629</td>\n <td>0.775657</td>\n <td>3.304899</td>\n <td>10.811328</td>\n </tr>\n <tr>\n <th>13</th>\n <td></td>\n <td>14</td>\n <td>0.799987</td>\n <td>0.765323</td>\n <td>0.999086</td>\n <td>1.094484</td>\n <td>0.778071</td>\n <td>0.784247</td>\n <td>0.852366</td>\n <td>0.845599</td>\n <td>0.099915</td>\n <td>0.875573</td>\n <td>-0.091417</td>\n <td>9.448372</td>\n </tr>\n <tr>\n <th>14</th>\n <td></td>\n <td>15</td>\n <td>0.899993</td>\n <td>0.588424</td>\n <td>0.825221</td>\n <td>1.064563</td>\n <td>0.642668</td>\n <td>0.681231</td>\n <td>0.829064</td>\n <td>0.827334</td>\n <td>0.082528</td>\n <td>0.958100</td>\n <td>-17.477885</td>\n <td>6.456346</td>\n </tr>\n <tr>\n <th>15</th>\n <td></td>\n <td>16</td>\n <td>1.000000</td>\n <td>0.056394</td>\n <td>0.418971</td>\n <td>1.000000</td>\n <td>0.326288</td>\n <td>0.390840</td>\n <td>0.778783</td>\n <td>0.783682</td>\n <td>0.041900</td>\n <td>1.000000</td>\n <td>-58.102852</td>\n <td>0.000000</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" group cumulative_data_fraction lower_threshold lift \\\n0 1 0.010040 0.920851 1.174234 \n1 2 0.020015 0.913837 1.207521 \n2 3 0.030055 0.910840 1.174234 \n3 4 0.040756 0.908798 1.181013 \n4 5 0.050334 0.906383 1.142366 \n5 6 0.100007 0.900004 1.188433 \n6 7 0.150010 0.890719 1.151748 \n7 8 0.200013 0.878214 1.136481 \n8 9 0.302860 0.855812 1.143031 \n9 10 0.400555 0.840171 1.103471 \n10 11 0.500033 0.830563 1.082835 \n11 12 0.600634 0.816120 1.066532 \n12 13 0.699980 0.799444 1.033049 \n13 14 0.799987 0.765323 0.999086 \n14 15 0.899993 0.588424 0.825221 \n15 16 1.000000 0.056394 0.418971 \n\n cumulative_lift response_rate score cumulative_response_rate \\\n0 1.174234 0.914474 0.928367 0.914474 \n1 1.190823 0.940397 0.917088 0.927393 \n2 1.185281 0.914474 0.912427 0.923077 \n3 1.184160 0.919753 0.909681 0.922204 \n4 1.176207 0.889655 0.907578 0.916010 \n5 1.182280 0.925532 0.903123 0.920740 \n6 1.172102 0.896962 0.895875 0.912814 \n7 1.163197 0.885073 0.884291 0.905878 \n8 1.156349 0.890173 0.866197 0.900545 \n9 1.143452 0.859364 0.846380 0.890501 \n10 1.131393 0.843293 0.835017 0.881110 \n11 1.120529 0.830598 0.823761 0.872649 \n12 1.108113 0.804521 0.809260 0.862980 \n13 1.094484 0.778071 0.784247 0.852366 \n14 1.064563 0.642668 0.681231 0.829064 \n15 1.000000 0.326288 0.390840 0.778783 \n\n cumulative_score capture_rate cumulative_capture_rate gain \\\n0 0.928367 0.011790 0.011790 17.423385 \n1 0.922746 0.012044 0.023834 20.752125 \n2 0.919299 0.011790 0.035623 17.423385 \n3 0.916774 0.012638 0.048261 18.101289 \n4 0.915024 0.010941 0.059203 14.236553 \n5 0.909113 0.059033 0.118236 18.843322 \n6 0.904700 0.057591 0.175827 15.174750 \n7 0.899598 0.056828 0.232655 13.648133 \n8 0.888255 0.117557 0.350212 14.303098 \n9 0.878042 0.107803 0.458015 10.347058 \n10 0.869482 0.107718 0.565734 8.283462 \n11 0.861825 0.107294 0.673028 6.653228 \n12 0.854364 0.102629 0.775657 3.304899 \n13 0.845599 0.099915 0.875573 -0.091417 \n14 0.827334 0.082528 0.958100 -17.477885 \n15 0.783682 0.041900 1.000000 -58.102852 \n\n cumulative_gain \n0 17.423385 \n1 19.082262 \n2 18.528088 \n3 18.416027 \n4 17.620720 \n5 18.227984 \n6 17.210239 \n7 16.319713 \n8 15.634899 \n9 14.345203 \n10 13.139261 \n11 12.052906 \n12 10.811328 \n13 9.448372 \n14 6.456346 \n15 0.000000 "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\n\nScoring History: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>timestamp</th>\n <th>duration</th>\n <th>number_of_trees</th>\n <th>training_rmse</th>\n <th>training_logloss</th>\n <th>training_auc</th>\n <th>training_pr_auc</th>\n <th>training_lift</th>\n <th>training_classification_error</th>\n <th>validation_rmse</th>\n <th>validation_logloss</th>\n <th>validation_auc</th>\n <th>validation_pr_auc</th>\n <th>validation_lift</th>\n <th>validation_classification_error</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>2020-06-03 19:45:43</td>\n <td>0.079 sec</td>\n <td>0.0</td>\n <td>0.412852</td>\n <td>0.524300</td>\n <td>0.500000</td>\n <td>0.000000</td>\n <td>1.000000</td>\n <td>0.217949</td>\n <td>0.415079</td>\n <td>0.528475</td>\n <td>0.500000</td>\n <td>0.000000</td>\n <td>1.000000</td>\n <td>0.221217</td>\n </tr>\n <tr>\n <th>1</th>\n <td></td>\n <td>2020-06-03 19:45:44</td>\n <td>1.041 sec</td>\n <td>10.0</td>\n <td>0.385338</td>\n <td>0.468347</td>\n <td>0.722202</td>\n <td>0.886199</td>\n <td>1.227541</td>\n <td>0.191756</td>\n <td>0.388970</td>\n <td>0.475106</td>\n <td>0.714339</td>\n <td>0.877976</td>\n <td>1.225307</td>\n <td>0.197966</td>\n </tr>\n <tr>\n <th>2</th>\n <td></td>\n <td>2020-06-03 19:45:45</td>\n <td>1.574 sec</td>\n <td>20.0</td>\n <td>0.377800</td>\n <td>0.453883</td>\n <td>0.731858</td>\n <td>0.890685</td>\n <td>1.241394</td>\n <td>0.184985</td>\n <td>0.381771</td>\n <td>0.461558</td>\n <td>0.721626</td>\n <td>0.879695</td>\n <td>1.200674</td>\n <td>0.193672</td>\n </tr>\n <tr>\n <th>3</th>\n <td></td>\n <td>2020-06-03 19:45:45</td>\n <td>1.862 sec</td>\n <td>30.0</td>\n <td>0.374872</td>\n <td>0.448219</td>\n <td>0.736972</td>\n <td>0.892789</td>\n <td>1.242155</td>\n <td>0.183525</td>\n <td>0.379027</td>\n <td>0.456469</td>\n <td>0.725341</td>\n <td>0.880711</td>\n <td>1.168075</td>\n <td>0.190964</td>\n </tr>\n <tr>\n <th>4</th>\n <td></td>\n <td>2020-06-03 19:45:45</td>\n <td>2.150 sec</td>\n <td>40.0</td>\n <td>0.372867</td>\n <td>0.444067</td>\n <td>0.743907</td>\n <td>0.896736</td>\n <td>1.249503</td>\n <td>0.182438</td>\n <td>0.377871</td>\n <td>0.454143</td>\n <td>0.728786</td>\n <td>0.882622</td>\n <td>1.174234</td>\n <td>0.190237</td>\n </tr>\n <tr>\n <th>5</th>\n <td></td>\n <td>2020-06-03 19:45:46</td>\n <td>2.319 sec</td>\n <td>50.0</td>\n <td>0.370722</td>\n <td>0.439517</td>\n <td>0.754041</td>\n <td>0.901579</td>\n <td>1.249420</td>\n <td>0.179274</td>\n <td>0.377134</td>\n <td>0.452553</td>\n <td>0.731689</td>\n <td>0.883971</td>\n <td>1.174234</td>\n <td>0.189907</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" timestamp duration number_of_trees training_rmse \\\n0 2020-06-03 19:45:43 0.079 sec 0.0 0.412852 \n1 2020-06-03 19:45:44 1.041 sec 10.0 0.385338 \n2 2020-06-03 19:45:45 1.574 sec 20.0 0.377800 \n3 2020-06-03 19:45:45 1.862 sec 30.0 0.374872 \n4 2020-06-03 19:45:45 2.150 sec 40.0 0.372867 \n5 2020-06-03 19:45:46 2.319 sec 50.0 0.370722 \n\n training_logloss training_auc training_pr_auc training_lift \\\n0 0.524300 0.500000 0.000000 1.000000 \n1 0.468347 0.722202 0.886199 1.227541 \n2 0.453883 0.731858 0.890685 1.241394 \n3 0.448219 0.736972 0.892789 1.242155 \n4 0.444067 0.743907 0.896736 1.249503 \n5 0.439517 0.754041 0.901579 1.249420 \n\n training_classification_error validation_rmse validation_logloss \\\n0 0.217949 0.415079 0.528475 \n1 0.191756 0.388970 0.475106 \n2 0.184985 0.381771 0.461558 \n3 0.183525 0.379027 0.456469 \n4 0.182438 0.377871 0.454143 \n5 0.179274 0.377134 0.452553 \n\n validation_auc validation_pr_auc validation_lift \\\n0 0.500000 0.000000 1.000000 \n1 0.714339 0.877976 1.225307 \n2 0.721626 0.879695 1.200674 \n3 0.725341 0.880711 1.168075 \n4 0.728786 0.882622 1.174234 \n5 0.731689 0.883971 1.174234 \n\n validation_classification_error \n0 0.221217 \n1 0.197966 \n2 0.193672 \n3 0.190964 \n4 0.190237 \n5 0.189907 "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nVariable Importances: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>variable</th>\n <th>relative_importance</th>\n <th>scaled_importance</th>\n <th>percentage</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>HelpfulnessNumerator</td>\n <td>4752.182129</td>\n <td>1.000000</td>\n <td>0.398990</td>\n </tr>\n <tr>\n <th>1</th>\n <td>HelpfulnessDenominator</td>\n <td>4481.750977</td>\n <td>0.943093</td>\n <td>0.376284</td>\n </tr>\n <tr>\n <th>2</th>\n <td>Time</td>\n <td>1361.051514</td>\n <td>0.286406</td>\n <td>0.114273</td>\n </tr>\n <tr>\n <th>3</th>\n <td>ProductId</td>\n <td>844.423889</td>\n <td>0.177692</td>\n <td>0.070897</td>\n </tr>\n <tr>\n <th>4</th>\n <td>UserId</td>\n <td>471.133972</td>\n <td>0.099141</td>\n <td>0.039556</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" variable relative_importance scaled_importance percentage\n0 HelpfulnessNumerator 4752.182129 1.000000 0.398990\n1 HelpfulnessDenominator 4481.750977 0.943093 0.376284\n2 Time 1361.051514 0.286406 0.114273\n3 ProductId 844.423889 0.177692 0.070897\n4 UserId 471.133972 0.099141 0.039556"},"metadata":{},"output_type":"display_data"},{"data":{"text/plain":""},"execution_count":15,"metadata":{},"output_type":"execute_result"}]},{"metadata":{},"cell_type":"markdown","source":"## You can see above that H2O gives you loads of details about your model. Let's inspect it a bit more and log some results to MLFlow"},{"metadata":{"trusted":false},"cell_type":"code","source":"%matplotlib inline \n\n# Print and Log Model params\nparams = dict(zip(gbm_baseline.summary().col_header[1:],\n gbm_baseline.summary().cell_values[0][1:]))\nprint(gbm_baseline.summary())\nmlflow.log_params(params)\n","execution_count":16,"outputs":[{"name":"stdout","output_type":"stream","text":"\nModel Summary: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>number_of_trees</th>\n <th>number_of_internal_trees</th>\n <th>model_size_in_bytes</th>\n <th>min_depth</th>\n <th>max_depth</th>\n <th>mean_depth</th>\n <th>min_leaves</th>\n <th>max_leaves</th>\n <th>mean_leaves</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>50.0</td>\n <td>50.0</td>\n <td>24560.0</td>\n <td>5.0</td>\n <td>5.0</td>\n <td>5.0</td>\n <td>20.0</td>\n <td>32.0</td>\n <td>29.62</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" number_of_trees number_of_internal_trees model_size_in_bytes \\\n0 50.0 50.0 24560.0 \n\n min_depth max_depth mean_depth min_leaves max_leaves mean_leaves \n0 5.0 5.0 5.0 20.0 32.0 29.62 "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\n"}]},{"metadata":{"trusted":false},"cell_type":"code","source":"from IPython.core.display import HTML\n#Plot and Log Scoring history\ngbm_baseline.plot()\nprint(\"AUC on Validation Data: \" + str(round(gbm_baseline.auc(valid = True), 3)))\n# Log training and validation metrics over time\nfor step, row in gbm_baseline.scoring_history().iterrows():\n row_dict = row.to_dict()\n for r in row_dict:\n if 'train' in r or 'valid' in r:\n mlflow.log_metric(r, row_dict[r],step=step)\n\ncur_run = mlflow.current_run_id()\ncur_exp = mlflow.current_exp_id()\nlink = f'/mlflow/#/metric/training_auc?runs=[\"{cur_run}\"]&experiment={cur_exp}&plot_metric_keys=[\\\"training_logloss\\\",\\\"validation_logloss\\\",\\\"training_rmse\\\",\\\"validation_rmse\\\"]'\nHTML(f'<font size=\"+2\">See your metrics plot <a href={link}>here</a></font>') \n\n","execution_count":17,"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAYgAAAEXCAYAAAC3c9OwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU1fn48c+TjZCNEAhbQkiAAAnIEsImoOIK7gutYt2hfKlra22ly7daW3+1tlrUr621uLVSKXVBtLjVFVyQgKwJmxAgbGFPWEOS5/fHvUkmYbIQZjLJ5Hm/XvNi5t5z7z03tvPMOeee54iqYowxxtQUEugKGGOMaZ4sQBhjjPHKAoQxxhivLEAYY4zxygKEMcYYryxAGGOM8coChDG1EJFnROR/A3TtFBE5JCKhgbi+MWABwrRAIjJGRL4QkYMisk9EPheRYb6+jqpOU9Xf+Pq8IpIqIioiYTW2vygiv3WvvUVVY1S1rJ5z3SIiC31dR2MAwuovYkzzISJxwNvAD4A5QAQwFjju4+uE1vflHAxEJExVSwNdD9M8WQvCtDR9AFT1FVUtU9Wjqvq+qq6oKCAi3xeRPBEpFpFcEclyt2eIyCcickBEVovI5R7HvCgifxGR+SJyGBjn+YteRM4RkQIR+bGIFIrIDhG51eP4DiLylogUichiEfnt6fyyr9nKcFsKG9172iQi3xORDOAZYJTbHXXALdtORP4uIrtFZLOI/FJEQjzO87mI/ElE9gG/cVthZ3hcu5OIHBWRxMbW3wQHCxCmpVkHlInISyIyQUTae+4Uke8ADwI3AXHA5cBeEQkH3gLeBzoBdwGzRKSvx+HXAw8DsYC3L/cuQDsgCZgMPO1x/aeBw26Zm92XT4hINPAkMEFVY4EzgWWqmgdMA750u6Pi3UOecuvZEzgb529xq8cpRwAbcf4ODwGzgRs89k8C/ququ311D6ZlsgBhWhRVLQLGAAr8DdgtIvNEpLNbZArwqKouVscGVd0MjARigEdUtURVP8Lpqprkcfo3VfVzVS1X1WNeLn8CeEhVT6jqfOAQ0NcdSL4GeEBVj6hqLvBSA25nj9uaOeD++r++jrLlwAARaauqO1R1tbdCbl2uBX6mqsWqmg88BtzoUWy7qj6lqqWqetSt6/UVrQy37D8aUH8T5CxAmBZHVfNU9RZVTQYGAN2AGe7u7sC3Xg7rBmxV1XKPbZtxWgMVttZz6b01+uuP4ASdRJzxPM/j6zsXQEdVja94Af/0VkhVD+N86U8DdojIf0SkX23nxBmX2eyxrc77VNVFOK2fs93z9gbmNaD+JshZgDAtmqquAV7ECRTgfPn18lJ0O9Dd41cyQAqwzfN0jazGbqAUSPbY1r2R5/JKVd9T1QuArsAanNYTnFznPTgtnR4e2xpyny/hdDPdCLxaSwvKtDIWIEyLIiL93IHiZPdzd5xuoq/cIjOB+0RkqDh6i0gPoOJX8k9FJFxEzgEuw+l/Py3u006vAw+KSJT7K/ym0z1vBRHpLCKXu2MRx3G6tiqesNoFJItIhEdd5gAPi0ise+/3Ai/Xc5l/AFfhBIm/+6rupmWzAGFammKcQdZF7tNGXwGrgB8DqOq/cQaa/+mWnQskqGoJzoD1BJxf2X8GbnJbIL5wJ87A8E6cL9tX8N2jtyE497cd2Icz8Hy7u+8jYDWwU0T2uNvuwgmGG3EG2/8JPF/XBVS1AFiK07pY4KN6mxZObMEgY3xPRH4PdFFVnz3N5G8i8jzOAPYvA10X0zzYRDljfMDtVooAVgLDcB6DnRLQSp0CEUkFrgaGBLYmpjmxLiZjfCMWZxziMM4YwGPAmwGtUQOJyG9wuun+oKqbAl0f03xYF5MxxhivrAVhjDHGq6Aag+jYsaOmpqae2kFaBgdWQHg8xKT5pV7GGNNcLVmyZI+qes27FVQBIjU1lZycnFM/cOl9sHYGXPYqxKT6vF7GGNNcicjm2vZZFxNAvx+BhMCaxwJdE2OMaTYsQABEJUHqjfDtc3DMElgaYwxYgKiS8RMoOwbrngp0TYwxplkIqjGI09KuHyRfAev+DzJ+CuExga6RMa3WiRMnKCgo4NgxyxnoK5GRkSQnJxMeHt7gYyxAeMq8Hwrmwrd/c8YljDEBUVBQQGxsLKmpqYhIoKvT4qkqe/fupaCggLS0hj+taV1MnjqOhE5nQ95jUFYS6NoY02odO3aMDh06WHDwERGhQ4cOp9wiswBRU+Z0OLoNNntdu8UY00QsOPhWY/6eFiCA/YdL2HfYbTF0vQjiB0Hu76Ha4mPGGNO6tPoAUXzsBGMf/Zi/fuquUinijEUUrYFtbwW2csaYgNi7dy+DBw9m8ODBdOnShaSkpMrPJSUN636+9dZbWbt2bZ1lnn76aWbNmuWLKvtFUCXry87O1sbMpL7rlW/4ZG0hX/7sPGLahEF5KbzVByI7w4VfOEHDGNNk8vLyyMjICHQ1AHjwwQeJiYnhvvvuq7ZdVVFVQkJazu9sb39XEVmiqtneyrecO/OjyWPSKD5Wyr9z3LXcQ8Ig4z7Y+xXstsW1jDGODRs2MGDAAKZNm0ZWVhY7duxg6tSpZGdn079/fx566KHKsmPGjGHZsmWUlpYSHx/P9OnTGTRoEKNGjaKwsBCAX/7yl8yYMaOy/PTp0xk+fDh9+/bliy++AODw4cNcc801DBo0iEmTJpGdnc2yZcua5H7tMVdgcPd4hqW25/nPN3HTqFRCQwR63gorH3TGIjqdFegqGtNq/fqt1eRuL/LpOTO7xfHAZf0bdWxubi4vvPACzzzzDACPPPIICQkJlJaWMm7cOCZOnEhmZma1Yw4ePMjZZ5/NI488wr333svzzz/P9OnTTzq3qvL1118zb948HnroId59912eeuopunTpwmuvvcby5cvJyspqVL0bw1oQrsljerJ131E+yN3pbAhrC33vge3zYf+KwFbOGNNs9OrVi2HDhlV+fuWVV8jKyiIrK4u8vDxyc3NPOqZt27ZMmDABgKFDh5Kfn+/13FdfffVJZRYuXMh1110HwKBBg+jfv3GBrTGsBeG6ILMzKQlRzFywifEDujob+9wOuY9A3qNw5suBraAxrVRjf+n7S3R0dOX79evX88QTT/D1118THx/PDTfc4HWuQUREROX70NBQSktLvZ67TZs2J5UJ5DixtSBcoSHCraNTydm8n2+27Hc2RrSH3lNh82w4lB/Q+hljmp+ioiJiY2OJi4tjx44dvPfeez6/xpgxY5gzZw4AK1eu9NpC8RcLEB6+k92d2MgwnlvosSyvpQI3xtQiKyuLzMxMBgwYwPe//31Gjx7t82vcddddbNu2jYEDB/LYY48xYMAA2rVr5/PreOPXx1xFZDzwBBAKzFTVR2rsPwdnYfeKb+TXVfUhEekO/B3oApQDz6rqE/Vdr7GPuXr63fw8Zi7cxKc/OYfk9lHOxq9uc1oRV2yGSK8LLxljfKg5PeYaaKWlpZSWlhIZGcn69eu58MILWb9+PWFhpz5C0GwecxWRUOBpYAKQCUwSkUwvRReo6mD3VfGMWCnwY1XNAEYCd9RyrM/dfGYqAC99kV+10VKBG2MC5NChQ4wePZpBgwZxzTXX8Ne//rVRwaEx/HmV4cAGVd0IICKzgSuAejvQVHUHsMN9XywieUBSQ449Xd3i23LJGV2Z/fVW7j4vndjIcGiXYanAjTEBER8fz5IlSwJybX+OQSQBWz0+F7jbaholIstF5B0ROelxBRFJBYYAi7xdRESmikiOiOTs3u2b1eCmjE2j+Hgpc3IKqjZm3g8l+51U4MYY0wr4M0B4y09Rc8BjKdBDVQcBTwFzq51AJAZ4DfihqnqdKaOqz6pqtqpmJyb6ZnxgYHI8w1MTeOHzTZSWuQn7LBW4MaaV8WeAKAC6e3xOBrZ7FlDVIlU95L6fD4SLSEcAEQnHCQ6zVPV1P9bTq8lj0yjYf5T3c3dVbbRU4MaYVsSfAWIxkC4iaSISAVwHzPMsICJdxE1SLiLD3frsdbc9B+Sp6uN+rGOtzs/oTI8OUcxcsLFqo6UCN8a0In4LEKpaCtwJvAfkAXNUdbWITBORaW6xicAqEVkOPAlcp85zt6OBG4FzRWSZ+7rYX3X1JjREuG10Gku3HGBpxcQ5SwVuTKtwzjnnnDTpbcaMGdx+++21HhMT4zy8sn37diZOnFjreet7FH/GjBkcOXKk8vPFF1/MgQMHGlp1n/LrRDlVna+qfVS1l6o+7G57RlWfcd//n6r2V9VBqjpSVb9wty9UVVHVgR6PwM73Z129mTg0mbiaE+dSvgPRabD6EQiiVOnGmCqTJk1i9uzZ1bbNnj2bSZMm1Xtst27dePXVVxt97ZoBYv78+cTHxzf6fKfDZlLXIbpNGNeP6ME7K3ewdZ/7HywkDDJ+bKnAjQliEydO5O233+b48eMA5Ofns337dgYPHsx5551HVlYWZ5xxBm+++eZJx+bn5zNgwAAAjh49ynXXXcfAgQO59tprOXr0aGW5H/zgB5Vpwh944AEAnnzySbZv3864ceMYN24cAKmpqezZsweAxx9/nAEDBjBgwIDKNOH5+flkZGTw/e9/n/79+3PhhRdWu87psGR99bj5zB7MXLCRl77I55eXunP1et4KK39tqcCNaQpLfgj7fbz+QfvBMHRGrbs7dOjA8OHDeffdd7niiiuYPXs21157LW3btuWNN94gLi6OPXv2MHLkSC6//PJa13v+y1/+QlRUFCtWrGDFihXVUnU//PDDJCQkUFZWxnnnnceKFSu4++67efzxx/n444/p2LFjtXMtWbKEF154gUWLFqGqjBgxgrPPPpv27duzfv16XnnlFf72t7/x3e9+l9dee40bbrjhtP9M1oKoR9d2bbl0YFdmL95K8bETzsawKOh7t6UCNyaIeXYzVXQvqSo///nPGThwIOeffz7btm1j165dtZ7js88+q/yiHjhwIAMHDqzcN2fOHLKyshgyZAirV6+uNwnfwoULueqqq4iOjiYmJoarr76aBQucXoy0tDQGDx4M1J1O/FRZC6IBJo/pydxl2/nX4q1MGdvT2djnDqcFYanAjfGvOn7p+9OVV17Jvffey9KlSzl69ChZWVm8+OKL7N69myVLlhAeHk5qaqrX9N6evLUuNm3axB//+EcWL15M+/btueWWW+o9T1158yrShIOTKtxXXUzWgmiAM5LbMSItgRc+z6+aOGepwI0JajExMZxzzjncdtttlYPTBw8epFOnToSHh/Pxxx+zefPmOs9x1llnMWvWLABWrVrFihVOj0NRURHR0dG0a9eOXbt28c4771QeExsbS3FxsddzzZ07lyNHjnD48GHeeOMNxo4d66vb9coCRANNHpPGtgNHeW+1R3PSUoEbE9QmTZrE8uXLK1d0+973vkdOTg7Z2dnMmjWLfv361Xn8D37wAw4dOsTAgQN59NFHGT58OOCsDDdkyBD69+/PbbfdVi1N+NSpU5kwYULlIHWFrKwsbrnlFoYPH86IESOYMmUKQ4YM8fEdV+fXdN9NzRfpvmtTVq6c99gntI+O4I3bPXK+WypwY3zO0n37R7NJ9x1sQkOE28ak8c2WAyzZvL9qh6UCN8YEKQsQp2Di0GTatQ3nuYUe6Tc8U4GfOBS4yhljjI9ZgDgFURFhXD8ihXdX7ayaOAeWCtwYPwim7u/moDF/TwsQp+jmUamEiPDC5/lVGytSga953FKBG+MDkZGR7N2714KEj6gqe/fuJTIy8pSOs3kQp6hLu0guG9SNfy3ewg8vSCcuMtzZkXk/fHKxkwq85y0BraMxLV1ycjIFBQX4ahEw4wTd5OTkUzrGAkQjTB6TxhvfbONfX2/l+2e5E+e6jof4gZD7KKTd5Dz+aoxplPDwcNLS0gJdjVbPvsUaYUBSO0b2rLHiXGUq8DxLBW6MCQoWIBppypiebD94jHdW7azamPJdiE61VODGmKDg1wAhIuNFZK2IbBCR6V72nyMiBz0WBfqVx77nRaRQRFb5s46NdW6/TqR1jGbmgo1VA2khYZBxn6UCN8YEBb8FCBEJBZ4GJgCZwCQRyfRSdIHHokAPeWx/ERjvr/qdrhB34tzygoPVJ871vBXaJDqJ/IwxpgXzZwtiOLBBVTeqagkwG7iioQer6mfAPn9VzheuyUoiPiqcmQs8VpyzVODGmCDhzwCRBGz1+FzgbqtplIgsF5F3RKS/H+vjc1ERYXxvRArv5+5ky16PiXN97oCwGCcVuDHGtFD+DBDelliqOXK7FOihqoOAp4C5p3wRkakikiMiOYF4ZvqmUamEhggvfOHRirBU4MaYIODPAFEAdPf4nAxs9yygqkWqesh9Px8IF5Hq6+zVQ1WfVdVsVc1OTGz6bKqd45yJc3MWb+Xg0RNVOywVuDGmhfNngFgMpItImohEANcB8zwLiEgXcZdbEpHhbn32+rFOfjF5TBqHS8r41+ItVRujkiH1Bvj2OThms0GNMS2P3wKEqpYCdwLvAXnAHFVdLSLTRGSaW2wisEpElgNPAtep+8yoiLwCfAn0FZECEZnsr7qerv7d2jGqZwde/DyfExUT58BNBX7UUoEbY1okWzDIRz7M28Xkl3J4ctIQLh/UrWrHZ1dB4adwxRYIjwlI3Ywxpja2YFATGNe3Ez1rTpwDSwVujGmxLED4SMXEuRUFB8nxnDhnqcCNMS2UBQgfuiYr2Z04t7H6jsz74UiBkwrcGGNaCAsQPtQ2IpQbRvTg/dxdbN57uGqHZypwLa/9BMYY04xYgPCxm0b1ICykxopzlgrcGNMCWYDwsU5xkVw+KIk5OVs5eMRj4pylAjfGtDAWIPxg8pg0jpSU8YrnxDlLBW6MaWEsQPhBZrc4Rvf2MnGu563QpqOlAjfGtAgWIPxkypie7Cw6xvyVO6o2hkVB33ssFbgxpkWwAOEnZ/dJpFdiNH+rOXEu/XYIi7ZU4MaYZs8ChJ+EhAiTx/Rk1bYivt7kse5RmwTo/T+WCtwY0+xZgPCjq7OSaB8VzsyFm6rvsFTgxpgWwAKEH0WGh3LDyB78N28X+Xs8Js5ZKnBjTAtgAcLPbhzVg/CQEF74vEYrwlKBG2OaOQsQftYpNpLLB3djTk5B9Ylz7TIg+UpY939w4lDgKmiMMbWwANEEJo9J4+iJMv759ZbqOywVuDGmGfNrgBCR8SKyVkQ2iMh0L/vPEZGDIrLMff2qoce2JBld4xjTuyMvfrGJklKPiXMdR0KnsywVuDGmWfJbgBCRUOBpYAKQCUwSkUwvRReo6mD39dApHttiTB6bxq6i49UnzgFkTrdU4MaYZsmfLYjhwAZV3aiqJcBs4IomOLZZOjs9kd6dYpi5sMbEOUsFboxppvwZIJKArR6fC9xtNY0SkeUi8o6I9D/FYxGRqSKSIyI5u3c330dGnYlzaazaVsQiz4lzlgrcGNNM+TNAiJdtNfNcLwV6qOog4Clg7ikc62xUfVZVs1U1OzExsdGVbQpXDUkiITqCmQtqPPJqqcCNMc2QPwNEAdDd43MysN2zgKoWqeoh9/18IFxEOjbk2JaoYuLch2t2sXG3x6OtlgrcGNMM+TNALAbSRSRNRCKA64B5ngVEpIuIiPt+uFufvQ05tqW6cWTFxLn86jssFbgxppnxW4BQ1VLgTuA9IA+Yo6qrRWSaiExzi00EVonIcuBJ4Dp1eD3WX3VtSomxbbhySDf+vWQrB454PNpqqcCNMc2MaBD1eWdnZ2tOTk6gq1GvtTuLuWjGZ/zkor7cMa531Y7j++DNFGeG9ZkvB66CxphWQ0SWqGq2t302kzoA+naJZWx6R176Ir/6xDlLBW6MaUYsQATIlLE9KSw+ztsraoy9WypwY0wzYQEiQM5K70h6pxhmLthUfeJcVDKkfs9SgRtjAs4CRICIOBPncncU8eXGvdV3ZvzUUoEbYwLOAkQAXTkkiQ7RETxXc+JcuwxIvsJSgRtjAsoCRABVTZwr5NvdNQJB5nRLBW6MCSgLEAF2w8geRISF8HzNdastFbgxJsAsQARYYmwbrhqcxGtLC9h/uEYgsFTgxpgAsgDRDEwem8axE+UnrzhnqcCNMQFkAaIZ6NM5lrP6JPLiF/kcLy2r2lEtFfjbgaugMaZVsgDRTEwZk8bu4uO8vbzGinMVqcBzLRW4MaZpWYBoJsamd6RP5xhmLqwxcS4kDPr9GPZ8CbsXBq6CxphWxwJEMyEiTBnTk7wdRXz5bY2Jc71uc1OBPxKYyhljWiULEM3I5YO70TEmgpk1H3kNi4I+d1sqcGNMk7IA0YxEhody48hUPlpTyIbCGhPn+twBYdGQ92hgKmeMaXX8GiBEZLyIrBWRDSIyvY5yw0SkTEQmemy7R0RWichqEfmhP+vZnNwwMsWZOPd5jVZEmwToNdVSgRtjmozfAoSIhAJPAxOATGCSiGTWUu73OKvHVWwbAHwfGA4MAi4VkXR/1bU56RDThmuyknhtSQH7ak6cy7jXUoEbY5qMP1sQw4ENqrpRVUuA2cAVXsrdBbwGFHpsywC+UtUj7vKjnwJX+bGuzcpto9M4XlrOrK82V99hqcCNMU3olAOEiISISFwDiiYBWz0+F7jbPM+VhPPF/0yNY1cBZ4lIBxGJAi4GutdSn6kikiMiObt3B8eXZnrnWM7pm8hLX26uPnEOLBW4MabJNChAiMg/RSRORKKBXGCtiPykvsO8bKs502sGcL+qVvsWVNU8nG6nD4B3geVAqbeLqOqzqpqtqtmJiYkNuJuWYfKYNPYcOs68ZTVWnLNU4MaYJtLQFkSmqhYBVwLzgRTgxnqOKaD6r/5koMa3HdnAbBHJByYCfxaRKwFU9TlVzVLVs4B9wPoG1jUojOndkb6dY3mu5sQ5sFTgxpgm0dAAES4i4TgB4k1VPcHJrYGaFgPpIpImIhHAdcA8zwKqmqaqqaqaCrwK3K6qcwFEpJP7bwpwNfBKA+saFESEyWPTWLOzmM831Jg4Z6nAjTFNoKEB4q9APhANfCYiPYCiug5wB5fvxHk6KQ+Yo6qrRWSaiExrwDVfE5Fc4C3gDlXd38C6Bo0rBnejY0wbZi7cePLOjPvdVOCtKm4aY5qQnNR90dADRcLcINBsZGdna05OTqCr4VNPfriexz9Yxwc/Oov0zrFVO1ThnUFQXgqXrHIefzXGmFMkIktUNdvbvoYOUt/jDlKLiDwnIkuBc31aS+PV90ak0MbbxDkRpxVhqcCNMX7S0J+dt7mD1BcCicCtgGWOawIdYtpwdVYyry3dxt5Dx6vv7HGtpQI3xvhNQwNExSOrFwMvqOpyvD/Gavxg8phUSkrLmbWoxopzlgrcGONHDQ0QS0TkfZwA8Z6IxAK2BmYT6d0plnF9E/n7l/kcO1Fj4pylAjfG+ElDA8RkYDowTFWPABE43UymiUwZ25M9h0qYt7zGVBJLBW6M8ZMGBQhVLceZ6PZLEfkjcKaq2rdREzqzVwf6dYnluQVeJs5ZKnBjjB809CmmR4B7cNJs5AJ3i8jv/FkxU52IMGVsT9buKmbhhj3Vd1oqcGOMHzS0i+li4AJVfV5VnwfGA5f4r1rGm8sGdSUxtg0zF2w6eaelAjfG+NipzK6K93jfztcVMfVrExbKzaN68Om63azbVVx9p6UCN8b4WEMDxO+Ab0TkRRF5CVgC/D//VcvU5voRPYgMD+H5mutWg0cq8P9r+ooZY4JOQwepXwFGAq+7r1GqOtufFTPeJURHcE1WMq9/s409NSfOVaYCf8pSgRtjTludAUJEsipeQFecFN5bgW7uNhMAt41Jo6S0nJdrrjgHTvqNkv3w7cymr5gxJqiE1bO/rhFPxfIxBUSvxBjO7deJf3y5mWln9yIyPLRqZ+IoNxX4Y5B+O4RGBK6ixpgWrc4WhKqOq+NlwSGApoxJY+/hEt5ctu3knZYK3BjjA/W1IAAQkau9bD4IrFTVQt9WyTTEqF4dyOgax8wFm/hudndEPFJjdZsA8WdA7u8h7UZLBW6MaZRTSbUxE/ie+/obcC/wuYjUt/So8QMRYcqYNNYXHuKz9Xtq7rRU4MaY09bQAFEOZKjqNap6DZAJHAdGAPfXdpCIjBeRtSKyQUSm11FumIiUichEj20/EpHVIrJKRF4RkcgG1rXVuGxQNzrFtmHmAi8rzlkqcGPMaWpogEhV1V0enwuBPqq6Dzjh7QARCQWeBibgBJRJIpJZS7nf4yxNWrEtCbgbyFbVAUAozprWxkNEWAg3n5nKgvV7WLuzxsQ5SwVujDlNDQ0QC0TkbRG5WURuBubhrE0dDRyo5ZjhwAZV3aiqJcBs4Aov5e4CXsMJOp7CgLYiEgZEAdtrHmjg+uEpRIaH8Jy3dastFbgx5jQ0NEDcAbwADAaGAC8Bd6jqYVUdV8sxSThzJioUuNsquS2Fq4BnPLer6jbgj8AWYAdwUFXf93YREZkqIjkikrN7d+tLMdE+OoKJQ5OZ+812dhfXmDhnqcCNMaehoTOpFVgIfAT8F/hMT8o5fRJvK87VPGYGcL+qVlsFR0Ta47Q20oBuQLSI3FBL3Z5V1WxVzU5MTKz/ZoLQbaPTKCkr5x/eJs5ZKnBjTCM1NN33d4GvgYnAd4FFngPKtSgAunt8TubkbqJsYLaI5Lvn/rOIXAmcD2xS1d2qegInvceZDalra9QzMYbzMzox66vNJ684Z6nAjTGN1NAupl/grCZ3s6rehDO+8L/1HLMYSBeRNBGJwBlknudZQFXTVDVVVVOBV4HbVXUuTtfSSBGJEucB//OAvAbfVSs0eUxP9h4uYe43XibO9fsRILDm8SavlzGm5WpogAipMSFub33HqmopcCfO00l5wBxVXS0i00RkWj3HLsIJGEuBle61nm1gXVulkT0T6N8tjpkLvaw4F90d0m5w8jNZKnBjTAM1NEC8KyLvicgtInIL8B9gfn0Hqep8Ve2jqr1U9WF32zOq+oyXsreo6qsenx9Q1X6qOkBVb1TV4zWPMVWcFefS2FB4iE/XeQkClgrcGHOKGjpI/ROcX/ADgUHAs6pa6wQ5ExiXnNGNznFteM7bWhGWCtwYc4oanKRHVV9T1XtV9Ueq+oY/K2Uax3Pi3HNMK9IAAB0zSURBVJqdRScXsFTgxphTUN96EMUiUuTlVSwiXr6BTKBdPzyFtuGhPOdt3WrPVOBlJU1fOWNMi1LfQHOsqsZ5ecWqalxTVdI0XHyUM3HuzWXbKSw+dnIBSwVujGkgywMdhG4dncqJ8nJe/tLLxDnPVOBa3vSVM8a0GBYgglDPxBjO69eZf3ibOOeZCnzZdBuwNsbUygJEkJoyNo39R07w+lIvE+d6XAs9roe8P8BbvWH9X6Dca1JeY0wrZgEiSI1IS2BAUhzPLdxIeXmNiXMhYTB6Flz4FcT1hcW3w38GwNbXbe0IY0wlCxBByllxriff7j7sfeIcQMcRcN4ncNY8J2gsuAY+GA2Ftn6EMcYCRFC7+IyudImLZKa3tSIqiEDyZTBhOYyYCYc3w3/HwqdXwEFLf2VMa2YBIohVTJz7fMNecrfXM20lJAx6TYbL1sOgh6HwE5g/ABZNhSO2VpMxrZEFiCBXOXHOW/oNb8KioP/P4bJvoc9dsOlFZyB7+S/hhM2NNKY1sQAR5NpFhfPd7GTmLd9GYZGXiXO1iewIQ2fApWucHE6rH4Z5vWDtUzYL25hWwgJEK3Dr6DRKy5W/e5s4V5+YnjD6FbhosTPBbsnd8J9M2DzHnngyJshZgGgFUjtGc0FGZ2Yt2szRkrL6D/CmQzac+yGcM9/phvr8WnhvBOz6xKd1NcY0H34NECIyXkTWisgGEZleR7lhIlJWsYypiPQVkWUeryIR+aE/6xrspozt6Uyc+6ag8ScRcVJ1jP8GRr4Ix3bAh+Pgk0vgwCqf1dUY0zz4LUCISCjwNDAByAQmiUhmLeV+j7PyHACqulZVB6vqYGAocASwFOOnYVhqewYmt+O5hZtOnjh3qkJCoefNcOk6GPx72P05vDMIvrrNSQRojAkK/mxBDAc2qOpGVS0BZgNXeCl3F/AaUOhlHzjrUX+rqo3oQDcVRITJY9LYuPswn6yr7U99isLaQuZP4fKN0PdHkD8L3kp3cjyVHPDNNYwxAePPAJEEbPX4XOBuqyQiScBVwElLkHq4Dqg1N7WITBWRHBHJ2b3b1luuy8VndKVru0hmelsr4nS0SYCsP8Kla6H7RMh91Hniac2foMxWijWmpfJngBAv22r2bcwA7ldVryOnIhIBXA78u7aLqOqzqpqtqtmJiYmNrmxrEB7qTJz74tu9rN5+0PcXiEmFM/8B45dAwlBYei+83Q82zbLU4sa0QP4MEAVAd4/PyUDNKbnZwGwRyQcmAn8WkSs99k8AlqrqLj/Ws1WZNCyFqIhTmDjXGAlD4Nz3Ydz7EBEPX94A72bDzv/675rGGJ/zZ4BYDKSLSJrbErgOmOdZQFXTVDVVVVOBV4HbVXWuR5FJ1NG9ZE6dM3GuO28t386Og0f9e7GuFzitiVEvQ8k++OgC+Ogi2L/Mv9c1xviE3wKEqpYCd+I8nZQHzFHV1SIyTUSm1Xe8iEQBFwCv+6uOrdWto1MRhMue+pw5OVtP/6mmukgIpH3PmZE95DHYtxjeyYIvbnISAxpjmi3RIJoNm52drTk5OYGuRouwouAAD85bzdItBxiY3I4HLuvP0B7t/X/hkgOQ+wisfcKZid3nTif3U5sE/1/bGHMSEVmiqtle91mAaL1UlbnLtvHIO2vYVXScq4Ykcf/4fnRpF+n/ix/eCit/BRtfgvB2TpDoexeENsG1jTGVLECYOh0+XsqfP9nA3z7bRFiocMe43kwek0ZkeKj/L35gpTNvYvt8iOoOA38Lqd9zJuMZY/zOAoRpkC17j/Db/+Tyfu4uUhKi+MUlGVyY2RkRb08s+9iuj+Gbn8K+HIgf6MzQ7nqRk97DGOM3dQUIS9ZnKqV0iOLZm7J5efIIIsND+J9/LOGG5xaxblex/y/eeRxctAhGz4bSQ/DJBOepp31L/H9tY4xX1oIwXpWWlTNr0RYe/2Adh46XcsOIFH50QR/ioyL8f/GyEtjwDKz6DRzfAz0mOavcxaT5/9rGtDLWxWQabd/hEh7/YC3/XLSFdm3DuffCvlw/PIXQkCbo+ik5CHmPOik7tBTS74D+v3AWMzLG+IQFCHPacrcX8eu3VrNo0z76dYnlgcv6M6pXh6a5+JFtsPJB2Pg8hMVA5nToe4+zLoUx5rRYgDA+oaq8s2onD/8nj20HjnLxGV34+cUZJLdvoi/qg7nOE0/b3oK2STDwIUi72Z54MuY0WIAwPnXsRBnPfraRP3+yAVX4n7N6Mu2cXkRFhDVNBQoXwDc/gb2LoF1/GPwIdLvEnngyphHsKSbjU5Hhodx9Xjof/fgcLurfhSc/2sB5j33KvOXbaZIfHJ3GwoVfwph/Q3kJfHoZfHgO7Pna/9c2phWxAGEarVt8W56cNIR/TxtFQnQEd7/yDd/965es2uaHVOI1iUDKRLhkNWQ/DUVr4P0RsPC7ULzB/9c3phWwLibjE2XlypycrfzhvbXsP1LCdcO6c9+FfekQ06ZpKnCiGPIegzV/dBYpSp8GA/4XIjs1zfWNaaFsDMI0mYNHT/Dkh+t56Yt82kaEcs956dx8ZirhoU3UWD26E1b+Gr79G4S2hYyfQsa9EBbdNNc3poWxAGGa3IbCYh56O4/P1u2mV2I0v7qsP2f3acIV/4rWwrKfQcEbENkFBv4aet4GIU00kG5MC2EBwgSEqvLRmkJ+83Yu+XuPcH5GJ35xSSZpHZvw1/zuL2DZT2H35xDbB5IuhYRs6DAMYnrZk0+m1bMAYQLqeGkZL3yez1MfrqekrJzbxqRx57jexEaGN00FVGHbPMj7g5PbqeyYsz083lk7u8MwN2hkQ1SKBQ3TqgQsQIjIeOAJIBSYqaqP1FJuGPAVcK2qvupuiwdmAgMABW5T1S/rup4FiOatsPgYj767lleXFJAY24afXtSXa7KSCWmKtB0Vyk84E+72LnYyx+7LgQMrnO0AbRKrgkXCMOfftl2brn7GNLGABAgRCQXW4SwbWoCzRvUkVc31Uu4D4BjwvEeAeAlYoKoz3TWto1T1QF3XtADRMizfeoAH31rNN1sOMKh7PA9elsmQlCZYza42ZcedILEvB/bmOMuiHlwNWu7sb9utqlsqIdt5WT4oEyQCFSBGAQ+q6kXu558BqOrvapT7IXACGAa8raqvikgcsBzoqadQQQsQLUd5edVqdoXFx7l6SBL3T+hH57hmsqJc6RHYv6x6S6NoLU5jFohOrd7SSMiCiPhA1tiYRqkrQPjzkY4kYKvH5wJgRI2KJQFXAefiBIgKPYHdwAsiMghYAtyjqodrXkREpgJTAVJSUnxZf+NHISHC1VnJXNS/C09/vIGZCzbx7uqd3Hlub24b3USr2dUlLAoSz3ReFU4Uwb6l1VsaW1+t2h+bXr2l0X4IhMc0fd2N8RF/tiC+A1ykqlPczzcCw1X1Lo8y/wYeU9WvRORFqloQ2ThjEqNVdZGIPAEUqer/1nVNa0G0XJv3Hua3/8njA3c1u19eksEFTbWa3ek4vs8Z+N632A0aOXDE/V0kIRCXUdUt1WEYtB9k626bZqXZdjGJyCag4hugI3AEpzXwFfCVqqa65cYC01X1krquaQGi5VuwfjcPvZXL+sJDjE3vyK8uzSS9c2ygq3Vqju6q6paqaGkc2+XskzCIH1A1AJ6QDe0GQGgTLMRkjBeBChBhOIPU5wHbcAapr1fV1bWUfxG3BeF+XgBMUdW1IvIgEK2qP6nrmhYggsOJsnJe/mozf/pgHYdLyrhxZA9+dH4f2kU10WOxvqYKR7dVBYuKlkbJPmd/SBunZVHZ0sh2Wh42qc80gUA+5noxMAPnMdfnVfVhEZkGoKrP1Cj7ItUDxGCcx1wjgI3Araq6v67rWYAILvsOl/DY+2t55WtnNbsfX9iXSU21mp2/qcLhTVXBoqK1Uequ/x0aBQlD3KDhtjZi051uK2N8yCbKmRbNczW7jK5xPHBZJiN7NtFqdk1Jy6F4fVVLY1+OMyhedtTZHx7nTOzzbGlEp9nEPnNaLECYFk9Vmb9yJ/9vvrOa3SVndOVnF/drutXsAqW8FIryqloaexfDgeXOOhgAEQleJvYlWdAwDWYBwgSNoyXOanZ/+dRZzW7a2b2YdnYv2ka0omVHy0rg4KqqgLEvBw6sBC1z9kd2rgoW7YdAVLKTsDAyEUJa6DiO8RsLECbobDtwlN/Nz+PtFTvo1i6Sn12cwaUDuzb/x2L9pfSo07KoHNNYDAfzqJzYB4BAm45OAGnbxQkaFf96vm/bxWmZtNa/ZStjAcIEra837ePBeavJ3VHE8NQEfnVZJgOS2gW6Ws3DiUNOypBjO511Mrz9e2xnVfJCTyHhTiCJ7Fx3IInsYpMBWzgLECaonbyaXQr3Xdin6Vaza8lUnRnix3Y6czXqCiTHCqu6sTyFRbvBo3PdgSSys833aIYsQJhW4eDREzzx3/X8/UtnNbsfnt+Hm0b1aLrV7IJdeRmU7HWDxq7aA8nRnVVzPGqKSKg/kLTtAhEdIKQVjSsFkAUI06psKCzm12/lsmD9Hnp3iuFXl2ZyVlOuZmecDLnHCj2Cxy7vgeToDig7cvLxEuqsJ96gLq44Gy85DRYgTKujqnyYV8hv/pPL5r1HOD+jM7+8JIPUplzNzjTMiUNeWiO1BBQtPfn40MjqgaRNJ4ho777inYWhKt5HuO/D4627y2UBwrRanqvZHS8tZ2ByO4b2aM/QHglk9YinU6wlzmsxtBxK9nvp1tp18lhJyf6quSK1CW3rPYjUFlAiPMu0C5pZ7RYgTKtXWHSMF77IZ/GmfawoOEhJmbMYUEpClBswnFefzrHBkcqjtVN1ns46ccAJFiUH3Nd+d5uX9yUHqsqfOFi1YJRX4nRteQsgJwUTLwEnNKrZdItZgDDGw/HSMlZtK2Lp5v0s2byfnM372XPoOAAxbcIYkhJfGTAGd49vurWzTfOh5VB6qO7g4hlQar4vPVT3+SWsgQGllv0+7B6zAGFMHVSVrfuOsmTLPpZs3s+SzQdYs7MIVedHXt/OsQzt0Z7s1PYMTUmge0Lb1jshzzRMealH0GhAcCk5ACf2V+2vt3ssqnqLpG0yjJndqKoGakU5Y1oEESGlQxQpHaK4akgyAMXHTrBs6wE3YOznzWXbmbVoCwAdY9owtEd85VjGgKQ42oTZI5nGQ0iYs255Y9cuLz3a8NZKyYGqLMA+ZgHCGC9iI8MZm57I2HTn8diycmXdrmKWbN7vdE1t2c97q51FgCJCQzijcvC7PVkp7UmMtUl65jSEtXVebbsGtBrWxWRMIxUWH2Pp5gMs3eK0MlZ6DH736BDF0JT2ZLldU+mdbPDbNE+BXDBoPPAEzoJBM1X1kVrKDcNZZvRajwWD8oFioAwore0GPFmAMIHkDH4frOyWWrJ5P3sOOX3JsW3CGGyD36YZCsgYhIiEAk8DFwAFwGIRmaequV7K/R54z8tpxqnqHn/V0RhfahMWytAeCQztkQBUDX7nbN5XGTCe+HA9qhAi0LdLXOVYRnaPBJLb2+C3aV78OQYxHNigqhsBRGQ2cAWQW6PcXcBrwDA/1sWYJuc5+H11ljP4XXTsBMu2OIPfS7fsZ+4323n5K2fwOzG2DUNT3HGMHu1t8NsEnD8DRBKw1eNzATDCs4CIJAFXAedycoBQ4H0RUeCvqvqst4uIyFRgKkBKSopvam6Mn8RFhnNWn8TK3FAVg985FYPfm/fz7uqdAESEhTAwqV1lwBjaoz0dLUOtaUL+DBDe2so1BzxmAPerapmXpvVoVd0uIp2AD0Rkjap+dtIJncDxLDhjED6otzFNJjREyOgaR0bXOG4c2QOoGvxe4nZNvfB5Pn/9bCMAqR2iKoPF0B42+G38y58BogDo7vE5Gdheo0w2MNsNDh2Bi0WkVFXnqup2AFUtFJE3cLqsTgoQxgSbTrGRjB/QhfEDugBw7EQZq7cfJCffaWF8tm43ry/dBjiD30N6tK/smhqcEk9MG3t63fiGP/+XtBhIF5E0YBtwHXC9ZwFVTat4LyIvAm+r6lwRiQZCVLXYfX8h8JAf62pMsxUZfvLg95Z9RyrThCzdvJ8ZH66rHPzu1yWu2pyM5PZtCbFWhmkEvwUIVS0VkTtxnk4KBZ5X1dUiMs3d/0wdh3cG3nBbFmHAP1X1XX/V1ZiWRETo0SGaHh2iTxr8rggYry8t4B9fbQYgKiKU3p1iSO8US5/OMaR3dt4nxVvgMHWziXLGBKGycmXtzmKWbT3Aul3FbCg8xLpdxRQWH68sY4HDgOViMqbVCQ0RMrvFkdktrtr2g0dOsL6wmHW7DlUGjgXrd/Pa0oLKMlERoaR3iqG3Gzj6dI6ld6cYCxytkAUIY1qRdlHhZKcmkJ2aUG27BQ7jjQUIY0ytgePAkRLWFx5ivRs41hcW1xo40jvHkt7JCRzpnWPo1s4CR0tnAcIYU6v4qAiGpSYwrAGB47N1u3l1iQWOYGIBwhhzyixwtA4WIIwxPlNf4Fi3q5j1uw6xvrCYT2sEjuiKp6o6u09VdXICR1K8JTEMFAsQxhi/s8DRMlmAMMYEzGkHjspuqqouKwscvmMBwhjT7JxO4IhpE0bfLrFkdI2lXxcnEWK/LrFEW46qU2Z/MWNMi9GQwLFuZzF5O4p5c9l2Xj7mrLUhAj0SoioDRkbXWDK6xtkiTfWwAGGMafG8BQ5VZduBo+TtKCZvRxF5O4pYs7OY93J3UpFhKLZNGP3cYFHx6ts5lrYRtlATWIAwxgQpESG5fRTJ7aO4ILNz5fbDx0tZu8sjaOwo5vWl2zh0fLN7HKR1iK5safTrEkdGtzi6tYtsda0NCxDGmFYluk0YWSlOKvQK5eVKwf6j5O4oYs1OJ3Cs3HaQ/6zcUVmmXdtw+nWJrdZF1adzLJHhwdvasABhjGn1QkKq1g+vWKgJoPjYCdbtKibXo5tqTs5WjpSUOccJpHWMruyeyuwaR7+usXSJC47WhgUIY4ypRWxkeLXFmsBpbWzZd6QyYOTucNKqv72iqrURHxVORsUTVF1jyewaR+9OMS2uteHXACEi44EncBYMmqmqj9RSbhjwFXCtqr7qsT0UyAG2qeql/qyrMcY0REiIkNoxmtSO0Uw4o2vl9qJjJ1jjtjTW7HQCxz+/3syxE+WAk4K9V2J0tSepMrvGkRjbptm2NvwWINwv96eBC3DWp14sIvNUNddLud/jrDxX0z1AHhDnZZ8xxjQbcZHhDE9LYHhaVWujrFzJ33u4cjA8b0cROfn7mLd8e2WZDtERzpNUXaqepOrdKYaIsJBA3EY1/mxBDAc2qOpGABGZDVwB5NYodxfwGjDMc6OIJAOXAA8D9/qxnsYY4xdOqyGGXokxXDqwavuBIyWs2Vk1rpG3o5i/f7WZklKntREWIvTuFFP9SSq3tdGU/BkgkoCtHp8LgBGeBUQkCbgKOJcaAQKYAfwUiPVjHY0xpsnFR0UwsmcHRvbsULmttKyc/L2Hqw2If/ntXt74ZltlmY4xbSqfoKr4t1diDOGh/mlt+DNAeOtUq7kA9gzgflUt8+yDE5FLgUJVXSIi59R5EZGpwFSAlJSU06qwMcYESlhoCL07xdK7UyyXD+pWuX3f4RLW7ChyH8F1gseLn+dTUua0NsJDhcxu7Zh7+5k+H8sQ1Zrf2T46scgo4EFVvcj9/DMAVf2dR5lNVAWSjsARnC/7EcCNQCkQiTMG8bqq3lDPNXcDmxtZ5Y7AnkYe21LZPQe/1na/YPd8qnqoaqK3Hf4MEGHAOuA8YBuwGLheVVfXUv5F4G3Pp5jc7ecA9/n7KSYRyVHVbH9eo7mxew5+re1+we7Zl/zWxaSqpSJyJ87TSaHA86q6WkSmufuf8de1jTHGnD6/zoNQ1fnA/BrbvAYGVb2llu2fAJ/4uGrGGGPqEfgHbZuPZwNdgQCwew5+re1+we7ZZ/w2BmGMMaZlsxaEMcYYryxAGGOM8arVBwgRGS8ia0Vkg4hMD3R9/EFEnheRQhFZ5bEtQUQ+EJH17r/t6zpHSyMi3UXkYxHJE5HVInKPuz1o71tEIkXkaxFZ7t7zr93tQXvP4ORzE5FvRORt93NQ3y+AiOSLyEoRWSYiOe42n993qw4QHgkFJwCZwCQRyQxsrfziRWB8jW3TgQ9VNR340P0cTEqBH6tqBjASuMP9bxvM930cOFdVBwGDgfEiMpLgvmeoSupZIdjvt8I4VR3sMf/B5/fdqgMEHgkFVbUEqEgoGFRU9TNgX43NVwAvue9fAq5s0kr5maruUNWl7vtinC+QJIL4vtVxyP0Y7r6UIL5nj6SeMz02B+391sPn993aA4S3hIJJAapLU+usqjvA+TIFOgW4Pn4jIqnAEGARQX7fbnfLMqAQ+EBVg/2eK5J6lntsC+b7raDA+yKyxM1HB36479a+olxDEgqaFkxEYnDSyf9QVYua68IsvqKqZcBgEYkH3hCRAYGuk7+cSlLPIDRaVbeLSCfgAxFZ44+LtPYWRAHQ3eNzMrC9lrLBZpeIdAVw/y0McH18TkTCcYLDLFV93d0c9PcNoKoHcDIQjCd473k0cLmI5ON0D58rIi8TvPdbSVW3u/8WAm/gdJf7/L5be4BYDKSLSJqIRADXAfMCXKemMg+42X1/M/BmAOvic+I0FZ4D8lT1cY9dQXvfIpLothwQkbbA+cAagvSeVfVnqpqsqqk4/9/9yM34HJT3W0FEokUktuI9cCGwCj/cd6ufSS0iF+P0Y1YkFHw4wFXyORF5BTgHJyXwLuABYC4wB0gBtgDfUdWaA9ktloiMARYAK6nqn/45zjhEUN63iAzEGZwMxfnxN0dVHxKRDgTpPVfwzPoc7PcrIj1xWg3gDBP8U1Uf9sd9t/oAYYwxxrvW3sVkjDGmFhYgjDHGeGUBwhhjjFcWIIwxxnhlAcIYY4xXFiCMMcZ4ZQHCmFqIyCcikl1/SZ9d7w9umu4/1LL/yiDNNmyaqdaei8kYvxCRMFUtPcXD/gdIVNXjtey/EngbyPXR9Yypk7UgTIsnIqnuwkB/c3+Bvy8ibT1bACLS0c3Zg4jcIiJzReQtEdkkIneKyL3uojNfiUiCx+lvEJEvRGSViAx3j492F2Fa7B5zhcd5/y0ibwHv11JXcVsKq9wFX651t88DooFFFdtqHHcmcDnwB3eRmF7u/f0/EfkUuMdNtfGaW6/FIjK6nvr2F2eBoWUiskJE0n3x38MED2tBmGCRDkxS1e+LyBzgmnrKD8BJAR4JbADuV9UhIvIn4Cac9CsA0ap6poicBTzvHvcLnLw/t7m5j74Wkf+65UcBA+tIcXA1zmI+g3BSnywWkc9U9XIROaSqg70dpKpfuEHkbVV9FcDNTBuvqme7n/8J/ElVF4pICvAekFFHfacBT6jqLDcXWWg9fzPTyliAMMFik6ouc98vAVLrKf+xu5BQsYgcBN5yt68EBnqUewWcRZdEJM79gr0QJ4vofW6ZSJz8N+CswVBX/psxwCtuWu5d7q//YTQ+SeS/PN6fD2RKVUrzODepW231/RL4hTiL7ryuqusbWQcTpCxAmGDh2W9fBrTFWXa0ohs1so7y5R6fy6n+/4uaycoUZx2Ra1R1recOERkBHK6nnr5ekMLzeiHAKFU9WqNeXusL5InIIpwV2d4TkSmq+pGP62daMBuDMMEsHxjqvp/YyHNUjBGMAQ6q6kGcrpu73C9eRGTIKZzvM+BacVZ+SwTOAr5u4LHFQGwd+98H7qz4ICIV3VVe6+tmBd2oqk/itGAGYowHCxAmmP0R+IGIfIHT398Y+93jnwEmu9t+g7Pe8woRWeV+bqg3gBXAcuAj4KequrOBx84GfuIONPfysv9uINsdcM7FGWOoq77XAqvEWaK0H/D3U7gP0wpYum9jjDFeWQvCGGOMVzZIbYwfiMgZwD9qbD6uqiMacOwvgO/U2PzvYFzt0DRv1sVkjDHGK+tiMsYY45UFCGOMMV5ZgDDGGOOVBQhjjDFe/X9gpFF0zwHJSQAAAABJRU5ErkJggg==\n","text/plain":"<Figure size 432x288 with 1 Axes>"},"metadata":{"needs_background":"light"},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"AUC on Validation Data: 0.732\n"},{"data":{"text/html":"<font size=\"+2\">See your metrics plot <a href=/mlflow/#/metric/training_auc?runs=[\"a92e93f393df\"]&experiment=1&plot_metric_keys=[\"training_logloss\",\"validation_logloss\",\"training_rmse\",\"validation_rmse\"]>here</a></font>","text/plain":"<IPython.core.display.HTML object>"},"execution_count":17,"metadata":{},"output_type":"execute_result"}]},{"metadata":{"trusted":false},"cell_type":"code","source":"# Print and Log Confusion Matrix\nprint(gbm_baseline.confusion_matrix(valid = True))\nmlflow.lm('fpr', gbm_baseline.fpr(valid=True)[0][0])\nmlflow.lm('tpr', gbm_baseline.tpr(valid=True)[0][0])\nmlflow.lm('fnr', gbm_baseline.fnr(valid=True)[0][0])\nmlflow.lm('tnr', gbm_baseline.fnr(valid=True)[0][0])\nmlflow.lm('F0point5', gbm_baseline.F0point5(valid=True)[0][1])\nmlflow.lm('F1', gbm_baseline.F1(valid=True)[0][1])\nmlflow.lm('F2', gbm_baseline.F2(valid=True)[0][1])\nmlflow.lm('auc', gbm_baseline.auc(valid = True))\nmlflow.lp('threshold', gbm_baseline.F1(valid=True)[0][0]) # First element is the threshold","execution_count":18,"outputs":[{"name":"stdout","output_type":"stream","text":"\nConfusion Matrix (Act/Pred) for max f1 @ threshold = 0.45593713188449025: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>0</th>\n <th>1</th>\n <th>Error</th>\n <th>Rate</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>677.0</td>\n <td>2672.0</td>\n <td>0.7979</td>\n <td>(2672.0/3349.0)</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>203.0</td>\n <td>11587.0</td>\n <td>0.0172</td>\n <td>(203.0/11790.0)</td>\n </tr>\n <tr>\n <th>2</th>\n <td>Total</td>\n <td>880.0</td>\n <td>14259.0</td>\n <td>0.1899</td>\n <td>(2875.0/15139.0)</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" 0 1 Error Rate\n0 0 677.0 2672.0 0.7979 (2672.0/3349.0)\n1 1 203.0 11587.0 0.0172 (203.0/11790.0)\n2 Total 880.0 14259.0 0.1899 (2875.0/15139.0)"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\n"}]},{"metadata":{"trusted":false},"cell_type":"code","source":"# Plot and Log Variable Importance\ngbm_baseline.varimp_plot()\nfor var in gbm_baseline.varimp():\n mlflow.lm(f'varimp_{var[0]}',var[-1])","execution_count":19,"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAA54AAAJTCAYAAABgjsk5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdebgkVX038O8PBhFEQQWX4ILiriAoboCKilswrqivO2pUNC4xJgouiWYTY3yNK8QVfDXuxqi44AZCUAEVGcW44WhQ3BVFEWQ47x9VF9qm7zJwz9y54+fzPP307apTp37VXQP3e8+p6mqtBQAAAHrZYqULAAAAYPMmeAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJwLKrqhOq6oJl6OfMqvrWBrS/QVW1qnrDZd03ALB8BE+AzUBV/ccYuJ68hLYfH9vef2PUtrkZQ3Wrqn1XupbeNjT4/zGoqv3Hz/8TC7SZ+wPIt6aWX6uqnlFVH62qdVV1XlX9rKqOWezfY1VtVVV/Pv77/XFVnT8+H1NVj6uqNZfhmO5WVW+pqm9W1a/Hvn9YVZ+oqudU1c4ztpn7dzD5uKCqflRVH6qqe87YZs1E2/VVtcsCNR0/0faRl/bYgE3Hpf6PFACblNcleViSJyQ5fL5G4y96d0tyVpIPdazn4Um26dg/rEZ/meRZSc5I8qkkP0qyS5IHJLl7Vb20tfbs6Y2q6jpJPpDklkl+mOHf7g+TXCPJnya5e5KnVNV9W2s/WGoxVbVDkqOS3DfJ+Uk+M/b92yQ7JbldksOSvKiqbttaO21GN29O8r3x522S3DTJvZMcUFWPb629acY2F2T4HfRxSf52Rl03SbLvRDtgM+AfM8BmoLV2bFV9I8meVXWr1toX52n6+CSV5M2ttcs8FXaBer63eCv4o/O5JHdqrR0/ubCqbpHkxCR/U1Vva619eWLddkk+miHQvSnJU1tr506sv0KSI5I8MsmHq+oOk+vnM46Qvi/JXTKE4Me01s6c0e4WSf4+yZXm6epNrbUTprZ5aJJ3JHnuWPO07yf5RZLHVdWLWmvrp9Y/YXz+UBIzM2AzYaotwObj9ePzE2atrKotkzw2SUvyhonlO1fV31XVieP0uvOr6vtV9bZx5GG6n4uuo6yqG1fVu6vqJ1V14dz001nXeFbV1lX1tKr6SFV9d5xm+PNx6uAlpuVNbbtDVb22qn5QVb+rqq9W1V9UVS31zamqK1TVc6vqy1X1m6o6Zzzmhy61j0X6P7OqvlVVV6qqV4yvz62qL1XVfcc2a6rqBeOUxt+N7S8xPXpiOufzq2qfqvpkVf1qfHykqm41Tw07VNVLquobY/8/r2Fa510X2cftq+rDY/tWVY+sqpZk5yS7Tk2nnDx3HjieJ9+ceE9PqaqnVtUlfseoqreOfVy7qp5SVV8Z6/xhVR1RVTPDzdj+VRPv28+q6qSqet48bV9bVWfUxVNZ/6uqbr3Q57cxtNbeMx06x+VfSfKe8eV+U6v/OkPoPD7Jn0+Hytbab5IclOTzGUZEn77Ech6TIXT+T5I/mxU652prrT0wQ2heqmPG550WaPP6DOfXvScXVtXlkjw6w+jr1zdgn8AmTvAE2HwclWG63MOratsZ6++d4Re9T7TWvjOx/C5Jnp3k50nem+TfkpyU5CFJThpHPGa50djuWknemuEXyV8vUN9OY9/bJfl4kv+bYfrgrZN8pKoOmme7rTOMyOyf5D/G/Vw1yavH/hZVVVdO8t9J/inJ7zOMwhyV5OpJ3lFVL1xKP0uwdZJPJLlnkvdneF9umOR9VbVfhvf3iUk+neSNGUaRXltVD5qnv73HtudmON6PJblHkhOqau+pY7xKks9m+Cx/keG9+c8k+yT5RFX9+Tz72DfDL/mXG2t6S5JvJnlRhs/zF+PPc48PTGz7L0n2yBBKXpXk/43H9Kqxr/m8LMNn8aUkr8kwbfRJGd6fP1BVt0vy5SRPTXJmklckeXuSczI1TbOq9kpyapKDMwSqVyb5YIYwd2JV3WOq/dw1h91G/zfA78fn6Vrm/pD0D621NmvDccTwn8eXT1zi/ubOh39prf12scYbOENi//H5lAXavC3DlN7p8/IBSXbMxX9IAzYXrTUPDw8Pj83kkeSdGUY0D5qx7r/GdQdOLb96ku1mtN8zyW+SfHBq+Q3GflqSv5+njhOSXDC17PJJdp7RdockX0vykyRbT607c9zPcUkuN7F8xyTfGdftPaO2N0z189Zx+V9NLd8mQwi+MMluS3yPTxj72neeWt8/eRwZgn3LEOw/l2T7iXU3zBA4Tp7qa/+J9/jgqXUPGpf/T5KaWP7Gcflrp9rfJEOA/F2Sa8+zj8fPc6xnJvnWAu/FrjOWbZEhVLQkt57nc/hOkmtNLN8qw1TTluRWE8u3znD9YEvykBn7mu7jjAwhffqzuVaG65rPnDqP1ox9XzDfMc7Y59z7dkaSF87zeOXYZt73bsa/gZ8kWZ/khhPLrzf2c36m/m3M6GO7cfuW5BqLtL3ceN61JNdd6rHP8+/gTRPH/ZLx/D8/ydokN5naZu79Xje+PnKs45oTbT6R4d/K5TNcX9qSPPLS1Ojh4bFpPVa8AA8PDw+P5XtkuHFQS3LC1PJrjr/g/TDJVhvQ34czjEpsObFsLtx9f/KX+KntLhE8F9nPszMVIsflc2HuDjO2+fNx3etn1PaGiWVXG38h/+w8+771uM0/L7HWxYLndWdsMxee7jRj3fFJzkuyxcSyuXDztUyEy6ltWpJ9xtdbZwhcZyfZYUb7F4/tnztjHycvcKwLBs8Ftrvt9P7G5XPB86AZ2zwhU0E7yUPHZe9dwj7nAvmL51n/rHH9PaaW3yTJjTfg2CYD+2KPRd+7DNdcv29s/4qpdXuPy89cYm0/zVR4n6fdn0zUuGbG+rvmkmH6vvP8O5j1+GmSQzP134dcMnjuM3meJLl+hj8CvXJ8LXh6eGxGDzcXAti8fCrJt5PsU1U3ba19bVz+2Ay/9B3ZWvv99EbjNYhPyhDCrppL3nzuKhlGZCad2lo7f0OKq6rdkvxNhumdf5IhME26xNc2ZBg9mXV92bHj856L7Pa2GUbhap4ptXM13HSRfpbip621785Y/oMk104y66ZP388wArVThrucTjq+tdZmbHNchvdwzwxTiG+WYYTo8621X85o/6kkh2T2e3XSjGVLUlU7Zvg8/zTD6NwVpprM+jyT2VMw/3d8vvLEstuPzx9ZQjl3GJ+vN8/nfOPx+aa5+BrEtNb+Zwl9z/LJ1tr+s1ZU1Q0yTFdeildkmF56bIb38g+6Gp9nnQMzd73E9otdG33XJNPXz74xfzjNes4d23hzofH6zF2SPDPD1N97VNXdWmsXztpJa+2/q+r0JI+vqhdn+ONDxTRb2CwJngCbkdba3M1fXpxhRPBZ4w14HpepmwrNqaq/ynDN3c8zTHP7bobRs5bkgUl2yyUDYjKMni5ZVe0z9r9Fkk9mmPr76wwjHLdK8mfz7OfH84Svuf1vv8iurzo+3258zGe7RfpZirPnWX5BkvWttXPmWZcMU0WnTQfROdPHPvd81jzt55bvsEBfG2S8pvSUJNfNcGObt2Q4hy7I8IeKp2X255kks8Lx3Puw5cSyuXq/v4SS5j7nxW4WtRyf87KoqpdneJ8+neEGP9N/yJn73K5WVVu31s5boK8r5OL3a77zYM7ctN4tM/wB6A/uQt1ae36S54/93itLC/4Z6/9GkidX1Z4Zrq19UJJ3L7DZGzJc733PjDdJaq2tXcr+gNVF8ATY/Lw5w9cfPLqqDk1yxyS7JvlUa236C+23yjCN7gcZpuf9aGr9HRfYz1JHYea8IMOo3EUjJBP7eUGG4DnL1aqqZoTPa4zP84W9TK2f+R2Jm7irz7N8+tjPnlo+7ZpT7SZt6Oc454kZQucLWmv/OLliPG+edin7nTQXUOcbOZ00d2wHtNY+vAz77mb8Y9ArMrxHn8gwjfUSX4HSWjujqs7K8PndKcP1yPO5a4Y/6pzRWlvwjwmttfOr6uQMI8p3y/DfjOX2+Qx/6LltFg6eb8nwh7LXZzh/L3GnYmDz4K62AJuZMTx+IMMNeO6fi+8a+boZza+e5IoZrgmdDp1XyuLTWDfEDTKMXp4wY92dF9jucrl4yuWk/cbnLy2y389nCFcLhehN1R3HkDJt7v2aO/bTM9w8aM95vpLkLuPzfN/vOp+5UbFZbjA+X+JOtFn489wQc1Os771gqz9su0l/zuPneUSG0PnRDCOdC33v5twshefNcy5k/Oqa544vZ/07X6jfv6mqyy9xmw0xN2V6wd81W2s/y3CN67UyzIB4Z4dagE2A4AmweZq7RupZGa4f+2mGr9aYdlaGwHKbcapekouu1XpV/vB6u8tqXZKdqurmkwur6kkZRl0WcthY09w2O+bikZEFR2taa2dl+DL721fVoTV8n+kfqOG7Sa+7+CFsdDfJcO3tRcavXtk3w3ccnpgk4xTMt2eYcvv3U+1vmOGrSM7PcHOfDfGzjNM8Z6xbNz7vN7W/vZI8ZwP3M5/3Z7j284FV9ZDplVV1rYmX/znW9PSa53thq2rv6ZBVVTepqhvPar/cxoD4xgyjxR9Kcv/W2u8W2eylGT7rOyf59xn1b5vhzrK3z/C1M69cYjlHZbiu9KZJPlhV840qz5qevaCqun6S+40vj13CJodm+O/UvdrwvaTAZshUW4DN0zEZvrLituPrV8+6EVBrbX1VvTrDl9SvraoPZLgu764ZQsxxWb7Rq5dnCJgnVtW7kvxqrO8OGUbN5vsuyzMzjMp+ZaK+AzNMy3tla+3EJez7yRlG6P45yUFVdUKG69yumeHGPHsleXCG61s3JR9J8sqqOiDD11PcMMN1t+dm+AqUyWmyczdtekZV3TbDZ7dThu9j3S7Jk1trf3At3xJ8MsOo90er6vgM4fVLrbWjM3wVxrOSvKqq9k/yrQzf7XqfDJ/nYtdaLqq1dl5VPTjDyOA7q+rgDDdD2iZDYLpThunbc20fOLb9aFX9d4bv9Dw3yXWS3CbDDZB2yvDHllTVmgx3Dl6fjfM70Ysy3Ojrt0lOS3LojEHML7bWLrqJT2vt1+N1lh/IcPOd+1TVRzJcm3uNJAdkmLnwxSw+enqR1toFVfWADN+9ep8kZ1TVcUm+Ota3U5JbZPj3eV6GmQOzPG78/JPhOuVdMsy02DbJ+1trH1xCLd/NpvdvD1hmgifAZmi8ydAbk8xde7fQXSIPTfLjDDcgelKG6+o+nmFE8cXLWNPRVXW/sd//k+FmMidlGDG7SeYPnudlCMIvTvLwDDeR+XaSf0rymiXu++zxusMnJXlYhuC6dYab93wzyV9muPPrpubEDMf5D7n4msmPJ3lea+0Lkw1baz+rqttlmHL5gCR/lSFAfDbD9a2fuBT7f1GSK2UIJnfMMO32jUmObq2dOb6nh2UIgPfKEOKelOQzWYbgmSSttc9X1R4ZztN7ZfgKjl9nCLovnGr7paraPcOx3yfDOX1hhpH9L2S4zvgXy1HXpXS98XnbXDw1dtol7h7bWls3jiQflOF9vW+GkchfZgjXz0tyVGvtgmyA8Q7If1ZVd0/y6Awhc58MAfLnGULooUn+X2ttvhs8PXayywzX2n4hw7Wbb9qQeoDNW82+USAAsFLGEaSPZ8aNewBgNXKNJwAAAF0JngAAAHQleAIAANCVazwBAADoyl1tWZKjjjqqPeYxj1npMgAAgE3XJb4jao6ptizJb37j+5wBAIBLR/AEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoKs1K10Aq8Pa75+dXQ45eqXLAAAAkqw77ICVLmGDGPEEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhq2YNnVZ0z9fqgqnr1Itss2mZs9/aqOq2qnrlAm/2q6kNLr/jSGWu+sKp2n1j2larapfe+l6Kq7l9VN1vpOgAAAFbNiGdVXSPJ3q213VtrL1/pekZnJnneSu28qrZcYPX9k2xQ8KyqNZetIgAAgEvaqMGzqnaqqvdW1cnjY58ZbY6sqiOq6viq+kZV3WdcdUySq1XVqVV1x6o6tqr2GrfZsarWzejrhVX1prHtGVX19Il1j6yqk8b+/r2qthwfR44jl2vnRlar6ulVdfo42vqOiV18KMnNq+rGM/Z9zsTPB1bVkRPHd3hVfXqs6c5jjV+bazO2u0dVfbaqvlhV766q7cbl66rqb6vqhCQPrqonjO/ll8f3dtuq2jvJfZO8dDy+Xatqj6r63HgM/1lVVx77O7aq/rmqjkvyjCV9kAAAABugR/DcZgw7p1bVqUn+fmLdK5K8vLV2myQPSvKGefrYJcmdkxyQ5IiqunyGIPXt1toerbXjN6CemyS5Z5LbJvm7qtqqqm6a5KFJ9mmt7ZFkfZJHJNkjyc6ttVu01nZL8uaxj0OS7Nla2z3JwRN9X5jkX5I8dwPqSZIrJ7lrkmcm+WCSlye5eZLdxoC4Y5LnJ9m/tXarJKck+auJ7X/XWtu3tfaOJO9rrd2mtXbLJF9L8vjW2olJPpDkb8b369tJ3pLkOeMxrE3ydxP97dBau3Nr7WWTRVbVE6vqlKo6Zf1vz97AQwQAABj0CJ7njmFnjzHU/e3Euv2TvHoMpB9IcqWquuKMPt7VWruwtfbNJGdkCI+X1tGttfNaaz9N8uMkV09ytyS3TnLyWMvdklx/3Nf1q+pVVXWvJL8a+zgtyduq6pFJLpjq/z+S3L6qrrcBNX2wtdYyBMAftdbWttYuTPLVDKH79hmmyf73WN9jklx3Yvt3Tvx8i3F0eG2G8Hzz6Z1V1fYZwuVx46Kjktxpnv4u0lp7XWttr9baXltuu/0GHB4AAMDFNvY1fVskuUNr7dzJhVU13a4t8joZAuBccL78Avs8b+Ln9RmOuZIc1Vo7dLpxVd0ywwjpXyR5SJLHZRh5vVOGUdcXVNVF4a61dkFVvSzJcxaoebq+uZounKrvwrG+9Uk+3lp72DzH9JuJn49Mcv/W2per6qAk+82zzUJ+s3gTAACAS2dj31zomCRPnXtRVXvM0+7BVbVFVe2aYSTy6zParMswapkkB25gHZ9McmBVXW2s4ypVdd1xiusWrbX3JnlBkltV1RZJrt1a+3SSZyfZIcl2U/0dmWE0d6eJZT+qqpuO2z9gA+v7XJJ9quoGY33bVtWN5ml7xSRnVdVWGUY85/x6XJfW2tlJflFVdxzXPSrJcQEAANgINvaI59OTvKaqThv3/Zn84TWTc76eIRhdPcnBrbXfzRgV/dck76qqRyX51IYU0Vo7vaqen+SYMRj+PsMI57lJ3jwuS5JDk2yZ5K3jdNXKcI3qLyfraa2dX1WvzHAN65xDMtx86H+TfCWXDKsL1feTcfTy7VW19bj4+Um+MaP5C5J8Psl3M0zdnZu6/I4krx9vqHRghum6R1TVthmmFD92qfUAAABcFjVcarjpGO/s+qHW2ntWuhYu9uTnvbh9ZP3uizcEAAC6W3fYAStdwiyXGC2cs2q+xxMAAIDVaWNPtV1Ua+2gla4BAACA5WPEEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKCrNStdAKvDbjtvn8OfcsBKlwEAAKxCRjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6GrNShfA6rD2+2dnl0OOXukyAACgq3WHHbDSJWyWjHgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHS1aPCsqnOmXh9UVa9eZJtF24zt3l5Vp1XVMxdos19VfWixvi6rseafVNWXquqbVfWxqtq7934XqOfgqnr0pdx2l6p6+HLXBAAAcGmsWakdV9U1kuzdWrvuStUwwztba09Nkqq6S5L3VdVdWmtf29iFtNaOuAyb75Lk4Un+Y6kbVNWa1toFl2GfAAAAM12mqbZVtVNVvbeqTh4f+8xoc2RVHVFVx1fVN6rqPuOqY5JcrapOrao7VtWxVbXXuM2OVbVuRl8vrKo3jW3PqKqnT6x7ZFWdNPb371W15fg4sqq+UlVr50ZWq+rpVXX6ONr6jlnH1lr7dJLXJXniuM2uVfXRqvrCeCw3mTi+V1bViWNNB47Lq6peOrHvh47L96uq46rqXeP7cVhVPWKsfW1V7TpxrH89/nxsVb1kbPONqrrjuHyXsZYvjo+5EdrDktxxfC+eWVWXr6o3j/1/aQzVc6O8766qD46fBwAAwLJbyojnNlV16sTrqyT5wPjzK5K8vLV2QlVdJ8nHktx0Rh+7JLlzkl2TfLqqbpDkvkk+1FrbI0mqaqk13yTJXZJcMcnXq+rwJDdI8tAk+7TWfl9Vr03yiCRfTbJza+0W4z52GPs4JMn1WmvnTSyb5YtJnjT+/LokB7fWvllVt0vy2iR3HdddM8m+Y20fSPKeJA9MskeSWybZMcnJVfWZsf0tM7xPP09yRpI3tNZuW1XPSPK0JH85o5Y1Y5s/TfJ3SfZP8uMkd2+t/a6qbpjk7Un2Go/vr1tr9xmP+1lJ0lrbbQzMx1TVjcZ+75Bk99baz6d3WFVPzBi8n/CXz0m2XuCdAgAAmMdSgue5c+EwGUbJMoSbZAg/N5sIjVeqqivO6ONdrbULk3yzqs7IENB+eSlrPrq1dl6S86rqx0munuRuSW6dIdwlyTYZQtkHk1y/ql6V5OhcPKp3WpK3VdX7k7x/gX1VklTVdkn2TvLuiWOdjGHvH4/v9Kq6+rhs3yRvb62tT/KjqjouyW2S/CrJya21s8a+vz1R19oMoXqW943PX8gQ5JNkqySvrqo9kqxPcqMZ283V8qokaa39T1V9d6Ltx2eFzrHt6zIE7jz5eS9uWT9P7wAAAAu4rNd4bpHkDq21cycXzhi9bIu8TpILcvHU38svsM/zJn5en+EYKslRrbVDpxtX1S2T3DPJXyR5SJLHJTkgyZ0yjLq+oKpuPs++9kzytbGuX04G8AVqqqnnxdpfOPH6wsz/mcy1WT/R5plJfpRhBHWLJL+bZ9uFavnNAusAAAAus8v6dSrHJHnq3Itx5G2WB1fVFuP1i9dP8vUZbdZlGLVMkgM3sI5PJjmwqq421nGVqrpuVe2YZIvW2nuTvCDJrapqiyTXHq/hfHaSHZJsN91hVd05wzTT17fWfpXkO1X14HFdjYF2IZ9J8tDxOtOdMgTdkzbwuBazfZKzxtHWRyXZclz+6wxTkSdreUSSjFNsr5PZnwEAAMCyu6wjnk9P8pqqOm3s6zNJDp7R7utJjsswLfbg8ZrE6Tb/muRdVfWoJJ/akCJaa6dX1fMzXLu4RZLfZxjhPDfJm8dlSXJohnD21qraPsNI4Mtba78c63loVe2bZNsk30nyoIk72j4iyeHjfrZK8o4kX16grP/McP3klzOM8D67tfbDuZsSLZPXJnnvGIg/nYtHL09LckFVfTnJkWO7I6pqbYaR5YPG61uXsRQAAIDZqrVZs16XcQdVR2a4idB7uu6Irp78vBe3j6zffaXLAACArtYddsBKl7CazTuydVmn2gIAAMCCLutU20W11g7qvQ8AAAA2XUY8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhqzUoXwOqw287b5/CnHLDSZQAAAKuQEU8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBGd38LQAABFsSURBVE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAulqz0gWwOqz9/tnZ5ZCjV7oM2GysO+yAlS4BAGCjMeIJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANDVmpUugKWpqqsm+eT48hpJ1if5yfj6t621vVekMAAAgEUInqtEa+1nSfZIkqp6YZJzWmv/uqJFAQAALIGptpuBqjpnfN6vqo6rqndV1Teq6rCqekRVnVRVa6tq17HdTlX13qo6eXzss7JHAAAAbM4Ez83PLZM8I8luSR6V5EattdsmeUOSp41tXpHk5a212yR50LjuEqrqiVV1SlWdsv63Z/evHAAA2CwJnpufk1trZ7XWzkvy7STHjMvXJtll/Hn/JK+uqlOTfCDJlarqitMdtdZe11rbq7W215bbbr8RSgcAADZHrvHc/Jw38fOFE68vzMWf9xZJ7tBaO3djFgYAAPxxMuL5x+mYJE+de1FVe6xgLQAAwGZO8Pzj9PQke1XVaVV1epKDV7ogAABg82Wq7SrUWnvh1Ovtxudjkxw7sXy/iZ8vWtda+2mSh3YuEwAAIIkRTwAAADoTPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoas1KF8DqsNvO2+fwpxyw0mUAAACrkBFPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALpas9IFsDqs/f7Z2eWQo1e6DDYR6w47YKVLAABgFTHiCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4JnB1W1vqpOraqvVNW7q2rby9DXQVX16suw7Z9MvD62qvZazn0AAAAsRvDs49zW2h6ttVskOT/JwZMra7Ax3vuDkvzJYo0AAAB6Ejz7Oz7JDapql6r6WlW9NskXk1y7qh5WVWvHkdGXzG1QVY+tqm9U1XFJ9plYfmRVHTjx+pyJn5899vXlqjpsbLdXkreNo6/bTBY13z4AAACWm+DZUVWtSXLvJGvHRTdO8pbW2p5Jfp/kJUnummSPJLepqvtX1TWTvChDGLx7kpstYT/3TnL/JLdrrd0yyb+01t6T5JQkjxhHX8+daL+kfVTVE6vqlKo6Zf1vz97wNwAAACCCZy/bVNWpGYLf95K8cVz+3dba58afb5Pk2NbaT1prFyR5W5I7JbndxPLzk7xzCfvbP8mbW2u/TZLW2s8Xab+kfbTWXtda26u1tteW226/hDIAAAAuac1KF7CZOre1tsfkgqpKkt9MLlpg+zbP8gsy/rGghg4vN9HXfNts6D4AAACWlRHPlfP5JHeuqh2rasskD0ty3Lh8v6q6alVtleTBE9usS3Lr8ef7Jdlq/PmYJI+bu3tuVV1lXP7rJFecZ9/z7QMAAGBZGfFcIa21s6rq0CSfzjBi+eHW2n8lSVW9MMlnk5yV4UZEW46bvT7Jf1XVSUk+mXEEtbX20araI8kpVXV+kg8neW6SI5McUVXnJrnD1L7n2wcAAMCyqtbMuGRxT37ei9tH1u++0mWwiVh32AErXQIAAJueeS8nNNUWAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALpas9IFsDrstvP2OfwpB6x0GQAAwCpkxBMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArtasdAGsDmu/f3Z2OeTolS5jk7fusANWugQAANjkGPEEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8NzEVNUuVfWVqWUvrKq/Xu5+J9YdW1V7XZb+AQAA5iN4/hGoqjUrXQMAAPDHS/BcRarq6VV1elWdVlXvGJddoareVFUnV9WXqup+4/KDqurdVfXBJMdM9bNNVb1j7OedSbbZ+EcDAAD8sRA8V5dDkuzZWts9ycHjsucl+VRr7TZJ7pLkpVV1hXHdHZI8prV216l+npzkt2M//5Tk1rN2VlVPrKpTquqU9b89e7mPBQAA+CMheG562gLLT0vytqp6ZJILxuX3SHJIVZ2a5Ngkl09ynXHdx1trP5/R152SvDVJWmunjf1ecoetva61tldrba8tt93+0hwLAACA4LkJ+lmSK08tu0qSnyY5IMlrMoxQfmG8drOSPKi1tsf4uE5r7Wvjdr9ZYD/zBVwAAIBlJXhuYlpr5yQ5q6ruliRVdZUk90pyQpJrt9Y+neTZSXZIsl2SjyV5WlXV2H7PJezmM0keMba/RZLdl/s4AAAA5rjb6abp0UleU1UvG1+/KMn3kny6qrbPMMr58tbaL6vqH5L8W5LTxvC5Lsl9Fun/8CRvrqrTkpya5KQOxwAAAJBE8NwktdZOz3CjoGn7zmh7bpInzVh+ZJIjJ16vS3KLiW3+z7IUCwAAsAhTbQEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgqzUrXQCrw247b5/Dn3LASpcBAACsQkY8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKCraq2tdA2sAs95znN+vdVWW319petg83HOOefsuN122/10petg8+GcYjk5n1huzimW2yZ6Tv30H//xH+81a4XgyZJU1Smttb1Wug42H84plptziuXkfGK5OadYbqvtnDLVFgAAgK4ETwAAALoSPFmq1610AWx2nFMsN+cUy8n5xHJzTrHcVtU55RpPAAAAujLiCQAAQFeCJwAAAF0JnvyBqrpXVX29qr5VVYfMWF9V9cpx/WlVdauVqJPVYwnn1CPGc+m0qjqxqm65EnWyOix2Pk20u01Vra+qAzdmfaw+Szmnqmq/qjq1qr5aVcdt7BpZXZbw/73tq+qDVfXl8Zx67ErUyepQVW+qqh9X1VfmWb9qfjcXPLlIVW2Z5DVJ7p3kZkkeVlU3m2p27yQ3HB9PTHL4Ri2SVWWJ59R3kty5tbZ7kn/IKrtQno1niefTXLuXJPnYxq2Q1WYp51RV7ZDktUnu21q7eZIHb/RCWTWW+N+pv0hyemvtlkn2S/KyqrrcRi2U1eTIJPdaYP2q+d1c8GTSbZN8q7V2Rmvt/CTvSHK/qTb3S/KWNvhckh2q6pobu1BWjUXPqdbaia21X4wvP5fkWhu5RlaPpfw3KkmeluS9SX68MYtjVVrKOfXwJO9rrX0vSVprzisWspRzqiW5YlVVku2S/DzJBRu3TFaL1tpnMpwj81k1v5sLnkzaOcn/Trw+c1y2oW1gzoaeL49P8pGuFbGaLXo+VdXOSR6Q5IiNWBer11L+G3WjJFeuqmOr6gtV9eiNVh2r0VLOqVcnuWmSHyRZm+QZrbULN055bIZWze/ma1a6ADYpNWPZ9PftLKUNzFny+VJVd8kQPPftWhGr2VLOp39L8pzW2vphMAEWtJRzak2SWye5W5Jtkny2qj7XWvtG7+JYlZZyTt0zyalJ7ppk1yQfr6rjW2u/6l0cm6VV87u54MmkM5Nce+L1tTL8NW5D28CcJZ0vVbV7kjckuXdr7Wf/v707Vq0iiqIw/C/UxtaAnSSFmM4moI0vkCewUUgnYm9nChufQCxE7LQQ0RRibWk6Q0gTFESwSWMRq4vbYgYJcsk9zUwy4f/aO8UuNjNn3X3mzEi1aXpa+mkNeN2HziVgPcmsqt6NU6ImpvW5d1BVh8Bhkk/AdcDgqXlaemoDeFJVBewn+QasAp/HKVFnzGTW5m611VHbwNUkK/1L7reBrf+u2QLu9ido3QR+VdXPsQvVZCzsqSRXgLfAHScIWmBhP1XVSlUtV9Uy8Aa4b+jUMVqee++BW0nOJ7kI3AD2Rq5T09HSU9/pJugkuQxcA76OWqXOksmszZ146p+qmiV5QHcS5DngRVXtJrnX//4M+ACsA/vAb7p/7aS5GnvqEXAJeNpPqWZVtXZSNev0auwnqVlLT1XVXpKPwBfgD/C8quZ+1kBqvE89Bl4m2aHbJvmwqg5OrGidakle0Z1+vJTkB7AJXIDprc3TTfklSZIkSRqGW20lSZIkSYMyeEqSJEmSBmXwlCRJkiQNyuApSZIkSRqUwVOSJEmSNCiDpyRJkiRpUAZPSZIkSdKg/gLCwfYNUXm8RgAAAABJRU5ErkJggg==\n","text/plain":"<Figure size 1008x720 with 1 Axes>"},"metadata":{"needs_background":"light"},"output_type":"display_data"}]},{"metadata":{"trusted":false},"cell_type":"code","source":"# Partial Dependence Plot\npdp_helpfulness = gbm_baseline.partial_plot(train, cols = [\"HelpfulnessNumerator\"])","execution_count":20,"outputs":[{"name":"stdout","output_type":"stream","text":"PartialDependencePlot progress: |█████████████████████████████████████████| 100%\n"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAf4AAALWCAYAAACnePHjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeZwcdZ3/8dcn90kCSbgRUEgwCILcChoE5FDEXRQ5hJV1RdxF190frq6uCup6r67rhawLCIiIKAoxioqMgMgtIFfkEEiADIFAJgfkmHx/f1QP6cz0zPRMpqe7ql7Px6MffXyrqz/VXTPvrm9XfStSSkiSpHIY0ewCJEnS8DH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDXxstIj4WEd+rc9oLIuKzja5pOETEoxFxaLPrqEeR3ve+RMQOEZEiYlSd0/9NRCyIiOURsWc/086JiIVDU6nUPAZ/CVQC6oXKP7f2iDg/IiYNcl49/vmllD6XUvqHIajz3RHRWalzeUT8tVLrzI2ddxlERFtEvFh5756JiJ9GxFaDmE+KiJ36aO/+OS2PiG9uXPUvzfusiLh4oDVthK8AZ6SUJqWU/tSA+Q+Jqi80v+j2+MURcVaTyhqQBn6GGiCDvzyOTilNAl4D7AP8x0BnUO9W1Eb6Y6XOKcChwAvA7RHxqmF47SI4o/L+zQSmAl9r0Ov8sRKWXZczBjqDYVqf+rM9cG+zixiA/SPidc0uortGf5Ytsq4UhsFfMimlJ4BfAq8CiIhTI+L+iFgWEY9ExPu6pu3auo+Ij0TEIuCHleduXbWlt3X3rbSI+HFELIqIpRFxXUTsOog6O1NKD6eU/hH4PXBW1fz3j4gbI+L5iLgrIuZUtbVFxOcj4pbK6/88IjYbwHM/ExF/qLwfv46I6VXtJ0fEYxHxbER8vLreiBgRER+NiIcr7Zd1vW7V1trfRcTjla3xj1c9d2Tl55KHK697e0RsV2nbJSJ+ExFLImJ+RBxX5/u3BPgJlc+5u4h4b0Q8VJnvlRGxdeXx6yqT3FX5fN9Zz+tVzXdKRFwYEYsr79V/RMSIStu7K+/t1yJiCVWf6QBfo9f3usa0NdeHiBgbEcuBkZVlfbgy/QZbpdHHTySR9aSdGRF3V+b9o4gYV9X+loi4s7Ku3RgRu1e1fSQinqh83vMj4pDK4/tGxG0R0RFZ79xXu73sl4De6nl3RNzQ7bGXlqeyLN+OiF9WPts/RMSWEfHfEfFcRDwQVT93RPa3/ZPKZ/nXiPhgVdtZEXF5ZD0OHcC7K7X/sbK8T0XENyNiTGX6mutVb+thVe3/FBEPAg/WWmYNjsFfMpVAOQro6tZ8GngLsAlwKvC1iHhN1VO2BDYj2zI6BTgSeLJqS+/JGi/zS2BnYHPgDuAHG1n2T4GDKvVvA/yC7J/fZsCZwE8iYkbV9KcAfw9sDawF/mcAzz2R7H3YHBhTmYaImA18Bzi5Mt9pwLZVz/sg8DbgDZX254BvdVuOA4FZwCHAJyPilZXH/xU4gexz2aRS+8qImAj8BrikUs8JwLejji9SkX1hOZb1n3N12xuBzwPHAVsBjwGXAqSUXl+Z7NWVz/dH/b1WN98g6615Odl7cQrZ+9llP+CRyvL85wDn3aWe97paj/UhpbSq0jMC2bK+YpC1HAccAewI7A68G6DyN3Qe8D6ydeW7wJWVLxyzgDOAfVJKk4HDgUcr8/s68PWU0ibAK4DLur3et4CZMfh9S44j6+2bDqwC/kj2NzoduBz4aqX+EcBVwF3ANmTr7Ici4vCqeR1Tec5Usr/xTuBfKvM6oPKcf4Ta61Vf62GVt5GtM7MHubyqJaXkpeAXsn8qy4Hnyf64vg2M72XanwH/XLk9B1gNjKtqnwMs7Pacs4CLe5nfVCABUyr3LwA+28u07wZuqPH4EcCayu2PABd1a78a+LvK7TbgC1VtsyvLMLLO5/5HVds/Ar+q3P4kcGlV28TKfA+t3L8fOKSqfStgDTAK2KHyHmxb1X4LcHzl9nzgmBrL/U7g+m6PfRf4VC/vXxuwsvI5P0H2z3hG9/cd+D/gS1XPm1SpdYfK/QTs1Mf69G6yAH2+6rJ/5T1eBcyumvZ9QFvV8x7vZ109q/K+Pt/t8lJNdb7Xo/pbH2ota4371e/bHKrWfbK/q3dV3f8ScE7l9neAz3RbtvlkX1Z2IvvCfSgwuts01wFnA9O7Pf7ScpGtlzdVHr8YOKu3v59u79sFwP9WtX0AuL/q/m7A85Xb+3X/rIB/B86v+pyu6+ez/BBwRR/vbT3r4Rv7eg0vg7u4xV8eb0spTU0pbZ9S+seU0gsAEXFkRNxU6Wp7nmyrc3rV8xanlF6s90Ui67b+QqUbtoP1WzLT+3haf7YBllRubw+8o9Kd+Hyl5gPJ/vl3WVB1+zFgdOX163nuoqrbK8n+GUG2tfjSfFNKK4Bnq6bdHriiar73k20BbVHHvLcDHq6x3NsD+3Wr9ySyXpjefLDyOW+TUjoppbS4xjRbk70vXcuyvLIs2/Qx3+5uqrxO1+Umsvd4TPW8K7er51v92fTmsm7zntqtvZ73ulpv68NQ6O0z3R74f90+u+2ArVNKD5GF4lnA0xFxaVUX93vI9s94ICJujYi31HjN/wW2iIijB1Fve9XtF2rcr65/6271f4wN3+MNPsuImBkRcyP7ma8D+Bx9v8/1rIf1rC8aIIO/xCJiLNnvwF8Btqj8g50HRNVk3U/f2N/pHE8k6wI8lKzLd4eul9uIUv8GuL5yewHZVnt1MExMKX2havrtqm6/jGwr4pk6n9ubp6rnGxETyLpwuywAjuw273Ep26eiPwvIunVrPf77bvOclFJ6fx3z7MuTZP/YAaj8pDCNrJdgYzxD9l5vX/XYy7rNdyhOBzrQ97q39aGWlcCEqvt9fcnqr8b/7FbjhJTSDwFSSpeklA4ke68S8MXK4w+mlE4g+ynki8Dllc/nJSmlNWS9Ap9hw7+rFdW1R8Rga++q/6/d6p+cUjqqupRuz/kO8ACwc8p+qvgYff/d17MeevrYBjD4y20MMBZYDKyNiCOBN/XznHZgWkRM6aV9Mll377Nk/4Q+N5jCKj0HO0bEN8i6WM+uNF0MHB0Rh1emGRfZTojVv7e/KyJmV8L508DlKaXOOp/bm8uBt0TEgZUdlj7Nhn8/5wD/GRHbV+qfERHH1Lm43wM+ExE7R2b3iJgGzCX7PffkiBhduexTtW/AYF0CnBoRe1S+/H0OuDml9GilvZ3sN/oBqbzHl5G9D5Mr78W/kr3vQ2mg73Vv60MtdwInVtaPI8i65gfjf4HTI2K/ymc6MSLeXHlfZkXEGyvv/YtkW9qdlWV5V0TMSCmtI/uJg662bi4i+9s9ouqxu4BdK5/rOAa582TFLUBHZDshjq+8H6+KiH36eM5koANYHhG7AN2/oHZfr/pbD9UgBn+JpZSWke0odRnZDlInAlf285wHyPbuf6TSBbh1t0kuJOu+ewK4D7hpgGUdENne1h1kv89uQrYT1J8rr7+ArEfhY2RfWBYAH2bDdfkist8zFwHjKstY73NrSindC/wT2T+rp8jer+rxDL5O9t79OiKWVZZ7vzqX+atkn8GvK8v9f2T7YCwj+yJ2PNnW0SKyrcCxdc63t2W5BvgEWW/PU2S9DcdXTXIW8P3K51vXUQRVPkC25fkIcAPZ+3XextRbw0Df65rrQy/+GTiaLHRPItvnZcBSSrcB7wW+SbauPERlxz+yz+8LZL0Oi8i27j9WaTsCuLfyN/B1sv1AevzUVvni8imynVS7HvsL2Reb35LtBX9D9+cNoP5OsvdhD+CvlVq/R9aL15szyf6HLCP74tN9x9CzqFqv6lgP1SCRkj0pKo6IaCPb0bCukQRVbK4PUk9u8UuSVCIGvyRJJWJXvyRJJeIWvyRJJWLwS5JUIrk749HUqVPTTjsV68yOK1asYOLEif1PmCMuUz64TPngMuVDKy3T7bff/kxKaUatttwF/xZbbMFtt93W7DKGVFtbG3PmzGl2GUPKZcoHlykfXKZ8aKVliojHemuzq1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBJpWPBHxHkR8XRE3NNLe0TE/0TEQxFxd0S8plG1SJKkTCO3+C8Ajuij/Uhg58rlNOA7DaxFkiTRwOBPKV0HLOljkmOAC1PmJmBqRGzVqHokSVJzf+PfBlhQdX9h5TFJktQgkVJq3MwjdgDmppReVaPtF8DnU0o3VO5fA/xbSun2GtOeRvZzADNmzNjrsssua1jNzbB8+XImTZrU7DKGlMuUDy5TPrhM+dBKy3TwwQffnlLau1bbqOEupspCYLuq+9sCT9aaMKV0LnAuwKxZs9KcOXMaXtxwamtrw2VqfS5TPrhM+eAyNU8zu/qvBE6p7N2/P7A0pfRUE+uRJKnwGrbFHxE/BOYA0yNiIfApYDRASukcYB5wFPAQsBI4tVG1SJKkTMOCP6V0Qj/tCfinRr2+JEnqyZH7JEkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg1+SpBIx+CVJKhGDX5KkEjH4JUkqEYNfkqQSMfglSSoRg7/KunWwcGGzq5AkqXFGNbuAVnLhhXDeedDWBiNGwMEHw8qVsNVWsOWW2fU++8BRR2XTP/kkTJ8OY8Y0tWxJkupm8Fe5/HK47z6IyO6/8pXw8MPwyCNw442weDGcckoW/CnBTjvBCy/AtGnrvxgcdxy8971Z+6WXZo93tU2Zsn7ekiQ1g8FfsXIlXHMNnHba+nD+9rc3nGbNGnjxxez2unXw9a/DokXw1FPrrzs6svbnnoMTT9zw+ePGwWc/C//v/8GCBTBnTvb4iy/ux7hx2e1PfSr7cvHAA/DmN/es80tfgmOPhdtvz75kdPftb8Phh8Pvfw9///c927//fTjwQPjlL+GMM3q2X3457Llndv2Rj/Rs/+UvYebMbD6f/nTP9uuug222gSuu2Jr3vKdn++23w9Sp2XJ897s92x94AEaPhrPOgosu2rBtzBi4//7s9oc/DD/96Ybtm24Kt92W3X7/++HXv96wfdtts/cFsvf4D3/YsH3WLJg3L7t97LFw550btm+33Wza2rLbRxwBDz64YfvrXw/nn5/dPuigrEeo2hFHwLe+ld1+zWtg6dIN2489NntfumpZu3bD9lNOydaPVatg9mx6+Md/zNatJUuynqnuPvxhOP101z3XPde97oq87tVi8Ff87ndZqL/lLb1PM3p0dgEYOTLbsu/NJptkvQfdvxjssUfWPnYsvPa12e329qVsscV4IOsZAJgwYX17tc03z64nT67dvtlm2fWmm9Zunzo1u542rXb75MnrX6dW+/isTLbcsnb72LHZ9YwZq2q2j6qscdtvX/v5XV+6dtyxZ3vXew9Zb0v39kmT1t/eZRdYvnzD9hkz1t+ePTvrlam23Xbrb++2W/YZbFj7ipdu77nnhvODrIeoy157wQ47bNg+a9b62/vuCytWbNi+007rbx9wAHR2btj+8pdn1xG137uXvSy7Hj26dvs222TXrnuue657GyriuvfIIz1f56XXS93XwBY3a9asNH/+/CGf7/veB5dcAs88s/5DHC5tbW3M6foaXBAuUz64TPngMuVDKy1TRNyeUtq7Vptb/BUHHph96x7u0JckaTgZ/BUnn9zsCiRJajyP4wfuvhuefrrZVUiS1Hhu8ZPtpDdiBPzxj82uRJKkxir9Fn97O9xyS99780uSVBSlD/6uY2cNfklSGZQ++K+6KhtcY/fdm12JJEmNV+rgX70afvObbGvfoXQlSWVQ6p37xoyBP/+55yhakiQVVamDH3oObSlJUpGVtqs/peyEBtdc0+xKJEkaPqUN/vvvh3POyU67K0lSWZQ2+K+6KruudQpISZKKqrTBP3dudnrLrtNFSpJUBqUM/mefhRtvdNAeSVL5lDL4FyyA2bPh6KObXYkkScOrlIfz7bFHdvy+JEllU7ot/s5OWLWq2VVIktQcpQv+66+H6dPh5pubXYkkScOvdME/d242Rv+uuza7EkmShl8pg//gg2HSpGZXIknS8CtV8D/4IMyf72F8kqTyKlXwz52bXTtanySprEoV/IceCl/5Cuy4Y7MrkSSpOUp1HP9uu2UXSZLKqjRb/HfcAb/+dXYcvyRJZVWa4P/v/4YTTmh2FZIkNVcpgr+zE+bNg6OOgpEjm12NJEnNU4rgv/nm7Ix8npRHklR2pQj+uXNh1Ch405uaXYkkSc1ViuD/wx/goINg6tRmVyJJUnOV4nC+3/0Onnmm2VVIktR8pdjiHzkSttii2VVIktR8hQ/+974XPv/5ZlchSVJrKHTwL18OF14Iixc3uxJJklpDoYP/t7+F1as9jE+SpC6FDv65c2GTTeDAA5tdiSRJraGwe/WvWwe/+AUccQSMHt3saipSgscfz7ohqoxfuBAefLBJRTWGy5QPLlM+uEz5kJdlKmzwL18Ohx8OxxzT5ELWrIHrr4ef/xyuvBIefbTHJPsNf1UN5zLlg8uUDy5TPuRlmQob/JtsAhdc0KQX7+iAX/4yC/p58+D552HcODj0UDjzzB4jCd13333Mnj27ScU2hsuUDy5TPrhM+dBSy/Sud/XaVNjgf+gheMUrIGKYXvDxx+Gqq7It+7a2bEt/+nR429uybofDDoOJE2s+9em2NmbPmTNMhQ4PlykfXKZ8cJnyoaWWqWzB/8QTsPPO8PWvwwc/2KAXSQnuvHN9F/6f/pQ9PnMmfOhD8Na3wgEHeDpASVJLKWTwz5uXXb/xjUM849Wrs635K6/MLgsWZF0Kr30tfOlLWdjPmjXELypJ0tApZPDPnQvbbw+77joEM3v++eybxJVXZr/bd3TA+PHZnoNnnw1vfjNsvvkQvJAkSY1XuOB/4YVs4J5TT92I3/cffXR9F/5118Hatdlg/8cdl/1ef8ghWfhLkpQzhQv+tjZYuXKQo/Xddx+ceCLcdVd2f/Zs+PCHsy78ffeFEYUe70iSVAKFC/4DD4TLL4c3vGEQT/7Vr7LQ/+IX4W//FnbaacjrkySpmQoX/JMnw7HHDvLJ7e0wZky2lT9sxwFKkjR8CtV3PX9+dgreZ54Z5AwWLYIttzT0JUmFVajgv/xy+NjHsn3xBqUr+CVJKqhCBf/cubDPPhuR3YsWZXvvS5JUUIUJ/qefhptvhre8ZSNm0t7uFr8kqdAKE/y//GU2iu6gg7+zExYvNvglSYVWmOBfsAB23BH23HOQM1i8GNats6tfklRohQn+//gP+MtfNmKH/Pb27NotfklSgRUi+FPKrkdtzKgEixZl1wa/JKnAGhr8EXFERMyPiIci4qM12jeNiCsi4u6IuCUiXjWY1znzzGz4/K4vAIPSFfx29UuSCqxhwR8RI4FvAUcCs4ETImJ2t8k+BtyZUtodOAX4+kBfJyX42c9g3LiNHHfHrn5JUgk0cot/X+ChlNIjKaXVwKXAMd2mmQ1cA5BSegDYISIGtMk9fz488shGHsYH2Rb/xIkwadJGzkiSpNbVyLH6twEWVN1fCOzXbZq7gL8FboiIfYHtgW2B9uqJIuI04DSAGTNm0NbW9lLbpZduB7yCadP+SFvbqkEX+8q77mKTKVO4uWrew2X58uUbLFMRuEz54DLlg8uUD7lZppRSQy7AO4DvVd0/GfhGt2k2Ac4H7gQuAm4FXt3XfGfOnJmqvf71Kb361WnjHXxwSq997RDMaOCuvfbaprxuI7lM+eAy5YPLlA+ttEzAbamXHG3kFv9CYLuq+9sCT3b70tEBnAoQEQH8tXKp2/HHZz30G629HXbZZQhmJElS62pk8N8K7BwROwJPAMcDJ1ZPEBFTgZUp2wfgH4DrKl8G6vb+9w9RtYsWwZw5QzQzSZJaU8N27ksprQXOAK4G7gcuSyndGxGnR8TplcleCdwbEQ+Q7f3/zwN5jeuuy8bo32irV8OSJR7KJ0kqvEZu8ZNSmgfM6/bYOVW3/wjsPJh5r10Lb3sbvPWtcMEFG1Xm+m8PHsonSSq43I7cd+ON8NxzcPTRQzAzR+2TJJVEboN/7lwYPRoOO2wIZuaofZKkksh18L/hDbDJJkMwM0ftkySVRC6D//HH4f77h2C0vi5u8UuSSqKhO/c1ysteBn/9K0yePEQzXLQIpkzJBvyXJKnAchn8ADvsMIQzW7TIbn5JUinkrqt/3brguOPg1luHcKbt7Qa/JKkUchf8K1aM5Mc/hhdeGMKZusUvSSqJHAb/KKZOhde+dghnumiRO/ZJkkohl8F/5JEwaqj2Tli5EpYtc4tfklQKuQv+zs4YmtH6ungMvySpRHIX/GPHruPww4dwhh7DL0kqkdwF//bbr2CzzYZwhm7xS5JKJLfH8Q+ZYTxBz8qVcOmlsGLF+sdOPTW7vuUWuPnmns857TQYOxZuuAH+9Kee7WecARFw7bVwzz0bto0aBe9/f3b717+G+fM3bJ8wAd7znuz23LnZoEjVpkyBU07Jbl9xBSxcuGH79OlwwgnZ7R/9aMNTJD/44DY8+ywce2x2/+KLs5MqVdt+++zsigDnnbfh+wKw885wxBHZ7e9+Nzt7crXZs+GQQ7Lb3/gGPeyxBxx0EKxZA+ec07N9n31g//2z1z3vvJ7tr3sdvOY18PzzcNFF2TL9+c/r2+fMgd12g8WLs8+1u8MOg112gSeegJ/+tGf7m98ML385PPooXHVVz/a3vQ222w7+8he4+uqe7ccdl3VU3Xsv/O53PdtPOgk22yxbb264oWe7615223WvZ7vrXnY7z+ten1JKubrMnDkzDalPfSolSGn16qGdbzf33JPSLrtkL1V9WbAgpWuvvTadfXbPNkjp+eez5595Zu32zs6s/X3v69k2YcL61z/ppJ7tW2yxvv2tb+3Z/opXrG8/+OCe7Xvssb597717th900Pr2WbN6th911Pr2bbbp2f7Od65v32STnu3vec/69oie7R/6UNa2fHnt9+4Tn8jan3qqdvuXvpS1P/hg7fbvfCdrv+OO2u0XXZS1//73tdt/9rOsfe7c2u3XXJO1//CHtdtvuSVr/9//rd1+//1Z+1e/Wrvddc91z3WvuOsecFtKtXM0svb8mDVrVprf/Svcxjj9dPjJT7Kvzg30+ONw1FHwX/8Fe++9/vGpU+H669vYd985Nccm2HRTGDEi+1b44os92zfbLPvmu3w5rFrVs33atOx62bKe3xwjeOlnk46ObOuk2ogR2esDLF0Ka9du2D5yZFY/ZFsmnZ3r22644QbmzDmQKVOy+889B+vWbfj80aPXn2SpVvuYMeuHZV6yJFutq40dC5MmZbeffbbnso8bBxMnZs9bsqRn+/jx2bf/det6fiuHrG38+Gy5nn8+W6YDDzzwpfaJE7PXWLs2e3+6mzQpq3HNmuz97W7y5GwZV6/OPp/uNtkke49Wrco+3+6mTMm2bl58sedWA2SfzciR2ZgXK1fWbnfdc91z3duwvSjrXkTcnlLau+cUdvU3dNS+BQvg3HPh05/Ozi9w993ZSlXLhAnZpTcTJ2aX3kyatH5lqKW/8xr0d5bDrhW5N11/COunX7vBc7r+kHrTX3t/+3V0/aHXEtF3+4gRfbePHJm1T5mytuZ0o0b1/fzRo/tuHzOm7/axY7NLb8aN6/s0E+PHZ5feuO713e6613u761736Vtn3etL7nbuG3INGrXvsstg993ha1/LziQIvYe+JEnDxSga4lH7Ojrg3e+Gd74TZs6EO+/MdsaQJKkVlDv4Uxryrv6jj872wv3EJ7I9UnfaachmLUnSRiv3b/zLlmV7n2xk8K9dm32HGD0aPvOZrEu/aj8cSZJaRrm3+Idg1L5HHoHXvx7OPju7//rXG/qSpNZV7uDfiFH7UoILLoBXvxruuw9e9aqhLU2SpEYod/APctS+JUuynfdOPTUbXevuu+H44xtQnyRJQ8zghwF39T/+OMybB5//fDZc5cte1oDaJElqgHLv3Ldo0foRMvqxenU2pvWxx2ZjIT/6aDZmsyRJeVLuLf72dth88yz8+3D//dkJNd7+9vUnjDD0JUl5VO7g72fUvpTgO9+BvfbKht/92c9gzz2HsT5JkoaYXf19/L5/8snwgx/A4YfD+efDVlsNY22SJDVAuYO/vT07qXUvjjwS9t03O/ez4+xLkoqgvMG/bl2P4XpXr4Yzz4Rdd4X3vQ9OOqmJ9UmS1ADl3Y597rnsRMxVXf0//jF84xvZaHySJBVReYO/xqh9Tz2VXX/iE02oR5KkYVDe4K8xal9HB0TAxIlNqkmSpAYz+Ku6+js6YJNNsvCXJKmIyrtzX40t/gMOyE6tK0lSUZU3+NvbYcwYmDr1pYfe+c7sIklSUZW7q3/LLTfo11+1KhutT5Kkoip38Hcbte8Nb8gG7ZEkqajKG/zdBu+BbOe+yZObVI8kScOgvMFf4wQ9XXv1S5JUVOUM/s5OWLy4R1f/0qUGvySp2MoZ/M88k43VX7XF39kJy5cb/JKkYitn8Nc4hr+zEz7+8WwHP0mSiqqcx/HXGLVvzBj47GebVI8kScOknFv8NU7Qs3p19rP/2rVNqkmSpGFQzuCv0dV/xx2w+ebw2982qSZJkoZBeYN/4kSYNOmlhzo6smt37pMkFVl5g7/boXwGv/VtSV0AACAASURBVCSpDMoZ/L2M2gcGvySp2MoZ/DVG7Vu6NLs2+CVJRVbe4O/W1f+618F//qdj9UuSiq18x/GvXg1LlvTY4t933+wiSVKRlW+L/+mns+tuwf/EE7BwYRPqkSRpGJUv+GuM2gfwr/8Khx7ahHokSRpG5Qv+GqP2gafklSSVQ/mCv8aofWDwS5LKobzBX2MAH4NfklR05Qz+KVNg3LgNHjb4JUllUL7D+WqM2gfwuc/B1ls3oR5JkoZR+YK/xqh9ACed1IRaJEkaZuXs6u/2+/7atXDzzfDMM02qSZKkYVK+4K/R1b94Mey/P/zkJ02qSZKkYVKu4F+5MtuLzzPzSZJKqlzB3zV4T41D+cAT9EiSiq+cwe8WvySppMoV/H2M2gcGvySp+Ax+YJ994JJLYMcdm1CTJEnDqFzH8XcF/4wZGzy87bZwwglNqEeSpGFWri3+9naYPh1Gj97g4QcfhLY2SKk5ZUmSNFzKFfy9jNr3ve/BkUdCRBNqkiRpGJUv+LsdygeeoEeSVB7lCv5eTtBj8EuSyqI8wZ9Sr139Br8kqSzKE/zLlsELL9jVL0kqtfIcztfLqH0AX/kKrFs3zPVIktQE5Qn+XgbvgWwAH0mSyqA8Xf19BP9Pfwr33DPM9UiS1ATlC/5uv/GnBMcfDxdf3ISaJEkaZuUJ/vZ2GDkSpk3b4OFVq2DNGpgypUl1SZI0jMoT/IsWweabZ+FfxTPzSZLKpFzB38uhfGDwS5LKoaHBHxFHRMT8iHgoIj5ao31KRFwVEXdFxL0RcWrDiulj1D4w+CVJ5dCw4I+IkcC3gCOB2cAJETG722T/BNyXUno1MAf4r4gY05CCehm1b9YsuOkmOOighryqJEktpZFb/PsCD6WUHkkprQYuBY7pNk0CJkdEAJOAJcDaIa8kpWyLv0ZX/8SJsN9+sNlmQ/6qkiS1nEYG/zbAgqr7CyuPVfsm8ErgSeDPwD+nlIZ+DL3nnst23a+xxf/AA3D++bB8+ZC/qiRJLaeRI/fVOrt96nb/cOBO4I3AK4DfRMT1KaWODWYUcRpwGsCMGTNoa2sbUCETHn2UfYH7lizh6W7P/dnPtubrX5/JlCl/YLPN1gxovkNl+fLlA16mVucy5YPLlA8uUz7kZplSSg25AAcAV1fd/3fg37tN8wvgoKr7vwP27Wu+M2fOTAN2zTUpQUrXXtuj6fOfz5pWrhz4bIfKtTXqyjuXKR9cpnxwmfKhlZYJuC31kqON7Oq/Fdg5Inas7LB3PHBlt2keBw4BiIgtgFnAI0NeSS+j9kG2V/+oUTBu3JC/qiRJLadhXf0ppbURcQZwNTASOC+ldG9EnF5pPwf4DHBBRPyZ7KeBj6SUnhnyYvo4M1/XKXmj1g8TkiQVTEPPzpdSmgfM6/bYOVW3nwTe1MgagGyLf8wYmDq1R1NX8EuSVAblOC1v16h9NTbrv/xlWLq0CTVJktQE5Qj+Xkbtg+z7QI2f/iVJKqRyjNXfy6h9kB3DP3fuMNcjSVKTlCf4e9ms//zn4Qc/GOZ6JElqkuIHf2cnLF7c6xa/O/dJksqk+MH/zDOwbp3BL0kSZQj+rsF7agT/mjXwwgsGvySpPMoT/DV+41+2LLueMmUY65EkqYmKfzhfH6P2TZ2aNY8fP8w1SZLUJMUP/j66+keMgM03H+Z6JElqonJ09U+YAJMm9Wh6+GH4xCfg0UeHvyxJkpqh+MHfx6h98+fDZz+7/tcASZKKrvjB38eofR0d2bV79UuSyqIcwd/LqH0GvySpbIof/H109Xedlc/glySVRbGDf/VqePbZPrv6I2DixGGuS5KkJil28D/9dHbdS1f/pz+dhf+IYr8LkiS9pNiR18fgPZBt7dc4yk+SpMIqdvD3MXgPwHe+A1/+8jDWI0lSk5U6+K+4IrtIklQW5Qj+Pg7nc49+SVKZFDv429uzU++NG1ez2eCXJJVNsYO/j1H7wOCXJJVP8YO/l25+gM5Og1+SVC7FPi1vezvssUevzU89BSkNYz2SJDVZ8bf4++jqh+xYfkmSyqK4wf/CC9mP+L109S9eDCedBDfcMMx1SZLURMUN/n5G7Xv6abjkEnjyyWGsSZKkJitu8PczeI+n5JUklZHBb/BLkkqk+MHfx6h9YPBLksqluMHf9Rv/5pvXbE4JZszIBvaTJKksihv8ixbB9OkwenTN5uOOy3bw2267Ya5LkqQmKnbw9zFqnyRJZVTc4G9v73Pwnu9+F971rmGsR5KkFlDc4O9n1L7bboPf/W4Y65EkqQUUM/hT6rer3zPzSZLKqJjBv3x5NmSvp+SVJGkDxQz+fgbvAYNfklROxTwtbx3Bv8UWvR7iL0lSYRU7+Pv4jf+nPx2mWiRJaiHF7Orv58x8kiSVVTGDf9EiGDkSpk2r2ZwSHHggfP/7w1yXJElNVtzgnzEjC/8aVq6EP/whG7JXkqQyKWbw9zNqn2fmkySVVTGDv59R+wx+SVJZ1RX8ETE+ImY1upghU8eofWDwS5LKp9/gj4ijgTuBX1Xu7xERVza6sEFLqd+u/lGjYL/9PHmfJKl86jmO/yxgX6ANIKV0Z0Ts0LCKNtZzz8GaNX0G/557wk03DWNNkiS1iHq6+temlJY2vJKhUseofZIklVU9wX9PRJwIjIyInSPiG8CNDa5r8OoYte/CC2GPPWBpfr7OSJI0JOoJ/g8AuwKrgB8CHcCHGlnURqlj1L7HH4e77oLx44epJkmSWkS/v/GnlFYCHwc+HhEjgYkppRcbXtlg1XlmvnHjYMyYYapJkqQWUc9e/ZdExCYRMRG4F5gfER9ufGmDtGhRluhTp/Y6iafklSSVVT1d/bNTSh3A24B5wMuAkxta1cZob89+34/odRKDX5JUVvUczjc6IkaTBf83U0prIiI1uK7B62fUPoBXvhImTRqmeiRJaiH1BP93gUeBu4DrImJ7sh38WtOiRbDddn1O8olPDFMtkiS1mH67+lNK/5NS2ialdFTKPAYcPAy1DU4/o/ZJklRm/W7xR8RY4Fhgh27Tf7pBNQ1eZ2d2rt1+gn///eHAA+ErXxmmuiRJahH17Nz3c+AYYC2wourSep55Btat6zf4H3oIXnhhmGqSJKmF1PMb/7YppSMaXslQqGPUvpTcq1+SVF71bPHfGBG7NbySoVDHqH2rVmXn8DH4JUllVM8W/4HAuyPir2TD9gaQUkq7N7Sywahz1D6AKVOGoR5JklpMPcF/ZMOrGCp1dPVHwDvfCbvsMkw1SZLUQuoZq/+xiHg1cFDloetTSnc1tqxBam+HCRP6HJ1nxgy49NJhrEmSpBZSz1j9/wz8ANi8crk4Ij7Q6MIGpWvUvj6G65Ukqczq2bnvPcB+KaVPppQ+CewPvLexZQ3SokV9dvMDXHVVtmPfXa3ZZyFJUkPVE/wBdFbd76w81nrqGLVv6VJYtiz7RUCSpLKpZ+e+84GbI+IKssA/Bvi/hlY1WIsWwetf3+ckXXv1ezifJKmM6tm576sR0UZ2WB/AqSmlPzW0qsFYswaefbbfLX6DX5JUZvV09XcJINGq3fxPP51d9/Mbf0cHjBoF48YNQ02SJLWYevbq/yTwfWBTYDpwfkT8R6MLG7A6Bu8BeM1r4H3vc8d/SVI51fMb/wnAnimlFwEi4gvAHcBnG1nYgNUZ/G9/e3aRJKmM6unqfxSo7hgfCzzckGo2Rh2j9kG2K0BKw1CPJEktqJ4t/lXAvRHxG7Lf+A8DboiI/wFIKX2wgfXVr+sEPf0E/+GHZ2fubWtrfEmSJLWaeoL/isqlS1tjStlIixZlZ94ZP77PyTo6+v01QJKkwqrncL7vd92OiE2B7VJKdze0qsGoY9Q+yIJ/5sxhqEeSpBZUz179bRGxSURsBtxFtlf/Vxtf2gDVMWofZMHvMfySpLKqZ+e+KSmlDuBvgfNTSnsBhza2rEHoOkFPP5YuhcmTh6EeSZJaUD3BPyoitgKOA+Y2uJ7BqyP4U4J/+Rd44xuHqSZJklpMPTv3fRq4GvhDSunWiHg58GBjyxqgF17I+vD7+Y0/Aj73uWGqSZKkFlTPzn0/Bn5cdf8R4NhGFjVgXYfy9bPFv2ZN9v1gypRs2F5Jksqmnp37ZkbENRFxT+X+7i03ZG+do/bdfz9Mnw5XXjkMNUmS1ILq+Y3/f4F/B9YAVA7lO76emUfEERExPyIeioiP1mj/cETcWbncExGdlaMHBqbOUfs8M58kqezqCf4JKaVbuj22tr8nRcRI4FvAkcBs4ISImF09TUrpyymlPVJKe5B9ufh9SmlJfaVXqbOr3+CXJJVdPcH/TES8gmy4XiLi7cBTdTxvX+ChlNIjKaXVwKXAMX1MfwLwwzrm21PXFv/mm/c5mcEvSSq7enZx+yfgXGCXiHgC+CtwUh3P2wZYUHV/IbBfrQkjYgJwBHBGHfPtadEimDYNRo/uczKDX5JUdn0Gf0SMAPZOKR0aEROBESmlZXXOu9YZ73s7L97RZIcL1uzmj4jTgNMAZsyYQVu3M+zseu+9TJg8mVv7PfPOJP7+76dxzz0L+Mtf1vUz7fBZvnx5j2XKO5cpH1ymfHCZ8iE3y5RS6vMCXNffNL087wDg6qr7/w78ey/TXgGcWM98Z86cmXo44ICUDjmk5+M5ce211za7hCHnMuWDy5QPLlM+tNIyAbelXnK0nt/4fxMRZ0bEdhGxWdeljufdCuwcETtGxBiyIwF6HEgXEVOANwA/r2OetdU5XG97OzzxxKBfRZKk3KvnN/6/r1z/U9VjCXh5X09KKa2NiDPIRv0bCZyXUro3Ik6vtJ9TmfRvgF+nlFYMqPL1L5Qleh1n5vvoR+Gaa+Dxxwf1SpIk5V49I/ft2Fd7RByWUvpNL8+dB8zr9tg53e5fAFzQXx29Wr4cVq70zHySJNWhnq7+/nxxCOYxeHWO2gcGvyRJQxH8tfbeHz51jtoHBr8kSUMR/L0dojc86hy1Dwx+SZLyf466AXT1f/KT2Tg/kiSV1VAE/6NDMI/BW7QIRoyoK9FPOGEY6pEkqYXVFfwR8Vpgh+rpU0oXVq7/tiGV1au9PRujf+TIPidbtw7uuAN22CE7Na8kSWXU72/8EXER8BXgQGCfymXvBtdVvzoH71m2DPbZBy66aBhqkiSpRdWzxb83MLsyBGDrqTP4PUGPJEn17dV/D9B/sjZLnaP2GfySJNW3xT8duC8ibgFWdT2YUnprw6qqV0pu8UuSNAD1BP9ZjS5i0J57DtasMfglSapTPWP1/344ChmUAYzat9tu8IMfwMyZDa5JkqQWVs9e/ftHxK0RsTwiVkdEZ0R0DEdx/RrAqH1bbw0nnugAPpKkcqtn575vAicADwLjgX+oPNZ8Axi175FH4LrroLOzwTVJktTC6hqrP6X0EDAypdSZUjofmNPQquo1gK7+Cy+EN7yhwfVIktTi6tm5b2VEjAHujIgvAU8BExtbVp3a22H0aNh0034nXbYMJk3qd4A/SZIKrZ4t/pMr050BrAC2A45tZFF16zqUL/o/M7Bn5pMkqb69+h+LiPHAVimls4ehpvrVeQw/GPySJEF9e/UfDdwJ/Kpyf4+IuLLRhdWlzlH7wOCXJAnqH8BnX6ANIKV0Z0Ts0LCKBmLRIti7vvMFfeYz8OKLDa5HkqQWV0/wr00pLY06fkcfVp2d8PTTdXf11/n9QJKkQqvrJD0RcSIwMiJ2johvADc2uK7+PfMMrFtXd1f/VVfBn//c4JokSWpx9QT/B4BdyU7QcwmwFPjnRhZVlwGM2gdw8snwve81sB5JknKgnuCfXbmMAsYBxwC3NrKougxg1L6U3LlPkiSo7zf+HwBnAvcA6xpbzgAMYNS+FSuy8Df4JUllV0/wL04pXdXwSgZqAF39npJXkqRMPcH/qYj4HnAN2e/8AKSUftqwquqxaBFMmJCNw9sPg1+SpEw9wX8qsAswmvVd/QlofvBvsUVdw/W+7GVw442w007DUJckSS2snuB/dUppt4ZXMlDt7XXv0T9hAhxwQIPrkSQpB+rZq/+miJjd8EoGagDj9D/8MFxwASxd2tiSJElqdfUE/4Fkp+SdHxF3R8SfI+LuRhfWrwEE/w03wKmnwpIlDa5JkqQWV09X/xENr2Kg1qyBZ5+te9S+ri19d+6TJJVdXaflHY5CBuTpp7PrAZySF2Dy5AbVI0lSTtTT1d96BjBqH2TBP24cjBnTwJokScqBfAd/nV39HR0wZUoD65EkKSfyGfwDPEHP2WfDtdc2sB5JknKinp37Ws8At/i32KLuSSVJKrR8bvEvWpTtoj9+fF2TX3wx/PznDa5JkqQcyGfwD2DUPoCvfAXOO6+B9UiSlBP5DP4BDN4D2c59HsMvSZLBL0lSqeQz+Nvb695bLyWDX5KkLrkL/kgpG4O3zi3+VauyEX4NfkmScng4X6xdm92oM/jHjs1+GRg3roFFSZKUE/kL/s7O7EadXf0RHsMvSVKX/HX1D3CLf8EC+NSn4OGHG1iUJEk5Ufjgf/hh+PSnsy8AkiSVXf6Cv6urf/PN65q+65S87twnSVIOg3/E2rUwbRqMHl3X9Aa/JEnr5S74o7NzwIP3AEye3KCCJEnKkfwF/9q1gwp+t/glScpj8Hd2Duj4vH/7tyz8PY5fkqQcHsc/YoBb/CNG2M0vSVKX3G3xk9KAgv+88+ALX2hgPZIk5Uj+gh8G1NX/85/DpZc2sBZJknIkn8HvKXklSRoUg1+SpBLJZ/APoKvf4Jckab18Bv/06XVPumaNwS9JUpfcHc63dtIkGDmy7ukffTQ7EECSJOVwi/+Frbce8HMiGlCIJEk5lLvgH4iODjj5ZLj22mZXIklSayh08D/7LFx8MTz2WLMrkSSpNRQ6+D1BjyRJGzL4JUkqEYNfkqQSKXTwd3bCtGkwZUqzK5EkqTXk7jj+gXjrW+GZZ5pdhSRJraPQW/ySJGlDhQ7+738fjjuu2VVIktQ6Ch38d9wBv/lNs6uQJKl1FDr4PTOfJEkbMvglSSoRg1+SpBIp9OF806bBVls1uwpJklpHoYP/0kubXYEkSa2l0F39kiRpQ4UO/sMOg3PPbXYVkiS1jsIGf2cn/Pa38OSTza5EkqTWUdjgX748u/YEPZIkrVfY4PeUvJIk9WTwS5JUIg0N/og4IiLmR8RDEfHRXqaZExF3RsS9EfH7oXtt2Htv2HLLoZqjJEn517Dj+CNiJPAt4DBgIXBrRFyZUrqvapqpwLeBI1JKj0fE5kP1+rNnw623DtXcJEkqhkZu8e8LPJRSeiSltBq4FDim2zQnAj9NKT0OkFJ6uoH1SJJUeo0M/m2ABVX3F1YeqzYT2DQi2iLi9og4Zahe/Mc/hte8Bp72q4QkSS9p5JC9UeOxVOP19wIOAcYDf4yIm1JKf9lgRhGnAacBzJgxg7a2tn5f/Nprt+VPf9qJW265nkmTOgdR/vBZvnx5XcuUJy5TPrhM+eAy5UNelqmRwb8Q2K7q/rZA9+F0FgLPpJRWACsi4jrg1cAGwZ9SOhc4F2DWrFlpzpw5/b5413t/5JEHMXLkYMofPm1tbdSzTHniMuWDy5QPLlM+5GWZGtnVfyuwc0TsGBFjgOOBK7tN83PgoIgYFRETgP2A+4fixTs6YNIkWj70JUkaTg3b4k8prY2IM4CrgZHAeSmleyPi9Er7OSml+yPiV8DdwDrgeymle4bi9Ts6PIZfkqTuGnpa3pTSPGBet8fO6Xb/y8CXh/q1d94ZVq0a6rlKkpRvDQ3+ZvrIR5pdgSRJraewQ/ZKkqSeChv8hxwCH/hAs6uQJKm1FDb4H3po/al5JUlSprDBv3Spe/VLktRdIYM/JQ/nkySplkIG/4oVWfgb/JIkbaiQwd/ZCccdB7vu2uxKJElqLYU8jn/KFPjRj5pdhSRJraeQW/ySJKm2Qgb/734Hm24KN9/c7EokSWothQz+55/PLuPGNbsSSZJaSyGDv6Mju3avfkmSNmTwS5JUIoUM/qVLs+vJk5tbhyRJraaQwb/77vCe98CYMc2uRJKk1lLI4/iPOSa7SJKkDRVyi7+zs9kVSJLUmgoZ/MccA/vv3+wqJElqPYUM/o4OmDCh2VVIktR6Chv8HsonSVJPBr8kSSVi8EuSVCKFPJzv/e+HvfdudhWSJLWeQgb/Zz7T7AokSWpNhevq7+yE557zWH5JkmopXPA/8ghsthn88IfNrkSSpNZTuOD3zHySJPXO4JckqUQMfkmSSqRwwb9sWXZt8EuS1FPhgn+33eDss2HzzZtdiSRJradwx/G/+tXZRZIk9VS4Lf7Fi+HJJ5tdhSRJralwwX/WWbD77s2uQpKk1lS44PcEPZIk9c7glySpRAx+SZJKxOCXJKlECnc437/9G0yc2OwqJElqTYUL/ne+s9kVSJLUugrX1X/HHdmx/JIkqadCBf/q1bDXXvDd7za7EkmSWlOhgr/rBD1TpjS3DkmSWlWhgt9T8kqS1DeDX5KkEjH4JUkqkUIF/8yZcNFF8KpXNbsSSZJaU6GO499iC3jXu5pdhSRJratQW/yPPw7XXw9r1jS7EkmSWlOhgv/SS+H1r4dVq5pdiSRJralQwd/RASNGOFa/JEm9KVzwb7IJRDS7EkmSWlMhg1+SJNVm8EuSVCKFOpzv4x9fP4iPJEnqqVDBv9deza5AkqTWVqiu/l/9Cu66q9lVSJLUugoV/O95D3zzm82uQpKk1lWo4F+61J37JEnqS2GCv7MTVqww+CVJ6kthgn/Zsuza4JckqXeFCf6uw/gMfkmSeleYw/k23xxuuAFe/vJmVyJJUusqTPCPGweve12zq5AkqbUVpqv/scfgwgthyZJmVyJJUusqTPDffDP83d/BU081uxJJklpXYYLfnfskSeqfwS9JUokULvgnTWpuHZIktbJCBf+kSTByZLMrkSSpdRUm+D/yEbjxxmZXIUlSayvMcfxbbJFdJElS7wqzxf/jH8MVVzS7CkmSWlthtvi/9rXsN/6/+ZtmVyJJUusqzBZ/R4eH8kmS1B+DX5KkEilU8E+e3OwqJElqbYUI/nXr3OKXJKkehdi5LwKeeALGjm12JZIktbbCBP9WWzW7CkmSWl8huvrb2+Hss2H+/GZXIklSaytE8D/2GJx1Fjz8cLMrkSSptTU0+CPiiIiYHxEPRcRHa7TPiYilEXFn5fLJwbyOp+SVJKk+DfuNPyJGAt8CDgMWArdGxJUppfu6TXp9SuktG/NaBr8kSfVp5Bb/vsBDKaVHUkqrgUuBYxrxQga/JEn1aWTwbwMsqLq/sPJYdwdExF0R8cuI2HUwL2TwS5JUn0gpNWbGEe8ADk8p/UPl/snAvimlD1RNswmwLqW0PCKOAr6eUtq5xrxOA04DmDFjxl6XXXbZBu2dnfDCC6OYOHEtEQ1ZnIZavnw5kyZNanYZQ8plygeXKR9cpnxopWU6+OCDb08p7V2rrZHH8S8Etqu6vy3wZPUEKaWOqtvzIuLbETE9pfRMt+nOBc4FmDVrVpozZ07Dim6GtrY2XKbW5zLlg8uUDy5T8zSyq/9WYOeI2DEixgDHA1dWTxARW0Zk2+gRsW+lnmcH+kKXXAKf/ewQVCxJUsE1LPhTSmuBM4CrgfuBy1JK90bE6RFxemWytwP3RMRdwP8Ax6dB/PZw1VVw4YVDVbkkScXV0CF7U0rzgHndHjun6vY3gW9u7Ot4gh5JkupTiJH7DH5JkupTiOBftszglySpHoUI/hdfNPglSapHIU7L+8AD2bH8kiSpb4XY4gcYObLZFUiS1PpyH/wvvgh/93fwm980uxJJklpf7oN/6dLsGP4HH2x2JZIktb7cB78n6JEkqX4GvyRJJWLwS5JUIrkP/jVrYNNNYcqUZlciSVLry/1x/G96EyxZ0uwqJEnKh9xv8UuSpPrlPvh/9CN4xztg7dpmVyJJUuvLffDfeSf8/OeO3CdJUj1yH/wdHdmOfRHNrkSSpNZXiOD3UD5Jkupj8EuSVCK5P5xv001h9OhmVyFJUj7kPvgvuKDZFUiSlB+57+qXJEn1y33wv/Wt8M1vNrsKSZLyIffB/+tfw+OPN7sKSZLyIdfBv2pVdnGvfkmS6pPr4F+2LLs2+CVJqk+ug7+jI7v2lLySJNUn18G/bh3stRdsvXWzK5EkKR9yfRz/TjvBbbc1uwpJkvIj11v8kiRpYHId/L/4Bey9NyxY0OxKJEnKh1wH/8KFcPvtMCrXP1hIkjR8ch38XXv1ezifJEn1yXXwL10KI0bAhAnNrkSSpHzIdfB3dGRb+xHNrkSSpHzIdfDvuCMcckizq5AkKT9yHfz/8i9w+eXNrkKSpPzIdfBLkqSByXXwH300nHZas6uQJCk/cn0E/IMPuke/JEkDkest/q69+iVJUn0MfkmSSiS3wd/ZCStWGPySJA1EboN/9Wp4xztgt92aXYkkSfmR2537xo+Hyy5rdhWSJOVLbrf4JUnSwOU2+G+6CTbbDH7/+2ZXIklSfuQ2+J9/Hp57DsaObXYlkiTlR26Dv6Mju3avfkmS6pf74J88ubl1SJKUJ7kPfrf4JUmqX26Df9YsOPlkmDSp2ZVIkpQfuT2O/81vzi6SJKl+ud3iX7eu2RVIkpQ/uQ3+44+HPfZodhWSJOVLboO/owPGjWt2FZIk5Uuug989+iVJGhiDX5KkEjH4JUkqkdweznfqqbDrrs2uQpKkfMlt8J99drMrkCQpf3LZ1b9uXXZ2vs7OZlciSVK+5DL4n3oKNt0U/u//ml2JJEn5ksvg9wQ9kiQNjsEvSVKJGPySJJWIwS9JUonkMvhnzYJPfQq22abZlUiSlC+5PI7/Va/KLpIkaWByucX/7LPZIX2SJGlgchn8X/gCvOIVza5CkqT8yWXwd3TAlCnNrkKSpPzJbfC7R78kSQNn8EuSVCIGvyRJJZLLw/k++EEYObLZVUiSAmN/JQAADMtJREFUlD+5DP53vKPZFUiSlE+57Oq/+25YvLjZVUiSlD+5DP699oKvfa3ZVUiSlD+5C/6UgrVr3blPkqTByF3wr1uXXRv8kiQNXA6DPwBH7pMkaTByF/ydnVnwu8UvSdLA5S74R49ex/e/D3vu2exKJEnKn4YGf0QcERHzI+KhiPhoH9PtExGdEfH2/uY5cmTilFNg222HtlZJksqgYcEfESOBbwFHArOBEyJidi/TfRG4up75rl0b3HgjrF49lNVKklQOjdzi3xd4KKX0SEppNXApcEyN6T4A/AR4up6ZLl8+ite9DpYuHbpCJUkqi0gpNWbGWbf9ESmlf6jcPxnYL6V0RtU02wCXAG8E/g+Ym1K6vMa8TgNOA5g4cae9Vqx4kKuv/j1jxjSm9uG2fPlyJk2a1OwyhpTLlA8uUz64TPnQSst08MEH355S2rtWWyPH6o8aj3VP6v8GPpJS6oyoNXnlSSmdC5wLMG3a7mnNGnjTm94wZIU2W1tbG3PmzGl2GUPKZcoHlykfXKZ8yMsyNTL4FwLbVd3fFniy2zR7A5dWQn86cFRErE0p/ay3mXZ2hofySZI0SI0M/luBnSNiR+AJ4HjgxOoJUko7dt2OiAvIuvp7DX3IRu4z+CVJGpyGBX9KaW1EnEG2t/5I4LyU0r0RcXql/ZzBzHezzdZwzqCeKUmSGrnFT0ppHjCv22M1Yzul9O565jl2bCeHHbbxtUmSVEa5G7lv5cqR3Hlns6uQJCmfchf87e3j+PKXm12FJEn5lLvgX7fOvfolSRosg1+SpBLJXfCn9P/bu/fYq+s6juPP1wC5KmqoU2GA5jTmDFHJ28xLM1HnZV2E5eYla3ZZqaOm2Vz4R2axcs5pGZotDVMzdWiJGaRLU1FA8UKKIJAgmAqY89KPd398Pr/47rfzuwHx/X34vh7b2fmez/mec94v+P3O+3wvv/Pxn/OZmZltruIaP7jxm5mZba7iGv+oUe9x5pl1V2FmZlam4hr/4MFtjBxZdxVmZmZlKq7xr18/gLVr667CzMysTMU1/tWrB7FsWd1VmJmZlam4xg8+uc/MzGxzufGbmZk1iBu/mZlZgxTZ+IcMqbsCMzOzMhXX+EePfg+p7irMzMzKVFzjHziwre4SzMzMilVc4zczM7PN58ZvZmbWIG78ZmZmDeLGb2Zm1iBu/GZmZg3ixm9mZtYgbvxmZmYN4sZvZmbWIG78ZmZmDeLGb2Zm1iBu/GZmZg3ixm9mZtYgbvxmZmYN4sZvZmbWIG78ZmZmDeLGb2Zm1iBu/GZmZg3ixm9mZtYgbvxmZmYN4sZvZmbWIG78ZmZmDeLGb2Zm1iBu/GZmZg3ixm9mZtYgbvxmZmYN4sZvZmbWIG78ZmZmDeLGb2Zm1iBu/GZmZg3ixm9mZtYgbvxmZmYN4sZvZmbWIG78ZmZmDaKIqLuGXpG0AVhcdx1b2QjgzbqL2MqcqQzOVAZnKkNfyjQ6InZrdUf/bV3JVrA4Ig6tu4itSdI8Z+r7nKkMzlQGZ6qPd/WbmZk1iBu/mZlZg5TY+G+su4D/A2cqgzOVwZnK4Ew1Ke7kPjMzM9t8JW7xm5mZ2WYqqvFLOknSYkmvSLq07np6StLNktZIWlQZ21XSQ5Jezte7VO67LGdcLOmz9VTdOUmjJM2R9KKk5yV9O4+XnGmQpCclLcyZpuXxYjO1k9RP0nxJs/LtojNJWibpOUkLJM3LY6Vn2lnSXZJeyr9XR5ScSdL++f+n/bJe0kUlZwKQdHF+f1gkaWZ+3ygvU0QUcQH6AUuAfYAdgIXAuLrr6mHtxwATgEWVsR8Dl+blS4Gr8/K4nG0gMDZn7ld3hg559gQm5OUdgX/kukvOJGBYXh4APAEcXnKmSrZLgN8Cs0r/2ct1LgNGdBgrPdOvgQvy8g7AzqVnqmTrB6wGRpecCdgbWAoMzrfvAM4tMVNJW/wTgVci4tWI+BC4HTi95pp6JCIeAd7qMHw66ZedfH1GZfz2iPggIpYCr5Cy9xkRsSoinsnLG4AXSb8UJWeKiHg33xyQL0HBmQAkjQROAWZUhovO1IliM0naibRxcBNARHwYEe9QcKYOTgCWRMRrlJ+pPzBYUn9gCPA6BWYqqfHvDayo3F6Zx0q1R0SsgtRIgd3zeFE5JY0BDiZtIRedKe8SXwCsAR6KiOIzAdcA3wU2VsZKzxTAbElPS/pqHis50z7AWuBX+ZDMDElDKTtT1WRgZl4uNlNE/BOYDiwHVgHrImI2BWYqqfGrxdj2+CcJxeSUNAz4PXBRRKzvatUWY30uU0S0RcR4YCQwUdKBXaze5zNJOhVYExFP9/QhLcb6VKbsqIiYAEwCviHpmC7WLSFTf9KhwBsi4mDg36Rdxp0pIRMAknYATgPu7G7VFmN9KlM+dn86abf9XsBQSWd39ZAWY30iU0mNfyUwqnJ7JGk3S6nekLQnQL5ek8eLyClpAKnp3xYRd+fhojO1y7tZ5wInUXamo4DTJC0jHRo7XtKtlJ2JiHg9X68B/kDafVpyppXAyryHCeAu0geBkjO1mwQ8ExFv5NslZ/oMsDQi1kbER8DdwJEUmKmkxv8UsJ+ksflT5GTgvppr2hL3Aefk5XOAeyvjkyUNlDQW2A94sob6OiVJpOORL0bETyt3lZxpN0k75+XBpF/ylyg4U0RcFhEjI2IM6fflLxFxNgVnkjRU0o7ty8CJwCIKzhQRq4EVkvbPQycAL1BwpoopbNrND2VnWg4cLmlIfg88gXR+U3mZ6j67sDcX4GTSGeRLgMvrrqcXdc8kHRP6iPQp8MvAx4CHgZfz9a6V9S/PGRcDk+quv0Weo0m7rJ4FFuTLyYVnOgiYnzMtAq7I48Vm6pDvWDad1V9sJtLx8IX58nz7+0DJmXKN44F5+efvHmCX7SDTEOBfwPDKWOmZppE2CBYBvyGdsV9cJn9zn5mZWYOUtKvfzMzMtpAbv5mZWYO48ZuZmTWIG7+ZmVmDuPGbmZk1iBu/2TYiaYwqMzT2YP0fSJrazToDJf05z4B2VhfrnSvput7Uu73Js8MNqbsOs7q58ZuV7WBgQESMj4jf1V1MnZR09Z52Eelvy3vznP22rCqzvseN32zb6ifpl3lO79mSBkvaV9Kf8qQzj0o6oOODJM2VdI2kx/Jc4BMl7Q7cCozPW/z7Ks1VPyI/5lBJc1s81y2Srs3P9aqkz1fu+46kpyQ9K2laHhsq6X5JC/Nrn5XHfyTphbzu9K6eW9KxkmZVXuc6Sefm5WWSfijpcUnzJE2Q9KCkJZIu7Ka2MUrz118PPAOMknRDfp7nK+t9i/T96nMkzcljUyQ9lzNdXXmddyVdKekJ4IjN+D8269P6112AWcPsB0yJiK9IugP4HHAecGFEvCzpU8D1wPEtHjs0Io5UmpTm5og4UNIFwNSIOBUgfZNoj+xJ+gbGA0hfLXqXpBNzfRNJE4zcl19rN+D1iDglv8ZwSbsCZwIHRES0f91xZ8/dg3pWRMQRkn4G3EKaZ2AQ6dv5ft5FbcuB/YHzIuLrub7LI+KtvLX+sKSDIuJaSZcAx0XEm5L2Aq4GDgHeJs32d0ZE3AMMBRZFxBU9/cc0K4kbv9m2tTQiFuTlp4ExpIk+7qw07YGdPHYmQEQ8ImmnDs22t+6JiI3AC5L2yGMn5sv8fHsYqdk+CkzPW8WzIuJRpfnI3wdmSLofmNXNc3enfd6N54BhEbEB2CDp/Zyzs9qWA69FxN8rz/VFpel6+5M+hIwjfRVu1WHA3IhYCyDpNuAY0tfltpEmoDLbLrnxm21bH1SW24A9gHciTQfcnY7fr93q+7b/w6ZDeIN6WIcq11dFxC86rizpENJ8DFdJmh0RV0qaSJqoZDLwTTbtpWj13NW6WtXW/piNHR6/kfQ+1bI2SWNI09i23x4LTAUOi4i3Jd3S4rWqdbXyfkS0dXG/WdF8jN+sXuuBpZK+AP87Qe2Tnazbfmz9aGBdRKxrsc4y0u5rSIcReuNB4HxJw/Lr7C1p97xb/L2IuBWYDkzI6wyPiAdIJ81198HlNWCc0l8hDCd9YNji2lqstxPpg8C6vLdhUuW+DcCOefkJ4NOSRuRDAlOAv/ayJrMieYvfrH5fAm6Q9H1gAHA7afa5jt6W9BipuZ3fyXNNA26S9D1Sc+uxiJgt6RPA4/mww7vA2cDHgZ9I2kiaYfJrpAZ6r6RBpK3ni7t57hX5nIZnSbOYze9q/V7U1tZhvYWS5pPODXgV+Fvl7huBP0paFRHHSboMmJPrfyAi7sWsATw7n1kB8tn5UyNiXt21mFnZvKvfzMysQbzFb2Zm1iDe4jczM2sQN34zM7MGceM3MzNrEDd+MzOzBnHjNzMzaxA3fjMzswb5L+Ciz8+J3wRsAAAAAElFTkSuQmCC\n","text/plain":"<Figure size 504x720 with 1 Axes>"},"metadata":{"needs_background":"light"},"output_type":"display_data"}]},{"metadata":{"trusted":false},"cell_type":"code","source":"mlflow.end_run()","execution_count":21,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# There is room for improvement \n## Let's now Tokenize words in the Review\n### But first, let's start our new run"},{"metadata":{"trusted":false},"cell_type":"code","source":"import pandas as pd\n# Start new mlflow run\nmlflow.start_run(run_name='review_tokenizer')\nmlflow.lp('source data table', f'{get_user()}.AMAZON_REVIEWS')\n# Get common stop words from H2O\ndata_path = \"https://splice-demo.s3.amazonaws.com/stop_words.csv\"\nSTOP_WORDS = pd.read_csv(data_path, header=0)\nSTOP_WORDS = list(STOP_WORDS['STOP_WORD'])\nprint(STOP_WORDS)","execution_count":22,"outputs":[{"name":"stdout","output_type":"stream","text":"['all', 'just', 'being', 'over', 'both', 'through', 'yourselves', 'its', 'before', 'o', 'hadn', 'herself', 'll', 'had', 'should', 'to', 'only', 'won', 'under', 'ours', 'has', 'do', 'them', 'his', 'very', 'they', 'not', 'during', 'now', 'him', 'nor', 'd', 'did', 'didn', 'this', 'she', 'each', 'further', 'where', 'few', 'because', 'doing', 'some', 'hasn', 'are', 'our', 'ourselves', 'out', 'what', 'for', 'while', 're', 'does', 'above', 'between', 'mustn', 't', 'be', 'we', 'who', 'were', 'here', 'shouldn', 'hers', 'by', 'on', 'about', 'couldn', 'of', 'against', 's', 'isn', 'or', 'own', 'into', 'yourself', 'down', 'mightn', 'wasn', 'your', 'from', 'her', 'their', 'aren', 'there', 'been', 'whom', 'too', 'wouldn', 'themselves', 'weren', 'was', 'until', 'more', 'himself', 'that', 'but', 'don', 'with', 'than', 'those', 'he', 'me', 'myself', 'ma', 'these', 'up', 'will', 'below', 'ain', 'can', 'theirs', 'my', 'and', 've', 'then', 'is', 'am', 'it', 'doesn', 'an', 'as', 'itself', 'at', 'have', 'in', 'any', 'if', 'again', 'no', 'when', 'same', 'how', 'other', 'which', 'you', 'shan', 'needn', 'haven', 'after', 'most', 'such', 'why', 'a', 'off', 'i', 'm', 'yours', 'so', 'y', 'the', 'having', 'once']\n"}]},{"metadata":{"trusted":false},"cell_type":"code","source":"# Inspect our reviews before tokenization\nreviews['Text']","execution_count":23,"outputs":[{"data":{"text/html":"<table>\n<thead>\n<tr><th>Text </th></tr>\n</thead>\n<tbody>\n<tr><td>These are made in China (do not buy ANY pet food from China). Dogswell has been using propylene glycol to soften their treats (what are they thinkng?). Do not purchase or support this company in any way until they clean up their act. And for whatever reason Amazon doesn't allow returns of this item, so I had to toss mine out. Bad business all around on this one. </td></tr>\n<tr><td>Everything is ok, except it just isn't as good as it is in the bags. Just considerably more bland -- less lemon and less zing. Boring. </td></tr>\n<tr><td>Best cat treat ever. There isn't anything comparable to the love my cat has for these treats, he snubs away any other kind now.<br />I know he likes to manipulate me with his cattiness but these treats are my way of manipulating him to come sit on my lap and have some chill time. :) </td></tr>\n<tr><td>My two Corgis were thoroughly spoiled by my late husband (I spent a year and a half dieting them down a combined total of 25 pounds!)<br /><br />They are accustomed to the finest of fare, and they absolutely love the Wellness brand of treats. </td></tr>\n<tr><td>We used to have drive down to the specialty pet food store for this product. So glad we discovered Amazon. As far as I can tell it is no more expensive & in some cases less - Prime membership is awesome. Loving Pets treats are some of the best according to my dog. They do not develop that nasty smell like some dog treats do. </td></tr>\n<tr><td>I really liked this coffee, it was just as good as everyone claimed it was. Strong, bold and flavorful! I would recommend! </td></tr>\n<tr><td>I am always looking for healthier, whole grain versions of foods I enjoy. Unfortunately, these Peacock brand noodles are yet another food masquerading as healthy. The product title in big letters on the front says "Brown Rice Vermicelli", making the consumer think "this is made with brown rice, so it should be a healthy choice". But the first indication that it is not is when looking at the fiber content on the nutrition facts - only 0.6g per 2oz serving. Then onto the ingredients list to see why so low... contains brown rice, sago starch and water. The sago starch comes from palms and must not have much (if any) fiber.<br /><br />The Annie Chun's Maifun Brown Rice Noodles (sold on Amazon and in my local healthy grocer) has become one of my staples and is my frame of reference when comparing to the Peacock brand. The Annie Chun's product is made with 100% whole grain, with ingredients brown rice flour and water. Per 2oz serving, it has 4g fiber and pretty much the same calories and other nutrients as the Peacock brand.<br /><br />If you do try this Peacock brand noodles and have not used rice noodles before, you will need to seek guidance elsewhere on preparation. As others have pointed out, the Peacock package gives almost no directions on how to prepare the product, aside from a brief mention in the recipes (in the header text it does say that they are "easy-to-cook" but does not say how). It also contains a very strange recipe for rice noodles: Aglio Olio style - this is an Italian recipe for noodles with olive oil/garlic/sprinkled with grated cheese that I think would not be very tasty. The second recipe appears to be for a soup with veggie strips. Neither recipe gives amounts or much direction. In comparison, the Annie Chun's package gives clear, specific directions on rice noodle preparation and two recipes.<br /><br />I use rice noodles = maifun = rice sticks = sometimes called vermicelli for making the Vietnamese salad "bun tofu", to serve with stir-fried veggies or in lettuce rolls. They can also be used in spring rolls/egg rolls. When cooking with thin rice noodles, be careful not to oversoak/overcook/overmix or they tend to disintegrate. Asian rice noodle vermicelli (maifun) are not the same as Italian vermicelli and are not readily interchangeable. If making an Italian recipe, the best results would be expected from Italian pasta and not maifun.<br /><br />A few final notes... Both Peacock and Annie Chun's brown rice noodles are gluten free. The Peacock is made in Singapore and the Annie Chun's in Thailand. The Peacock noodles do taste fine (kind of bland), but so do the Annie Chun's. At this time, they are both approximately the same price. Peacock come in an plastic bag with some noodle crushage upon shipping; Annie Chun's are perfect upon removal from their cellophane bag in a box. Overall, I highly recommend the Annie Chun's Maifun as a healthier option over the Peacock brand. On a related note, the Annie Chun's soba and brown rice pad thai noodles are also excellent.<br /><br />Rating for this product: 2.5 stars rounded down to 2 stars.</td></tr>\n<tr><td>I have tried many different drink mix, this is the best tasting by far. It does not have the after taste of the sweetener and I really like it, it is pretty strong, so I use a big water bottle (20 oz) for one tube, it still a little stronger than I like, but it is just my taste. </td></tr>\n<tr><td>This is my favorite item to order for my Keurig. There are so many flavors, my finicky palate never gets bored! The only downside is there are probably 5-6 decaf varieties. I don't drink decaf (I REQUIRE copious amounts of caffeine), so they sit on the shelf... </td></tr>\n<tr><td>Tastes like Earl Grey, but it's green tea so it's healthier. </td></tr>\n</tbody>\n</table>"},"metadata":{},"output_type":"display_data"},{"data":{"text/plain":""},"execution_count":23,"metadata":{},"output_type":"execute_result"}]},{"metadata":{},"cell_type":"markdown","source":"## Now we can train our Doc2Vec model\n<blockquote>We are going to use the popular Gensim doc2vec implementation for scikit-learn. We use scikit-learn here because of it's implementation of <code>Pipelines</code> which allow us to create custom transformations on the data before training/running our model. This gives us the ultimate flexibility. We will use doc2vec, which is just an extension of word2vec but for full documents (sentences). We can also time how long it takes to train the word vectorizer, and log the vector size so we can change it and see how performance changes. Then we can look at some word synonyms to see how well the tokenizer did</i></br><footer>Splice Machine</footer></blockquote><br>"},{"metadata":{"trusted":false},"cell_type":"code","source":"def tokenize( reviews ):\n review_tokens = []\n for review in reviews[0]:\n # Remove non-letters\n review = re.sub(\"[^a-zA-Z]\",\" \", review)\n review = review.lower().split()\n\n stops = set(STOP_WORDS)\n review = [w for w in review if w not in stops]\n review_tokens.append(review)\n return(review_tokens)","execution_count":24,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Train Doc2Vec Model\n<blockquote>This will take a few minutes to run as the model needs to generate mappings of every sentence to vectors</i></br><footer>Splice Machine</footer></blockquote><br>"},{"metadata":{"scrolled":true,"trusted":false},"cell_type":"code","source":"from sklearn.feature_extraction.text import TfidfTransformer, CountVectorizer\nfrom sklearn.pipeline import FeatureUnion, Pipeline as skPipe\nfrom sklearn.base import BaseEstimator, TransformerMixin\nfrom sklearn.preprocessing import FunctionTransformer\nfrom gensim.sklearn_api import W2VTransformer, D2VTransformer\nimport pandas as pd\n\npdf = reviews['Text'].as_data_frame().astype('string')\n\nword2vec_model = skPipe(verbose=True,\n steps = [\n ('preprocessor', FunctionTransformer(tokenize, validate=False)),\n ('word2vec', D2VTransformer()),\n ('postprocessor', FunctionTransformer(lambda X: X.astype('double'), validate=True))\n ])\nwith mlflow.timer('word2vec_train_time'):\n # tokenize and build vocab\n word2vec_model.fit([pdf.dropna()['Text']])","execution_count":25,"outputs":[{"name":"stderr","output_type":"stream","text":"The usage of `cmp` is deprecated and will be removed on or after 2021-06-01. Please use `eq` and `order` instead.\n`scipy.sparse.sparsetools` is deprecated!\nscipy.sparse.sparsetools is a private module for scipy.sparse, and should not be used.\n"},{"name":"stdout","output_type":"stream","text":"Starting Code Block word2vec_train_time... [Pipeline] ...... (step 1 of 3) Processing preprocessor, total= 4.4s\n[Pipeline] .......... (step 2 of 3) Processing word2vec, total= 2.3min\n"},{"name":"stderr","output_type":"stream","text":"The default validate=True will be replaced by validate=False in 0.22.\n"},{"name":"stdout","output_type":"stream","text":"[Pipeline] ..... (step 3 of 3) Processing postprocessor, total= 0.0s\nDone.\nCode Block word2vec_train_time:\nRan in 140.889 secs\nRan in 2.348 mins\n"}]},{"metadata":{},"cell_type":"markdown","source":"## Now we can use the Doc2Vec Model to see the most similar sentences of an input"},{"metadata":{"trusted":false},"cell_type":"code","source":"from copy import deepcopy\ndoc2vec_model = deepcopy(word2vec_model.steps[1][1].gensim_model)\n\n# Tokenize our input\ninp = \"Tastes like Earl Grey\"\ntokens = tokenize([[inp]])\nnew_vector = doc2vec_model.infer_vector(tokens[0])\n# Get our vectorized sentence\nsims = doc2vec_model.docvecs.most_similar([new_vector])\nprint(f'Vector of input: {sims}\\n')\n# Get the most similar review\nindex = sims[0][0]\noutput = pdf.dropna()['Text'].iloc[index]\n\nprint(f'Input: {inp}\\nMost similar Output: {output}')","execution_count":27,"outputs":[{"name":"stdout","output_type":"stream","text":"Vector of input: [(93226, 0.8579040765762329), (1883, 0.8531466722488403), (23673, 0.8494805693626404), (87876, 0.848807156085968), (72010, 0.8470171689987183), (21202, 0.8461479544639587), (42383, 0.846132755279541), (26856, 0.8412899971008301), (28805, 0.8412365913391113), (26706, 0.8408089280128479)]\n\nInput: Tastes like Earl Grey\nMost similar Output: You have to taste this, it taste like you made it from scratch.\n"}]},{"metadata":{},"cell_type":"markdown","source":"## Now we can save this model and end our run\n<blockquote>We want to save this vectorizer as an <b>independent</b> <code>run</code>. This is because we may want to build more than 1 model that utilizes these word vectors. We don't want to duplicate those identical word vectors, so we can use the outputs for <b>more than one model</b>. This is the idea of a <i>feature store</i> where we use features from one dataset on multiple models. This is crucial to creating efficient ML workflow systems."},{"metadata":{"trusted":false},"cell_type":"code","source":"mlflow.log_model(word2vec_model, 'word2vec_model')\nmlflow.lp('vector_size',100)\nmlflow.end_run()","execution_count":28,"outputs":[{"name":"stdout","output_type":"stream","text":"Saving artifact of size: 56999.04 KB to Splice Machine DB\n"}]},{"metadata":{},"cell_type":"markdown","source":"## Let's vectorize our reviews\n<blockquote>Now that we have a word embedding for each word in our vocabulary, we will aggregate the words for each review using the <code>transform</code> function. This will give us one aggregated word embedding for each review.</blockquote>"},{"metadata":{"trusted":false},"cell_type":"code","source":"# Calculate a vector for each review\nreview_vecs = h2o.H2OFrame(word2vec_model.transform([pdf.fillna(\"\")['Text']]))\n# Add the review vectors to the original dataframe\n# Add aggregated word embeddings \next_reviews = reviews.cbind(review_vecs)\next_reviews.head()","execution_count":29,"outputs":[{"name":"stderr","output_type":"stream","text":"The default validate=True will be replaced by validate=False in 0.22.\n"},{"name":"stdout","output_type":"stream","text":"Parse progress: |█████████████████████████████████████████████████████████| 100%\n"},{"data":{"text/html":"<table>\n<thead>\n<tr><th>ProductId </th><th>UserId </th><th>Summary </th><th style=\"text-align: right;\"> Score</th><th style=\"text-align: right;\"> HelpfulnessDenominator</th><th style=\"text-align: right;\"> Id</th><th>ProfileName </th><th style=\"text-align: right;\"> HelpfulnessNumerator</th><th style=\"text-align: right;\"> Time</th><th>Text </th><th style=\"text-align: right;\"> PositiveReview</th><th style=\"text-align: right;\"> C1</th><th style=\"text-align: right;\"> C2</th><th style=\"text-align: right;\"> C3</th><th style=\"text-align: right;\"> C4</th><th style=\"text-align: right;\"> C5</th><th style=\"text-align: right;\"> C6</th><th style=\"text-align: right;\"> C7</th><th style=\"text-align: right;\"> C8</th><th style=\"text-align: right;\"> C9</th><th style=\"text-align: right;\"> C10</th><th style=\"text-align: right;\"> C11</th><th style=\"text-align: right;\"> C12</th><th style=\"text-align: right;\"> C13</th><th style=\"text-align: right;\"> C14</th><th style=\"text-align: right;\"> C15</th><th style=\"text-align: right;\"> C16</th><th style=\"text-align: right;\"> C17</th><th style=\"text-align: right;\"> C18</th><th style=\"text-align: right;\"> C19</th><th style=\"text-align: right;\"> C20</th><th style=\"text-align: right;\"> C21</th><th style=\"text-align: right;\"> C22</th><th style=\"text-align: right;\"> C23</th><th style=\"text-align: right;\"> C24</th><th style=\"text-align: right;\"> C25</th><th style=\"text-align: right;\"> C26</th><th style=\"text-align: right;\"> C27</th><th style=\"text-align: right;\"> C28</th><th style=\"text-align: right;\"> C29</th><th style=\"text-align: right;\"> C30</th><th style=\"text-align: right;\"> C31</th><th style=\"text-align: right;\"> C32</th><th style=\"text-align: right;\"> C33</th><th style=\"text-align: right;\"> C34</th><th style=\"text-align: right;\"> C35</th><th style=\"text-align: right;\"> C36</th><th style=\"text-align: right;\"> C37</th><th style=\"text-align: right;\"> C38</th><th style=\"text-align: right;\"> C39</th><th style=\"text-align: right;\"> C40</th><th style=\"text-align: right;\"> C41</th><th style=\"text-align: right;\"> C42</th><th style=\"text-align: right;\"> C43</th><th style=\"text-align: right;\"> C44</th><th style=\"text-align: right;\"> C45</th><th style=\"text-align: right;\"> C46</th><th style=\"text-align: right;\"> C47</th><th style=\"text-align: right;\"> C48</th><th style=\"text-align: right;\"> C49</th><th style=\"text-align: right;\"> C50</th><th style=\"text-align: right;\"> C51</th><th style=\"text-align: right;\"> C52</th><th style=\"text-align: right;\"> C53</th><th style=\"text-align: right;\"> C54</th><th style=\"text-align: right;\"> C55</th><th style=\"text-align: right;\"> C56</th><th style=\"text-align: right;\"> C57</th><th style=\"text-align: right;\"> C58</th><th style=\"text-align: right;\"> C59</th><th style=\"text-align: right;\"> C60</th><th style=\"text-align: right;\"> C61</th><th style=\"text-align: right;\"> C62</th><th style=\"text-align: right;\"> C63</th><th style=\"text-align: right;\"> C64</th><th style=\"text-align: right;\"> C65</th><th style=\"text-align: right;\"> C66</th><th style=\"text-align: right;\"> C67</th><th style=\"text-align: right;\"> C68</th><th style=\"text-align: right;\"> C69</th><th style=\"text-align: right;\"> C70</th><th style=\"text-align: right;\"> C71</th><th style=\"text-align: right;\"> C72</th><th style=\"text-align: right;\"> C73</th><th style=\"text-align: right;\"> C74</th><th style=\"text-align: right;\"> C75</th><th style=\"text-align: right;\"> C76</th><th style=\"text-align: right;\"> C77</th><th style=\"text-align: right;\"> C78</th><th style=\"text-align: right;\"> C79</th><th style=\"text-align: right;\"> C80</th><th style=\"text-align: right;\"> C81</th><th style=\"text-align: right;\"> C82</th><th style=\"text-align: right;\"> C83</th><th style=\"text-align: right;\"> C84</th><th style=\"text-align: right;\"> C85</th><th style=\"text-align: right;\"> C86</th><th style=\"text-align: right;\"> C87</th><th style=\"text-align: right;\"> C88</th><th style=\"text-align: right;\"> C89</th><th style=\"text-align: right;\"> C90</th><th style=\"text-align: right;\"> C91</th><th style=\"text-align: right;\"> C92</th><th style=\"text-align: right;\"> C93</th><th style=\"text-align: right;\"> C94</th><th style=\"text-align: right;\"> C95</th><th style=\"text-align: right;\"> C96</th><th style=\"text-align: right;\"> C97</th><th style=\"text-align: right;\"> C98</th><th style=\"text-align: right;\"> C99</th><th style=\"text-align: right;\"> C100</th></tr>\n</thead>\n<tbody>\n<tr><td>B00141QYSQ </td><td>A1YS02UZZGRDCT</td><td>Do Not Buy </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 41471</td><td>Evan Eberhardt </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\">1.34836e+09</td><td>These are made in China (do not buy ANY pet food from China). Dogswell has been using propylene glycol to soften their treats (what are they thinkng?). Do not purchase or support this company in any way until they clean up their act. And for whatever reason Amazon doesn't allow returns of this item, so I had to toss mine out. Bad business all around on this one. </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 0.0311221 </td><td style=\"text-align: right;\"> 0.0719161 </td><td style=\"text-align: right;\">-0.1281 </td><td style=\"text-align: right;\"> 0.0273494 </td><td style=\"text-align: right;\">-0.0642016 </td><td style=\"text-align: right;\">0.0994345 </td><td style=\"text-align: right;\"> 0.0135567 </td><td style=\"text-align: right;\">-0.0737052 </td><td style=\"text-align: right;\"> 0.122077 </td><td style=\"text-align: right;\"> 0.0350201 </td><td style=\"text-align: right;\"> 0.0756314 </td><td style=\"text-align: right;\"> 0.0923928</td><td style=\"text-align: right;\">-0.132129 </td><td style=\"text-align: right;\"> 0.0115625 </td><td style=\"text-align: right;\"> 0.0306404 </td><td style=\"text-align: right;\">-0.0373009 </td><td style=\"text-align: right;\">-0.0684029 </td><td style=\"text-align: right;\"> 0.0680291 </td><td style=\"text-align: right;\">-0.023424 </td><td style=\"text-align: right;\">-0.0835139 </td><td style=\"text-align: right;\"> 0.00894095</td><td style=\"text-align: right;\"> 0.0749535</td><td style=\"text-align: right;\"> 0.129895 </td><td style=\"text-align: right;\"> 0.132389 </td><td style=\"text-align: right;\"> 0.0092737 </td><td style=\"text-align: right;\"> 0.108408 </td><td style=\"text-align: right;\"> 0.0476859 </td><td style=\"text-align: right;\">-0.0485954 </td><td style=\"text-align: right;\">-0.121283 </td><td style=\"text-align: right;\"> 0.0735992 </td><td style=\"text-align: right;\">-0.0445242 </td><td style=\"text-align: right;\">-0.0080538 </td><td style=\"text-align: right;\">-0.095095 </td><td style=\"text-align: right;\">-0.0605023 </td><td style=\"text-align: right;\">-0.00707685</td><td style=\"text-align: right;\"> 0.0376109 </td><td style=\"text-align: right;\"> 0.0138715 </td><td style=\"text-align: right;\"> 0.145876 </td><td style=\"text-align: right;\">-0.0366801 </td><td style=\"text-align: right;\"> 0.0694044</td><td style=\"text-align: right;\">-0.0392646 </td><td style=\"text-align: right;\">-0.0860089</td><td style=\"text-align: right;\"> 0.0646098</td><td style=\"text-align: right;\">-0.0416264 </td><td style=\"text-align: right;\"> 0.0500176 </td><td style=\"text-align: right;\">-0.0586521</td><td style=\"text-align: right;\"> 0.0536527 </td><td style=\"text-align: right;\"> 0.0441543 </td><td style=\"text-align: right;\">-0.0336521 </td><td style=\"text-align: right;\">-0.021085 </td><td style=\"text-align: right;\"> 0.0222311</td><td style=\"text-align: right;\"> 0.119254 </td><td style=\"text-align: right;\">-0.0199307 </td><td style=\"text-align: right;\">-0.0894037 </td><td style=\"text-align: right;\">-0.0437724 </td><td style=\"text-align: right;\"> 0.0452685 </td><td style=\"text-align: right;\"> 0.101266 </td><td style=\"text-align: right;\">-0.0958144</td><td style=\"text-align: right;\">-0.0453501 </td><td style=\"text-align: right;\">-0.127768 </td><td style=\"text-align: right;\"> 0.0535161</td><td style=\"text-align: right;\">-0.0619117 </td><td style=\"text-align: right;\">-0.0233732 </td><td style=\"text-align: right;\">-0.089195 </td><td style=\"text-align: right;\">0.112413 </td><td style=\"text-align: right;\">-0.0247127 </td><td style=\"text-align: right;\"> 0.0762887 </td><td style=\"text-align: right;\">-0.101294 </td><td style=\"text-align: right;\"> 0.0329748 </td><td style=\"text-align: right;\"> 0.0639789 </td><td style=\"text-align: right;\">0.118965 </td><td style=\"text-align: right;\">-0.0389742 </td><td style=\"text-align: right;\">-0.0120773 </td><td style=\"text-align: right;\">-0.0522605 </td><td style=\"text-align: right;\">-0.0395143 </td><td style=\"text-align: right;\">-0.107343 </td><td style=\"text-align: right;\"> 0.000138754</td><td style=\"text-align: right;\">-0.202054 </td><td style=\"text-align: right;\">-0.0322408 </td><td style=\"text-align: right;\"> 0.0204981</td><td style=\"text-align: right;\"> 0.0339285 </td><td style=\"text-align: right;\">-0.0551402</td><td style=\"text-align: right;\"> 0.0426714 </td><td style=\"text-align: right;\">-0.0427338 </td><td style=\"text-align: right;\"> 0.107102 </td><td style=\"text-align: right;\"> 0.0563949 </td><td style=\"text-align: right;\">-0.041101 </td><td style=\"text-align: right;\">-0.0848916</td><td style=\"text-align: right;\"> 0.0479946 </td><td style=\"text-align: right;\">-0.0431166 </td><td style=\"text-align: right;\">-0.00818688</td><td style=\"text-align: right;\">-0.00537665</td><td style=\"text-align: right;\"> 0.0358918</td><td style=\"text-align: right;\">-0.0373974 </td><td style=\"text-align: right;\">-0.102815 </td><td style=\"text-align: right;\"> 0.0306642</td><td style=\"text-align: right;\"> 0.0515081</td><td style=\"text-align: right;\"> 0.120599 </td><td style=\"text-align: right;\"> 0.0901687 </td><td style=\"text-align: right;\"> 0.115301 </td></tr>\n<tr><td>B0089SPEO2 </td><td>A3JOYNYL458QHP</td><td>Less lemon and less zing </td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 28582</td><td>coleridge </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.32391e+09</td><td>Everything is ok, except it just isn't as good as it is in the bags. Just considerably more bland -- less lemon and less zing. Boring. </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">-0.0354048 </td><td style=\"text-align: right;\"> 0.0247745 </td><td style=\"text-align: right;\">-0.0545446 </td><td style=\"text-align: right;\"> 0.0766969 </td><td style=\"text-align: right;\">-0.0542439 </td><td style=\"text-align: right;\">0.00712698</td><td style=\"text-align: right;\"> 0.0679792 </td><td style=\"text-align: right;\"> 0.00729536 </td><td style=\"text-align: right;\"> 0.0194643</td><td style=\"text-align: right;\">-0.000942943</td><td style=\"text-align: right;\"> 0.0149142 </td><td style=\"text-align: right;\"> 0.0336201</td><td style=\"text-align: right;\">-0.0262167</td><td style=\"text-align: right;\"> 0.00263732 </td><td style=\"text-align: right;\">-0.00473881</td><td style=\"text-align: right;\"> 0.0025277 </td><td style=\"text-align: right;\"> 0.0319848 </td><td style=\"text-align: right;\"> 0.0724836 </td><td style=\"text-align: right;\">-0.0159643</td><td style=\"text-align: right;\">-0.0369099 </td><td style=\"text-align: right;\"> 0.0349859 </td><td style=\"text-align: right;\">-0.0440495</td><td style=\"text-align: right;\">-0.0209405 </td><td style=\"text-align: right;\"> 0.00411849</td><td style=\"text-align: right;\"> 0.00909809</td><td style=\"text-align: right;\">-0.0124248 </td><td style=\"text-align: right;\"> 0.0167648 </td><td style=\"text-align: right;\">-0.021823 </td><td style=\"text-align: right;\"> 0.00381749</td><td style=\"text-align: right;\">-0.0172089 </td><td style=\"text-align: right;\">-0.00290564</td><td style=\"text-align: right;\">-0.0122623 </td><td style=\"text-align: right;\"> 0.00216576</td><td style=\"text-align: right;\"> 0.0540102 </td><td style=\"text-align: right;\">-0.0128816 </td><td style=\"text-align: right;\">-0.0172007 </td><td style=\"text-align: right;\"> 0.00682313</td><td style=\"text-align: right;\"> 0.0245506</td><td style=\"text-align: right;\">-0.0577924 </td><td style=\"text-align: right;\">-0.0198054</td><td style=\"text-align: right;\">-0.0630254 </td><td style=\"text-align: right;\">-0.0439116</td><td style=\"text-align: right;\"> 0.0178368</td><td style=\"text-align: right;\"> 0.0637651 </td><td style=\"text-align: right;\">-0.0437091 </td><td style=\"text-align: right;\">-0.0649058</td><td style=\"text-align: right;\">-0.0402683 </td><td style=\"text-align: right;\">-0.0244872 </td><td style=\"text-align: right;\"> 0.083092 </td><td style=\"text-align: right;\"> 0.0160841 </td><td style=\"text-align: right;\">-0.0193159</td><td style=\"text-align: right;\">-0.0250807</td><td style=\"text-align: right;\"> 0.00872545</td><td style=\"text-align: right;\">-0.00508996</td><td style=\"text-align: right;\"> 0.0469757 </td><td style=\"text-align: right;\">-0.0459333 </td><td style=\"text-align: right;\">-0.0293787</td><td style=\"text-align: right;\">-0.0118928</td><td style=\"text-align: right;\">-0.0353911 </td><td style=\"text-align: right;\"> 0.00960084</td><td style=\"text-align: right;\"> 0.0197541</td><td style=\"text-align: right;\">-0.00492643</td><td style=\"text-align: right;\">-0.0258128 </td><td style=\"text-align: right;\"> 0.0134544 </td><td style=\"text-align: right;\">0.0435747 </td><td style=\"text-align: right;\"> 0.0105644 </td><td style=\"text-align: right;\"> 0.0530692 </td><td style=\"text-align: right;\">-0.000385565</td><td style=\"text-align: right;\">-0.0257902 </td><td style=\"text-align: right;\"> 0.0121527 </td><td style=\"text-align: right;\">0.00295611</td><td style=\"text-align: right;\">-0.0238126 </td><td style=\"text-align: right;\"> 0.00596643</td><td style=\"text-align: right;\">-0.0372999 </td><td style=\"text-align: right;\"> 0.0326834 </td><td style=\"text-align: right;\">-0.0418227 </td><td style=\"text-align: right;\">-0.0677815 </td><td style=\"text-align: right;\"> 0.00903367</td><td style=\"text-align: right;\">-0.0430189 </td><td style=\"text-align: right;\">-0.0240547</td><td style=\"text-align: right;\">-0.00625893</td><td style=\"text-align: right;\">-0.0332857</td><td style=\"text-align: right;\">-0.00795872</td><td style=\"text-align: right;\">-0.00244623</td><td style=\"text-align: right;\">-0.0130634</td><td style=\"text-align: right;\"> 0.0290811 </td><td style=\"text-align: right;\">-0.00679918</td><td style=\"text-align: right;\"> 0.0219198</td><td style=\"text-align: right;\">-0.0891907 </td><td style=\"text-align: right;\">-0.0157458 </td><td style=\"text-align: right;\"> 0.0173031 </td><td style=\"text-align: right;\"> 0.0519746 </td><td style=\"text-align: right;\">-0.0118705</td><td style=\"text-align: right;\">-0.0143847 </td><td style=\"text-align: right;\"> 0.09397 </td><td style=\"text-align: right;\"> 0.0110028</td><td style=\"text-align: right;\"> 0.0328517</td><td style=\"text-align: right;\"> 0.0368655</td><td style=\"text-align: right;\"> 0.0645739 </td><td style=\"text-align: right;\"> 0.047565 </td></tr>\n<tr><td>B001PMCDK2 </td><td>A14TTMM0Z03Y2W</td><td>my cat goes crazy for these! </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">389965</td><td>Lindsay S. Bradford </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.3106e+09 </td><td>Best cat treat ever. There isn't anything comparable to the love my cat has for these treats, he snubs away any other kind now.<br />I know he likes to manipulate me with his cattiness but these treats are my way of manipulating him to come sit on my lap and have some chill time. :) </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\"> 0.0722773 </td><td style=\"text-align: right;\"> 0.0639396 </td><td style=\"text-align: right;\">-0.0335094 </td><td style=\"text-align: right;\"> 0.0664469 </td><td style=\"text-align: right;\">-0.0652916 </td><td style=\"text-align: right;\">0.0689586 </td><td style=\"text-align: right;\"> 0.00527908</td><td style=\"text-align: right;\"> 0.0413024 </td><td style=\"text-align: right;\"> 0.0791953</td><td style=\"text-align: right;\"> 0.0910244 </td><td style=\"text-align: right;\"> 0.0555998 </td><td style=\"text-align: right;\"> 0.136567 </td><td style=\"text-align: right;\">-0.0604036</td><td style=\"text-align: right;\"> 0.00816555 </td><td style=\"text-align: right;\">-0.0229452 </td><td style=\"text-align: right;\">-0.0184804 </td><td style=\"text-align: right;\">-0.156753 </td><td style=\"text-align: right;\"> 0.0496772 </td><td style=\"text-align: right;\">-0.0695818</td><td style=\"text-align: right;\"> 0.0520854 </td><td style=\"text-align: right;\"> 0.0916832 </td><td style=\"text-align: right;\"> 0.0356442</td><td style=\"text-align: right;\">-0.0426297 </td><td style=\"text-align: right;\"> 0.180441 </td><td style=\"text-align: right;\"> 0.0691552 </td><td style=\"text-align: right;\"> 0.0166221 </td><td style=\"text-align: right;\">-0.0345218 </td><td style=\"text-align: right;\">-0.0499622 </td><td style=\"text-align: right;\"> 0.00111568</td><td style=\"text-align: right;\"> 0.0759182 </td><td style=\"text-align: right;\">-0.023927 </td><td style=\"text-align: right;\">-0.07914 </td><td style=\"text-align: right;\">-0.0675944 </td><td style=\"text-align: right;\">-0.0155206 </td><td style=\"text-align: right;\"> 0.0650796 </td><td style=\"text-align: right;\"> 0.0760519 </td><td style=\"text-align: right;\"> 0.0215851 </td><td style=\"text-align: right;\"> 0.0226041</td><td style=\"text-align: right;\">-0.0835985 </td><td style=\"text-align: right;\"> 0.0163129</td><td style=\"text-align: right;\"> 0.0167953 </td><td style=\"text-align: right;\">-0.0885574</td><td style=\"text-align: right;\">-0.0367466</td><td style=\"text-align: right;\">-0.0578619 </td><td style=\"text-align: right;\"> 0.0156162 </td><td style=\"text-align: right;\">-0.0248519</td><td style=\"text-align: right;\"> 0.0831028 </td><td style=\"text-align: right;\"> 0.00309607</td><td style=\"text-align: right;\">-0.00960189</td><td style=\"text-align: right;\"> 0.051891 </td><td style=\"text-align: right;\">-0.0101117</td><td style=\"text-align: right;\"> 0.0763692</td><td style=\"text-align: right;\"> 0.0693945 </td><td style=\"text-align: right;\">-0.0432502 </td><td style=\"text-align: right;\">-0.0587365 </td><td style=\"text-align: right;\"> 0.0761859 </td><td style=\"text-align: right;\">-0.0310571</td><td style=\"text-align: right;\">-0.108232 </td><td style=\"text-align: right;\">-0.105073 </td><td style=\"text-align: right;\">-0.0249751 </td><td style=\"text-align: right;\"> 0.0113496</td><td style=\"text-align: right;\">-0.0585231 </td><td style=\"text-align: right;\">-0.0157676 </td><td style=\"text-align: right;\">-0.0404807 </td><td style=\"text-align: right;\">0.0492526 </td><td style=\"text-align: right;\">-0.138522 </td><td style=\"text-align: right;\"> 0.00527175</td><td style=\"text-align: right;\"> 0.0012283 </td><td style=\"text-align: right;\"> 0.0671212 </td><td style=\"text-align: right;\"> 0.00836717 </td><td style=\"text-align: right;\">0.143647 </td><td style=\"text-align: right;\">-0.010005 </td><td style=\"text-align: right;\">-0.0288803 </td><td style=\"text-align: right;\">-0.0533496 </td><td style=\"text-align: right;\"> 0.0189405 </td><td style=\"text-align: right;\">-0.05666 </td><td style=\"text-align: right;\">-0.128763 </td><td style=\"text-align: right;\">-0.06002 </td><td style=\"text-align: right;\">-0.0298549 </td><td style=\"text-align: right;\"> 0.0422105</td><td style=\"text-align: right;\">-0.0321987 </td><td style=\"text-align: right;\">-0.107224 </td><td style=\"text-align: right;\"> 0.0706086 </td><td style=\"text-align: right;\"> 0.0965514 </td><td style=\"text-align: right;\"> 0.0849076</td><td style=\"text-align: right;\"> 0.0487253 </td><td style=\"text-align: right;\">-0.0208525 </td><td style=\"text-align: right;\">-0.0516878</td><td style=\"text-align: right;\"> 0.0413375 </td><td style=\"text-align: right;\">-0.119434 </td><td style=\"text-align: right;\">-0.0490209 </td><td style=\"text-align: right;\">-0.0426325 </td><td style=\"text-align: right;\">-0.141985 </td><td style=\"text-align: right;\">-0.0731703 </td><td style=\"text-align: right;\"> 0.00255836</td><td style=\"text-align: right;\">-0.0352337</td><td style=\"text-align: right;\"> 0.0615106</td><td style=\"text-align: right;\"> 0.130422 </td><td style=\"text-align: right;\"> 0.132418 </td><td style=\"text-align: right;\"> 0.0860906</td></tr>\n<tr><td>B002Q8JOSI </td><td>A17UQD2RSSQH5X</td><td>My dogs tell me these treats are YUMMY</td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">212536</td><td>in the dark </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.31613e+09</td><td>My two Corgis were thoroughly spoiled by my late husband (I spent a year and a half dieting them down a combined total of 25 pounds!)<br /><br />They are accustomed to the finest of fare, and they absolutely love the Wellness brand of treats. </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\"> 0.00664075</td><td style=\"text-align: right;\"> 0.0294306 </td><td style=\"text-align: right;\">-0.00529865 </td><td style=\"text-align: right;\"> 0.0971298 </td><td style=\"text-align: right;\">-0.0554535 </td><td style=\"text-align: right;\">0.041721 </td><td style=\"text-align: right;\"> 0.0711783 </td><td style=\"text-align: right;\"> 0.000611028</td><td style=\"text-align: right;\"> 0.0224197</td><td style=\"text-align: right;\"> 0.0377034 </td><td style=\"text-align: right;\">-0.0358093 </td><td style=\"text-align: right;\"> 0.033314 </td><td style=\"text-align: right;\">-0.0671549</td><td style=\"text-align: right;\">-0.033238 </td><td style=\"text-align: right;\"> 0.0135984 </td><td style=\"text-align: right;\"> 0.0541221 </td><td style=\"text-align: right;\">-0.0676616 </td><td style=\"text-align: right;\"> 0.00970966</td><td style=\"text-align: right;\"> 0.0540937</td><td style=\"text-align: right;\"> 0.00182744</td><td style=\"text-align: right;\"> 0.0213856 </td><td style=\"text-align: right;\"> 0.0461527</td><td style=\"text-align: right;\">-0.00813494</td><td style=\"text-align: right;\"> 0.0593336 </td><td style=\"text-align: right;\">-0.0452856 </td><td style=\"text-align: right;\">-0.0446647 </td><td style=\"text-align: right;\">-0.00545645 </td><td style=\"text-align: right;\">-0.0844449 </td><td style=\"text-align: right;\">-0.012087 </td><td style=\"text-align: right;\"> 0.01034 </td><td style=\"text-align: right;\"> 0.0414165 </td><td style=\"text-align: right;\"> 0.000379321</td><td style=\"text-align: right;\">-0.0102823 </td><td style=\"text-align: right;\">-0.048552 </td><td style=\"text-align: right;\">-0.0478385 </td><td style=\"text-align: right;\"> 0.0280377 </td><td style=\"text-align: right;\"> 0.0373936 </td><td style=\"text-align: right;\"> 0.0682869</td><td style=\"text-align: right;\"> 0.00163835 </td><td style=\"text-align: right;\">-0.0111299</td><td style=\"text-align: right;\">-0.0219148 </td><td style=\"text-align: right;\">-0.0924591</td><td style=\"text-align: right;\"> 0.0605706</td><td style=\"text-align: right;\">-0.0393773 </td><td style=\"text-align: right;\"> 0.00369886</td><td style=\"text-align: right;\">-0.0124289</td><td style=\"text-align: right;\">-0.0573726 </td><td style=\"text-align: right;\">-0.0531469 </td><td style=\"text-align: right;\"> 0.0102651 </td><td style=\"text-align: right;\"> 0.0505138 </td><td style=\"text-align: right;\">-0.0210899</td><td style=\"text-align: right;\"> 0.0418745</td><td style=\"text-align: right;\">-0.00207173</td><td style=\"text-align: right;\">-0.0217494 </td><td style=\"text-align: right;\">-0.0760661 </td><td style=\"text-align: right;\">-0.00915289</td><td style=\"text-align: right;\"> 0.0539563</td><td style=\"text-align: right;\">-0.0754855</td><td style=\"text-align: right;\">-0.0416121 </td><td style=\"text-align: right;\">-0.0607258 </td><td style=\"text-align: right;\">-0.0297508</td><td style=\"text-align: right;\">-0.0212294 </td><td style=\"text-align: right;\">-0.0355835 </td><td style=\"text-align: right;\"> 0.0276151 </td><td style=\"text-align: right;\">0.0650853 </td><td style=\"text-align: right;\">-0.0233705 </td><td style=\"text-align: right;\">-0.0188454 </td><td style=\"text-align: right;\"> 0.0202355 </td><td style=\"text-align: right;\">-0.0565332 </td><td style=\"text-align: right;\">-0.00200412 </td><td style=\"text-align: right;\">0.0732944 </td><td style=\"text-align: right;\">-0.0131517 </td><td style=\"text-align: right;\">-0.0254612 </td><td style=\"text-align: right;\">-0.059395 </td><td style=\"text-align: right;\"> 0.0546296 </td><td style=\"text-align: right;\"> 0.00453196</td><td style=\"text-align: right;\">-0.0866346 </td><td style=\"text-align: right;\">-0.0316054 </td><td style=\"text-align: right;\"> 0.00321747</td><td style=\"text-align: right;\"> 0.10967 </td><td style=\"text-align: right;\">-0.0123553 </td><td style=\"text-align: right;\">-0.0371342</td><td style=\"text-align: right;\"> 0.0453067 </td><td style=\"text-align: right;\">-0.0184933 </td><td style=\"text-align: right;\"> 0.0944639</td><td style=\"text-align: right;\">-0.0483836 </td><td style=\"text-align: right;\">-0.0253078 </td><td style=\"text-align: right;\"> 0.0512179</td><td style=\"text-align: right;\">-0.0190948 </td><td style=\"text-align: right;\">-0.0654936 </td><td style=\"text-align: right;\"> 0.0123237 </td><td style=\"text-align: right;\">-0.0115953 </td><td style=\"text-align: right;\">-0.0616764</td><td style=\"text-align: right;\">-0.0494044 </td><td style=\"text-align: right;\"> 0.0845929 </td><td style=\"text-align: right;\">-0.0297328</td><td style=\"text-align: right;\">-0.0123154</td><td style=\"text-align: right;\"> 0.0593596</td><td style=\"text-align: right;\">-0.0221123 </td><td style=\"text-align: right;\"> 0.0738102</td></tr>\n<tr><td>B00176G870 </td><td>A2F2MZW8EOGH5J</td><td>Yummy to the tummy </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">115971</td><td>daemoncycler "When you arrive at a fork in th...</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.33479e+09</td><td>We used to have drive down to the specialty pet food store for this product. So glad we discovered Amazon. As far as I can tell it is no more expensive & in some cases less - Prime membership is awesome. Loving Pets treats are some of the best according to my dog. They do not develop that nasty smell like some dog treats do. </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">-0.0569507 </td><td style=\"text-align: right;\"> 0.100785 </td><td style=\"text-align: right;\">-0.139421 </td><td style=\"text-align: right;\"> 0.0916591 </td><td style=\"text-align: right;\">-0.0669327 </td><td style=\"text-align: right;\">0.139871 </td><td style=\"text-align: right;\">-0.053762 </td><td style=\"text-align: right;\">-0.0516505 </td><td style=\"text-align: right;\"> 0.118875 </td><td style=\"text-align: right;\"> 0.0320348 </td><td style=\"text-align: right;\"> 0.072005 </td><td style=\"text-align: right;\"> 0.183638 </td><td style=\"text-align: right;\">-0.0507139</td><td style=\"text-align: right;\"> 0.0662295 </td><td style=\"text-align: right;\">-0.0175711 </td><td style=\"text-align: right;\">-0.0563984 </td><td style=\"text-align: right;\">-0.0718676 </td><td style=\"text-align: right;\"> 0.0765699 </td><td style=\"text-align: right;\"> 0.0315671</td><td style=\"text-align: right;\">-0.0192526 </td><td style=\"text-align: right;\"> 0.0754111 </td><td style=\"text-align: right;\"> 0.0841884</td><td style=\"text-align: right;\"> 0.0744908 </td><td style=\"text-align: right;\"> 0.0465411 </td><td style=\"text-align: right;\">-0.0424741 </td><td style=\"text-align: right;\"> 0.0661015 </td><td style=\"text-align: right;\"> 0.0406489 </td><td style=\"text-align: right;\">-0.18337 </td><td style=\"text-align: right;\">-0.0585701 </td><td style=\"text-align: right;\"> 0.0298698 </td><td style=\"text-align: right;\">-0.0111099 </td><td style=\"text-align: right;\"> 0.0202008 </td><td style=\"text-align: right;\">-0.0343549 </td><td style=\"text-align: right;\">-0.0418548 </td><td style=\"text-align: right;\">-0.0324968 </td><td style=\"text-align: right;\"> 0.00647369</td><td style=\"text-align: right;\">-0.0611532 </td><td style=\"text-align: right;\"> 0.0551729</td><td style=\"text-align: right;\">-0.00687854 </td><td style=\"text-align: right;\">-0.0248682</td><td style=\"text-align: right;\">-0.0540005 </td><td style=\"text-align: right;\">-0.198181 </td><td style=\"text-align: right;\"> 0.0737756</td><td style=\"text-align: right;\">-0.0429527 </td><td style=\"text-align: right;\"> 0.107321 </td><td style=\"text-align: right;\">-0.0898639</td><td style=\"text-align: right;\"> 0.0765675 </td><td style=\"text-align: right;\">-0.0155799 </td><td style=\"text-align: right;\"> 0.0183806 </td><td style=\"text-align: right;\"> 0.127784 </td><td style=\"text-align: right;\">-0.0959386</td><td style=\"text-align: right;\"> 0.116064 </td><td style=\"text-align: right;\">-0.0179106 </td><td style=\"text-align: right;\"> 0.0426698 </td><td style=\"text-align: right;\">-0.033683 </td><td style=\"text-align: right;\"> 0.0445723 </td><td style=\"text-align: right;\"> 0.008436 </td><td style=\"text-align: right;\">-0.149899 </td><td style=\"text-align: right;\"> 0.0567 </td><td style=\"text-align: right;\">-0.061969 </td><td style=\"text-align: right;\"> 0.106395 </td><td style=\"text-align: right;\">-0.0474444 </td><td style=\"text-align: right;\">-0.0159554 </td><td style=\"text-align: right;\">-0.0205738 </td><td style=\"text-align: right;\">0.0895157 </td><td style=\"text-align: right;\">-0.061288 </td><td style=\"text-align: right;\"> 0.0101447 </td><td style=\"text-align: right;\">-0.0686496 </td><td style=\"text-align: right;\"> 0.112418 </td><td style=\"text-align: right;\">-0.0104366 </td><td style=\"text-align: right;\">0.0889506 </td><td style=\"text-align: right;\">-0.0832996 </td><td style=\"text-align: right;\">-0.0569302 </td><td style=\"text-align: right;\">-0.0880074 </td><td style=\"text-align: right;\"> 0.0605617 </td><td style=\"text-align: right;\">-0.0295004 </td><td style=\"text-align: right;\">-0.0154352 </td><td style=\"text-align: right;\">-0.157382 </td><td style=\"text-align: right;\">-0.0701636 </td><td style=\"text-align: right;\"> 0.0672711</td><td style=\"text-align: right;\"> 0.0324169 </td><td style=\"text-align: right;\">-0.06646 </td><td style=\"text-align: right;\">-0.0100355 </td><td style=\"text-align: right;\">-0.0226492 </td><td style=\"text-align: right;\"> 0.159023 </td><td style=\"text-align: right;\"> 0.0748435 </td><td style=\"text-align: right;\">-0.061932 </td><td style=\"text-align: right;\">-0.0430855</td><td style=\"text-align: right;\"> 0.00506526</td><td style=\"text-align: right;\">-0.0945003 </td><td style=\"text-align: right;\">-0.00753332</td><td style=\"text-align: right;\"> 0.0870276 </td><td style=\"text-align: right;\">-0.0298816</td><td style=\"text-align: right;\">-0.00772067</td><td style=\"text-align: right;\">-0.0255571 </td><td style=\"text-align: right;\">-0.0150707</td><td style=\"text-align: right;\"> 0.0882468</td><td style=\"text-align: right;\"> 0.07531 </td><td style=\"text-align: right;\"> 0.044203 </td><td style=\"text-align: right;\"> 0.144223 </td></tr>\n<tr><td>B001CHFUGY </td><td>A2M8VROSDPU4JT</td><td>Very good coffee </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">434484</td><td>Officefan "Officefankt" </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.27725e+09</td><td>I really liked this coffee, it was just as good as everyone claimed it was. Strong, bold and flavorful! I would recommend! </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">-0.0640295 </td><td style=\"text-align: right;\"> 0.0737147 </td><td style=\"text-align: right;\">-0.000756723</td><td style=\"text-align: right;\"> 0.0261864 </td><td style=\"text-align: right;\">-0.038828 </td><td style=\"text-align: right;\">0.0207082 </td><td style=\"text-align: right;\">-0.0415848 </td><td style=\"text-align: right;\">-0.0312121 </td><td style=\"text-align: right;\"> 0.0150354</td><td style=\"text-align: right;\"> 0.0221745 </td><td style=\"text-align: right;\">-0.0195091 </td><td style=\"text-align: right;\"> 0.0577201</td><td style=\"text-align: right;\">-0.0569465</td><td style=\"text-align: right;\">-0.00376638 </td><td style=\"text-align: right;\"> 0.00802344</td><td style=\"text-align: right;\"> 0.0536572 </td><td style=\"text-align: right;\"> 0.0169469 </td><td style=\"text-align: right;\"> 0.0172603 </td><td style=\"text-align: right;\">-0.0174735</td><td style=\"text-align: right;\"> 0.0103346 </td><td style=\"text-align: right;\"> 0.0331177 </td><td style=\"text-align: right;\"> 0.0563277</td><td style=\"text-align: right;\"> 0.00481703</td><td style=\"text-align: right;\">-0.0410948 </td><td style=\"text-align: right;\"> 0.00314931</td><td style=\"text-align: right;\"> 0.0392393 </td><td style=\"text-align: right;\"> 0.0101468 </td><td style=\"text-align: right;\">-0.0415722 </td><td style=\"text-align: right;\">-0.0644963 </td><td style=\"text-align: right;\"> 0.0480652 </td><td style=\"text-align: right;\"> 0.0112925 </td><td style=\"text-align: right;\">-0.0561226 </td><td style=\"text-align: right;\">-0.0406242 </td><td style=\"text-align: right;\">-0.00624946</td><td style=\"text-align: right;\">-0.0116796 </td><td style=\"text-align: right;\">-0.040125 </td><td style=\"text-align: right;\">-0.0292939 </td><td style=\"text-align: right;\"> 0.0120191</td><td style=\"text-align: right;\"> 0.000726164</td><td style=\"text-align: right;\"> 0.0390184</td><td style=\"text-align: right;\">-0.0648858 </td><td style=\"text-align: right;\">-0.0544326</td><td style=\"text-align: right;\"> 0.0364144</td><td style=\"text-align: right;\">-0.0354804 </td><td style=\"text-align: right;\"> 0.0287168 </td><td style=\"text-align: right;\">-0.0408558</td><td style=\"text-align: right;\"> 0.00537148</td><td style=\"text-align: right;\">-0.0166096 </td><td style=\"text-align: right;\">-0.0148217 </td><td style=\"text-align: right;\"> 0.00864069</td><td style=\"text-align: right;\">-0.0132793</td><td style=\"text-align: right;\"> 0.0107762</td><td style=\"text-align: right;\"> 0.0475812 </td><td style=\"text-align: right;\">-0.0110306 </td><td style=\"text-align: right;\">-0.0419987 </td><td style=\"text-align: right;\"> 0.00737521</td><td style=\"text-align: right;\"> 0.0581882</td><td style=\"text-align: right;\"> 0.0135915</td><td style=\"text-align: right;\">-0.0274563 </td><td style=\"text-align: right;\">-0.0064732 </td><td style=\"text-align: right;\"> 0.0601388</td><td style=\"text-align: right;\">-0.0345679 </td><td style=\"text-align: right;\">-0.0332279 </td><td style=\"text-align: right;\">-0.0221325 </td><td style=\"text-align: right;\">0.0363207 </td><td style=\"text-align: right;\"> 0.00927576</td><td style=\"text-align: right;\">-0.00137429</td><td style=\"text-align: right;\">-0.057251 </td><td style=\"text-align: right;\"> 0.00328567</td><td style=\"text-align: right;\">-0.00798443 </td><td style=\"text-align: right;\">0.0287494 </td><td style=\"text-align: right;\">-0.0203596 </td><td style=\"text-align: right;\">-0.0159451 </td><td style=\"text-align: right;\">-0.0333854 </td><td style=\"text-align: right;\"> 0.00618047</td><td style=\"text-align: right;\">-0.0343008 </td><td style=\"text-align: right;\"> 0.00322222 </td><td style=\"text-align: right;\">-0.079853 </td><td style=\"text-align: right;\">-0.0570092 </td><td style=\"text-align: right;\">-0.0158511</td><td style=\"text-align: right;\"> 0.0513264 </td><td style=\"text-align: right;\">-0.0356655</td><td style=\"text-align: right;\"> 0.059126 </td><td style=\"text-align: right;\">-0.00116156</td><td style=\"text-align: right;\"> 0.075408 </td><td style=\"text-align: right;\"> 0.00785019 </td><td style=\"text-align: right;\">-0.0360608 </td><td style=\"text-align: right;\">-0.0549585</td><td style=\"text-align: right;\"> 0.0156058 </td><td style=\"text-align: right;\"> 0.00302057</td><td style=\"text-align: right;\"> 0.0368497 </td><td style=\"text-align: right;\"> 0.0135622 </td><td style=\"text-align: right;\">-0.0201859</td><td style=\"text-align: right;\">-0.00640041</td><td style=\"text-align: right;\"> 0.018656 </td><td style=\"text-align: right;\">-0.0105721</td><td style=\"text-align: right;\"> 0.0240232</td><td style=\"text-align: right;\"> 0.0520951</td><td style=\"text-align: right;\"> 0.0568436 </td><td style=\"text-align: right;\"> 0.0579182</td></tr>\n<tr><td>B0041CIR62 </td><td>A16I6WJUEBJ1C3</td><td>okay but not as healthy as it appears </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">138997</td><td>doctorsirena "doctorsirena" </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.34369e+09</td><td>I am always looking for healthier, whole grain versions of foods I enjoy. Unfortunately, these Peacock brand noodles are yet another food masquerading as healthy. The product title in big letters on the front says "Brown Rice Vermicelli", making the consumer think "this is made with brown rice, so it should be a healthy choice". But the first indication that it is not is when looking at the fiber content on the nutrition facts - only 0.6g per 2oz serving. Then onto the ingredients list to see why so low... contains brown rice, sago starch and water. The sago starch comes from palms and must not have much (if any) fiber.<br /><br />The Annie Chun's Maifun Brown Rice Noodles (sold on Amazon and in my local healthy grocer) has become one of my staples and is my frame of reference when comparing to the Peacock brand. The Annie Chun's product is made with 100% whole grain, with ingredients brown rice flour and water. Per 2oz serving, it has 4g fiber and pretty much the same calories and other nutrients as the Peacock brand.<br /><br />If you do try this Peacock brand noodles and have not used rice noodles before, you will need to seek guidance elsewhere on preparation. As others have pointed out, the Peacock package gives almost no directions on how to prepare the product, aside from a brief mention in the recipes (in the header text it does say that they are "easy-to-cook" but does not say how). It also contains a very strange recipe for rice noodles: Aglio Olio style - this is an Italian recipe for noodles with olive oil/garlic/sprinkled with grated cheese that I think would not be very tasty. The second recipe appears to be for a soup with veggie strips. Neither recipe gives amounts or much direction. In comparison, the Annie Chun's package gives clear, specific directions on rice noodle preparation and two recipes.<br /><br />I use rice noodles = maifun = rice sticks = sometimes called vermicelli for making the Vietnamese salad "bun tofu", to serve with stir-fried veggies or in lettuce rolls. They can also be used in spring rolls/egg rolls. When cooking with thin rice noodles, be careful not to oversoak/overcook/overmix or they tend to disintegrate. Asian rice noodle vermicelli (maifun) are not the same as Italian vermicelli and are not readily interchangeable. If making an Italian recipe, the best results would be expected from Italian pasta and not maifun.<br /><br />A few final notes... Both Peacock and Annie Chun's brown rice noodles are gluten free. The Peacock is made in Singapore and the Annie Chun's in Thailand. The Peacock noodles do taste fine (kind of bland), but so do the Annie Chun's. At this time, they are both approximately the same price. Peacock come in an plastic bag with some noodle crushage upon shipping; Annie Chun's are perfect upon removal from their cellophane bag in a box. Overall, I highly recommend the Annie Chun's Maifun as a healthier option over the Peacock brand. On a related note, the Annie Chun's soba and brown rice pad thai noodles are also excellent.<br /><br />Rating for this product: 2.5 stars rounded down to 2 stars.</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">-0.452577 </td><td style=\"text-align: right;\">-0.113423 </td><td style=\"text-align: right;\"> 0.138709 </td><td style=\"text-align: right;\"> 0.182277 </td><td style=\"text-align: right;\"> 0.215801 </td><td style=\"text-align: right;\">0.375206 </td><td style=\"text-align: right;\"> 0.136874 </td><td style=\"text-align: right;\">-0.419679 </td><td style=\"text-align: right;\">-0.0637614</td><td style=\"text-align: right;\"> 0.0869655 </td><td style=\"text-align: right;\"> 0.170927 </td><td style=\"text-align: right;\"> 0.0831557</td><td style=\"text-align: right;\"> 0.117467 </td><td style=\"text-align: right;\"> 0.263678 </td><td style=\"text-align: right;\"> 0.0261424 </td><td style=\"text-align: right;\">-0.014187 </td><td style=\"text-align: right;\"> 2.95397e-05</td><td style=\"text-align: right;\"> 0.010737 </td><td style=\"text-align: right;\"> 0.190601 </td><td style=\"text-align: right;\">-0.424746 </td><td style=\"text-align: right;\">-0.206713 </td><td style=\"text-align: right;\"> 0.303866 </td><td style=\"text-align: right;\"> 0.0229484 </td><td style=\"text-align: right;\"> 0.278378 </td><td style=\"text-align: right;\">-0.413625 </td><td style=\"text-align: right;\">-0.415573 </td><td style=\"text-align: right;\"> 0.0516297 </td><td style=\"text-align: right;\">-0.317631 </td><td style=\"text-align: right;\">-0.311793 </td><td style=\"text-align: right;\">-0.268484 </td><td style=\"text-align: right;\"> 0.133884 </td><td style=\"text-align: right;\">-0.152781 </td><td style=\"text-align: right;\">-0.308243 </td><td style=\"text-align: right;\"> 0.396307 </td><td style=\"text-align: right;\"> 0.046927 </td><td style=\"text-align: right;\"> 0.17968 </td><td style=\"text-align: right;\"> 0.272848 </td><td style=\"text-align: right;\"> 0.703343 </td><td style=\"text-align: right;\">-0.181181 </td><td style=\"text-align: right;\"> 0.303255 </td><td style=\"text-align: right;\"> 0.0247568 </td><td style=\"text-align: right;\">-0.167266 </td><td style=\"text-align: right;\"> 0.333582 </td><td style=\"text-align: right;\">-0.164405 </td><td style=\"text-align: right;\"> 0.273838 </td><td style=\"text-align: right;\">-0.860874 </td><td style=\"text-align: right;\"> 0.204537 </td><td style=\"text-align: right;\"> 0.17322 </td><td style=\"text-align: right;\"> 0.464163 </td><td style=\"text-align: right;\">-0.064438 </td><td style=\"text-align: right;\">-0.166222 </td><td style=\"text-align: right;\"> 0.455148 </td><td style=\"text-align: right;\"> 0.582417 </td><td style=\"text-align: right;\">-0.326859 </td><td style=\"text-align: right;\">-0.103194 </td><td style=\"text-align: right;\"> 0.292087 </td><td style=\"text-align: right;\"> 0.264221 </td><td style=\"text-align: right;\">-0.158491 </td><td style=\"text-align: right;\"> 0.79183 </td><td style=\"text-align: right;\"> 0.108343 </td><td style=\"text-align: right;\">-0.116043 </td><td style=\"text-align: right;\">-0.178905 </td><td style=\"text-align: right;\">-0.154207 </td><td style=\"text-align: right;\">-0.219633 </td><td style=\"text-align: right;\">0.186179 </td><td style=\"text-align: right;\">-0.0761969 </td><td style=\"text-align: right;\">-0.251955 </td><td style=\"text-align: right;\">-0.422602 </td><td style=\"text-align: right;\">-0.0589413 </td><td style=\"text-align: right;\"> 0.209728 </td><td style=\"text-align: right;\">0.217822 </td><td style=\"text-align: right;\"> 0.00609156</td><td style=\"text-align: right;\">-0.150387 </td><td style=\"text-align: right;\">-0.47817 </td><td style=\"text-align: right;\">-0.812136 </td><td style=\"text-align: right;\"> 0.429651 </td><td style=\"text-align: right;\"> 0.179498 </td><td style=\"text-align: right;\">-0.540474 </td><td style=\"text-align: right;\"> 0.149226 </td><td style=\"text-align: right;\">-0.257322 </td><td style=\"text-align: right;\">-0.338138 </td><td style=\"text-align: right;\"> 0.674082 </td><td style=\"text-align: right;\"> 0.234841 </td><td style=\"text-align: right;\">-0.0127675 </td><td style=\"text-align: right;\"> 0.0860772</td><td style=\"text-align: right;\"> 0.0177764 </td><td style=\"text-align: right;\"> 0.0524732 </td><td style=\"text-align: right;\">-0.440476 </td><td style=\"text-align: right;\"> 0.134106 </td><td style=\"text-align: right;\"> 0.0706629 </td><td style=\"text-align: right;\">-0.465079 </td><td style=\"text-align: right;\"> 0.0521292 </td><td style=\"text-align: right;\">-0.0426664</td><td style=\"text-align: right;\">-0.171736 </td><td style=\"text-align: right;\"> 0.291847 </td><td style=\"text-align: right;\">-0.646853 </td><td style=\"text-align: right;\"> 0.181061 </td><td style=\"text-align: right;\"> 0.257701 </td><td style=\"text-align: right;\"> 0.165145 </td><td style=\"text-align: right;\"> 0.42037 </td></tr>\n<tr><td>B001R3BQFW </td><td>AM50E42AFUVNL </td><td>Taste great. </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">126555</td><td>T. Higley "Tina" </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.32356e+09</td><td>I have tried many different drink mix, this is the best tasting by far. It does not have the after taste of the sweetener and I really like it, it is pretty strong, so I use a big water bottle (20 oz) for one tube, it still a little stronger than I like, but it is just my taste. </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">-0.0448968 </td><td style=\"text-align: right;\"> 0.0637996 </td><td style=\"text-align: right;\">-0.0264709 </td><td style=\"text-align: right;\"> 0.0365484 </td><td style=\"text-align: right;\">-0.0477997 </td><td style=\"text-align: right;\">0.1581 </td><td style=\"text-align: right;\"> 0.00405392</td><td style=\"text-align: right;\"> 0.0221248 </td><td style=\"text-align: right;\"> 0.126317 </td><td style=\"text-align: right;\"> 0.0537661 </td><td style=\"text-align: right;\"> 0.00186567</td><td style=\"text-align: right;\"> 0.191703 </td><td style=\"text-align: right;\">-0.0857201</td><td style=\"text-align: right;\">-0.0257879 </td><td style=\"text-align: right;\"> 0.0510087 </td><td style=\"text-align: right;\">-0.00382809</td><td style=\"text-align: right;\">-0.10507 </td><td style=\"text-align: right;\"> 0.0853127 </td><td style=\"text-align: right;\">-0.0472438</td><td style=\"text-align: right;\">-0.112961 </td><td style=\"text-align: right;\"> 0.0210565 </td><td style=\"text-align: right;\"> 0.0023231</td><td style=\"text-align: right;\"> 0.0381847 </td><td style=\"text-align: right;\"> 0.020215 </td><td style=\"text-align: right;\">-0.00684582</td><td style=\"text-align: right;\"> 0.0336772 </td><td style=\"text-align: right;\"> 0.000759529</td><td style=\"text-align: right;\">-0.152974 </td><td style=\"text-align: right;\">-0.143636 </td><td style=\"text-align: right;\"> 0.049318 </td><td style=\"text-align: right;\">-0.0337021 </td><td style=\"text-align: right;\"> 0.0425444 </td><td style=\"text-align: right;\">-0.049597 </td><td style=\"text-align: right;\"> 0.0165418 </td><td style=\"text-align: right;\">-0.00176342</td><td style=\"text-align: right;\">-0.0176381 </td><td style=\"text-align: right;\">-0.0761584 </td><td style=\"text-align: right;\"> 0.0703652</td><td style=\"text-align: right;\">-0.0896073 </td><td style=\"text-align: right;\"> 0.0946777</td><td style=\"text-align: right;\">-0.0769748 </td><td style=\"text-align: right;\">-0.10356 </td><td style=\"text-align: right;\"> 0.0486011</td><td style=\"text-align: right;\"> 0.0529407 </td><td style=\"text-align: right;\"> 0.0283809 </td><td style=\"text-align: right;\">-0.100215 </td><td style=\"text-align: right;\"> 0.0704391 </td><td style=\"text-align: right;\">-0.0395672 </td><td style=\"text-align: right;\"> 0.0186939 </td><td style=\"text-align: right;\"> 0.0291437 </td><td style=\"text-align: right;\">-0.0784622</td><td style=\"text-align: right;\"> 0.08415 </td><td style=\"text-align: right;\"> 0.0509581 </td><td style=\"text-align: right;\">-0.010384 </td><td style=\"text-align: right;\">-0.0655889 </td><td style=\"text-align: right;\"> 0.0313172 </td><td style=\"text-align: right;\"> 0.161041 </td><td style=\"text-align: right;\">-0.0693216</td><td style=\"text-align: right;\"> 0.0230501 </td><td style=\"text-align: right;\">-0.0481256 </td><td style=\"text-align: right;\"> 0.0522207</td><td style=\"text-align: right;\">-0.0185826 </td><td style=\"text-align: right;\">-0.073891 </td><td style=\"text-align: right;\">-0.0442866 </td><td style=\"text-align: right;\">0.0857146 </td><td style=\"text-align: right;\">-0.0156655 </td><td style=\"text-align: right;\"> 0.0475037 </td><td style=\"text-align: right;\">-0.119135 </td><td style=\"text-align: right;\"> 0.0641011 </td><td style=\"text-align: right;\"> 0.0828759 </td><td style=\"text-align: right;\">0.099265 </td><td style=\"text-align: right;\">-0.0904073 </td><td style=\"text-align: right;\">-0.0209445 </td><td style=\"text-align: right;\">-0.0531817 </td><td style=\"text-align: right;\"> 0.086743 </td><td style=\"text-align: right;\">-0.0347895 </td><td style=\"text-align: right;\"> 0.0701843 </td><td style=\"text-align: right;\">-0.178539 </td><td style=\"text-align: right;\">-0.0872093 </td><td style=\"text-align: right;\"> 0.0371891</td><td style=\"text-align: right;\"> 0.0652032 </td><td style=\"text-align: right;\">-0.0987059</td><td style=\"text-align: right;\"> 0.0230445 </td><td style=\"text-align: right;\">-0.0699361 </td><td style=\"text-align: right;\"> 0.139951 </td><td style=\"text-align: right;\">-0.0344978 </td><td style=\"text-align: right;\">-0.0641258 </td><td style=\"text-align: right;\">-0.0340536</td><td style=\"text-align: right;\">-0.0221485 </td><td style=\"text-align: right;\">-0.0848722 </td><td style=\"text-align: right;\">-0.0422028 </td><td style=\"text-align: right;\"> 0.020315 </td><td style=\"text-align: right;\"> 0.0234233</td><td style=\"text-align: right;\"> 0.00761684</td><td style=\"text-align: right;\">-0.0300293 </td><td style=\"text-align: right;\">-0.0190704</td><td style=\"text-align: right;\"> 0.0275005</td><td style=\"text-align: right;\"> 0.0642959</td><td style=\"text-align: right;\"> 0.0682783 </td><td style=\"text-align: right;\"> 0.138162 </td></tr>\n<tr><td>B005HGAV8I </td><td>A2I5KDNOESGJ1H</td><td>variety galore </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">438837</td><td>TJ </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.33402e+09</td><td>This is my favorite item to order for my Keurig. There are so many flavors, my finicky palate never gets bored! The only downside is there are probably 5-6 decaf varieties. I don't drink decaf (I REQUIRE copious amounts of caffeine), so they sit on the shelf... </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\"> 0.0180591 </td><td style=\"text-align: right;\"> 0.0699772 </td><td style=\"text-align: right;\"> 0.0587516 </td><td style=\"text-align: right;\">-0.0706088 </td><td style=\"text-align: right;\"> 0.00662619</td><td style=\"text-align: right;\">0.114756 </td><td style=\"text-align: right;\">-0.144086 </td><td style=\"text-align: right;\"> 0.0275614 </td><td style=\"text-align: right;\">-0.0864321</td><td style=\"text-align: right;\"> 0.0979501 </td><td style=\"text-align: right;\"> 0.0339819 </td><td style=\"text-align: right;\">-0.169916 </td><td style=\"text-align: right;\"> 0.032254 </td><td style=\"text-align: right;\">-0.000541935</td><td style=\"text-align: right;\"> 0.0469471 </td><td style=\"text-align: right;\"> 0.0967113 </td><td style=\"text-align: right;\">-0.0881062 </td><td style=\"text-align: right;\">-0.0478772 </td><td style=\"text-align: right;\">-0.0772477</td><td style=\"text-align: right;\"> 0.0541128 </td><td style=\"text-align: right;\"> 0.0453607 </td><td style=\"text-align: right;\">-0.0475929</td><td style=\"text-align: right;\">-0.062531 </td><td style=\"text-align: right;\"> 0.0431321 </td><td style=\"text-align: right;\"> 0.046509 </td><td style=\"text-align: right;\">-0.00622096 </td><td style=\"text-align: right;\"> 0.0683421 </td><td style=\"text-align: right;\"> 0.00586513</td><td style=\"text-align: right;\"> 0.0312982 </td><td style=\"text-align: right;\"> 0.0244473 </td><td style=\"text-align: right;\">-0.0195214 </td><td style=\"text-align: right;\">-0.0243877 </td><td style=\"text-align: right;\"> 0.00131361</td><td style=\"text-align: right;\"> 0.0688599 </td><td style=\"text-align: right;\"> 0.0185295 </td><td style=\"text-align: right;\"> 0.00569342</td><td style=\"text-align: right;\"> 0.0909279 </td><td style=\"text-align: right;\">-0.0444268</td><td style=\"text-align: right;\"> 0.00835721 </td><td style=\"text-align: right;\">-0.0653897</td><td style=\"text-align: right;\">-0.059535 </td><td style=\"text-align: right;\"> 0.010251 </td><td style=\"text-align: right;\"> 0.0192812</td><td style=\"text-align: right;\"> 0.00349173</td><td style=\"text-align: right;\">-0.0662491 </td><td style=\"text-align: right;\"> 0.0736086</td><td style=\"text-align: right;\"> 0.0397233 </td><td style=\"text-align: right;\">-0.0803349 </td><td style=\"text-align: right;\">-0.0402739 </td><td style=\"text-align: right;\">-0.00139385</td><td style=\"text-align: right;\">-0.0287986</td><td style=\"text-align: right;\"> 0.0184184</td><td style=\"text-align: right;\"> 0.111406 </td><td style=\"text-align: right;\">-0.00877129</td><td style=\"text-align: right;\"> 0.0332851 </td><td style=\"text-align: right;\">-0.031644 </td><td style=\"text-align: right;\">-0.0445515</td><td style=\"text-align: right;\"> 0.0795765</td><td style=\"text-align: right;\"> 0.00433454</td><td style=\"text-align: right;\">-0.0518782 </td><td style=\"text-align: right;\"> 0.0171259</td><td style=\"text-align: right;\">-0.0580711 </td><td style=\"text-align: right;\">-0.0864474 </td><td style=\"text-align: right;\"> 0.0490716 </td><td style=\"text-align: right;\">0.00418072</td><td style=\"text-align: right;\">-0.113097 </td><td style=\"text-align: right;\">-0.0315454 </td><td style=\"text-align: right;\"> 0.0728978 </td><td style=\"text-align: right;\">-0.106284 </td><td style=\"text-align: right;\"> 0.0650408 </td><td style=\"text-align: right;\">0.0351158 </td><td style=\"text-align: right;\"> 0.00593883</td><td style=\"text-align: right;\">-0.0179361 </td><td style=\"text-align: right;\"> 0.00257075</td><td style=\"text-align: right;\"> 0.0165245 </td><td style=\"text-align: right;\">-0.00649849</td><td style=\"text-align: right;\">-0.0536741 </td><td style=\"text-align: right;\">-0.0414687 </td><td style=\"text-align: right;\">-0.0469103 </td><td style=\"text-align: right;\">-0.0308045</td><td style=\"text-align: right;\">-0.0177595 </td><td style=\"text-align: right;\"> 0.0449958</td><td style=\"text-align: right;\"> 0.120506 </td><td style=\"text-align: right;\">-0.00437376</td><td style=\"text-align: right;\"> 0.0102097</td><td style=\"text-align: right;\">-0.0482851 </td><td style=\"text-align: right;\"> 0.00225975</td><td style=\"text-align: right;\">-0.054261 </td><td style=\"text-align: right;\">-0.0365799 </td><td style=\"text-align: right;\">-0.0697037 </td><td style=\"text-align: right;\">-0.095618 </td><td style=\"text-align: right;\">-0.0294475 </td><td style=\"text-align: right;\"> 0.0174716</td><td style=\"text-align: right;\"> 0.06957 </td><td style=\"text-align: right;\"> 0.101367 </td><td style=\"text-align: right;\"> 0.0650128</td><td style=\"text-align: right;\"> 0.0111191</td><td style=\"text-align: right;\">-0.023346 </td><td style=\"text-align: right;\">-0.0656063 </td><td style=\"text-align: right;\">-0.042266 </td></tr>\n<tr><td>B000GFYRHQ </td><td>A3A7YUR6FS6ZCI</td><td>Bigelow Earl Grey Green Tea </td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">245379</td><td>Tea Lover </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.17841e+09</td><td>Tastes like Earl Grey, but it's green tea so it's healthier. </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">-0.0148666 </td><td style=\"text-align: right;\"> 0.00547086</td><td style=\"text-align: right;\">-0.0598337 </td><td style=\"text-align: right;\"> 0.00896689</td><td style=\"text-align: right;\">-0.018512 </td><td style=\"text-align: right;\">0.00104907</td><td style=\"text-align: right;\"> 0.0193834 </td><td style=\"text-align: right;\">-0.0303619 </td><td style=\"text-align: right;\"> 0.0139678</td><td style=\"text-align: right;\">-0.00858391 </td><td style=\"text-align: right;\"> 0.0359025 </td><td style=\"text-align: right;\"> 0.0562054</td><td style=\"text-align: right;\">-0.0571819</td><td style=\"text-align: right;\">-0.0157056 </td><td style=\"text-align: right;\">-0.0115961 </td><td style=\"text-align: right;\">-0.0183816 </td><td style=\"text-align: right;\">-0.0173937 </td><td style=\"text-align: right;\"> 0.0157471 </td><td style=\"text-align: right;\">-0.0117092</td><td style=\"text-align: right;\">-0.0329408 </td><td style=\"text-align: right;\">-0.0213905 </td><td style=\"text-align: right;\"> 0.0359655</td><td style=\"text-align: right;\">-0.00932292</td><td style=\"text-align: right;\"> 0.00242552</td><td style=\"text-align: right;\">-0.0265486 </td><td style=\"text-align: right;\"> 0.000809336</td><td style=\"text-align: right;\"> 0.0293861 </td><td style=\"text-align: right;\">-0.0132304 </td><td style=\"text-align: right;\">-0.0448134 </td><td style=\"text-align: right;\"> 0.00535186</td><td style=\"text-align: right;\"> 0.0132919 </td><td style=\"text-align: right;\"> 0.0232046 </td><td style=\"text-align: right;\">-0.0398275 </td><td style=\"text-align: right;\"> 0.0242108 </td><td style=\"text-align: right;\">-0.00336366</td><td style=\"text-align: right;\"> 0.00886357</td><td style=\"text-align: right;\"> 0.00396398</td><td style=\"text-align: right;\"> 0.0441849</td><td style=\"text-align: right;\"> 0.0122714 </td><td style=\"text-align: right;\"> 0.0319533</td><td style=\"text-align: right;\">-0.00348067</td><td style=\"text-align: right;\">-0.0294214</td><td style=\"text-align: right;\"> 0.0153113</td><td style=\"text-align: right;\">-0.0184586 </td><td style=\"text-align: right;\"> 0.0173887 </td><td style=\"text-align: right;\">-0.0453563</td><td style=\"text-align: right;\"> 0.0025473 </td><td style=\"text-align: right;\"> 0.0130461 </td><td style=\"text-align: right;\"> 0.0404173 </td><td style=\"text-align: right;\"> 0.00737931</td><td style=\"text-align: right;\">-0.0420292</td><td style=\"text-align: right;\"> 0.0284435</td><td style=\"text-align: right;\"> 0.0341711 </td><td style=\"text-align: right;\"> 0.0170331 </td><td style=\"text-align: right;\">-0.00459612</td><td style=\"text-align: right;\"> 0.0128041 </td><td style=\"text-align: right;\"> 0.0366783</td><td style=\"text-align: right;\">-0.0333503</td><td style=\"text-align: right;\">-0.00117786</td><td style=\"text-align: right;\"> 0.00162055</td><td style=\"text-align: right;\"> 0.0079412</td><td style=\"text-align: right;\"> 0.0146446 </td><td style=\"text-align: right;\"> 0.00372818</td><td style=\"text-align: right;\"> 0.00244897</td><td style=\"text-align: right;\">0.0106038 </td><td style=\"text-align: right;\"> 0.0119089 </td><td style=\"text-align: right;\"> 0.0500549 </td><td style=\"text-align: right;\">-0.0508029 </td><td style=\"text-align: right;\"> 0.0211623 </td><td style=\"text-align: right;\">-0.000432948</td><td style=\"text-align: right;\">0.0534101 </td><td style=\"text-align: right;\">-0.0262044 </td><td style=\"text-align: right;\">-0.0137686 </td><td style=\"text-align: right;\"> 0.0017875 </td><td style=\"text-align: right;\">-0.00189653</td><td style=\"text-align: right;\"> 0.00516932</td><td style=\"text-align: right;\"> 0.0155711 </td><td style=\"text-align: right;\">-0.0218159 </td><td style=\"text-align: right;\">-0.0490738 </td><td style=\"text-align: right;\"> 0.0248318</td><td style=\"text-align: right;\"> 0.00470242</td><td style=\"text-align: right;\">-0.0251454</td><td style=\"text-align: right;\"> 0.0269307 </td><td style=\"text-align: right;\">-0.0166039 </td><td style=\"text-align: right;\"> 0.0529522</td><td style=\"text-align: right;\"> 0.000367695</td><td style=\"text-align: right;\">-0.0106417 </td><td style=\"text-align: right;\"> 0.0047698</td><td style=\"text-align: right;\"> 0.00729426</td><td style=\"text-align: right;\"> 0.0195029 </td><td style=\"text-align: right;\">-0.0165866 </td><td style=\"text-align: right;\"> 0.0460096 </td><td style=\"text-align: right;\">-0.0206906</td><td style=\"text-align: right;\"> 0.00637032</td><td style=\"text-align: right;\">-0.00561091</td><td style=\"text-align: right;\">-0.0369974</td><td style=\"text-align: right;\"> 0.0197267</td><td style=\"text-align: right;\"> 0.0345398</td><td style=\"text-align: right;\"> 0.00685155</td><td style=\"text-align: right;\"> 0.0188883</td></tr>\n</tbody>\n</table>"},"metadata":{},"output_type":"display_data"},{"data":{"text/plain":""},"execution_count":29,"metadata":{},"output_type":"execute_result"}]},{"metadata":{},"cell_type":"markdown","source":"## Model 2: GBM with Review vectors\n<blockquote>\n Now we can train a GBM like before, but include the review vectors. This should hopefully increase improvement! We'll log everything to mlflow so we can compare the results.\n </i></br><footer>Splice Machine</footer></blockquote><br>"},{"metadata":{"trusted":false},"cell_type":"code","source":"from h2o.estimators import H2OGradientBoostingEstimator\nmlflow.end_run()\nmlflow.start_run(run_name='GBM with word vectors')\nRATIOS = [0.7,0.15]\n# Train Test Split\next_train,ext_test,ext_valid = ext_reviews.split_frame(ratios=RATIOS)\n# Log our ratios\nmlflow.lp('ratios',RATIOS)\n# Log what word vectors we're using\nmlflow.lp('word vectors', 'reviews')\n\nnon_token_predictors = ['ProductId', 'UserId', 'HelpfulnessNumerator', 'HelpfulnessDenominator', 'Time']\npredictors = non_token_predictors + review_vecs.names\nresponse = 'PositiveReview'\n\nmlflow.lp('label', response)\n# There are a lot of predictors here (C1-C100 + features) so let's shorten that\nmlflow.lp('predictors', non_token_predictors + [f'C1-C{len(review_vecs.columns)}'])\n\ngbm_embeddings = H2OGradientBoostingEstimator(stopping_metric = \"AUC\", stopping_tolerance = 0.001,\n stopping_rounds = 5, score_tree_interval = 10,\n model_id = \"gbm_embeddings.hex\"\n )\nwith mlflow.timer('train_time'):\n gbm_embeddings.train(x = predictors, y = response, \n training_frame = ext_train, validation_frame = ext_test\n )\n\n# Log the model params to mlflow\nmlflow.log_params(gbm_embeddings.get_params())\n# Log the model to MLFlow\nmlflow.log_model(gbm_embeddings, 'vectorized_model')\n# Log the training notebook to MLFlow\nmlflow.log_artifact('MLManager H2O Demo.ipynb', 'training_notebook')\ngbm_embeddings","execution_count":30,"outputs":[{"name":"stdout","output_type":"stream","text":"Starting Code Block train_time... gbm Model Build progress: |███████████████████████████████████████████████| 100%\nDone.\nCode Block train_time:\nRan in 5.513 secs\nRan in 0.092 mins\nSaving artifact of size: 1597.877 KB to Splice Machine DB\nSaving artifact of size: 666.985 KB to Splice Machine DB\nModel Details\n=============\nH2OGradientBoostingEstimator : Gradient Boosting Machine\nModel Key: gbm_embeddings.hex\n\n\nModel Summary: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>number_of_trees</th>\n <th>number_of_internal_trees</th>\n <th>model_size_in_bytes</th>\n <th>min_depth</th>\n <th>max_depth</th>\n <th>mean_depth</th>\n <th>min_leaves</th>\n <th>max_leaves</th>\n <th>mean_leaves</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>50.0</td>\n <td>50.0</td>\n <td>22916.0</td>\n <td>5.0</td>\n <td>5.0</td>\n <td>5.0</td>\n <td>29.0</td>\n <td>32.0</td>\n <td>31.82</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" number_of_trees number_of_internal_trees model_size_in_bytes \\\n0 50.0 50.0 22916.0 \n\n min_depth max_depth mean_depth min_leaves max_leaves mean_leaves \n0 5.0 5.0 5.0 29.0 32.0 31.82 "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\n\nModelMetricsBinomial: gbm\n** Reported on train data. **\n\nMSE: 0.11763099104636422\nRMSE: 0.34297374687629406\nLogLoss: 0.38139378118952527\nMean Per-Class Error: 0.21461955222976004\nAUC: 0.8683543868961463\nAUCPR: 0.9550113279093856\nGini: 0.7367087737922926\n\nConfusion Matrix (Act/Pred) for max f1 @ threshold = 0.603683879583344: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>0</th>\n <th>1</th>\n <th>Error</th>\n <th>Rate</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>6784.0</td>\n <td>8432.0</td>\n <td>0.5542</td>\n <td>(8432.0/15216.0)</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>2227.0</td>\n <td>52529.0</td>\n <td>0.0407</td>\n <td>(2227.0/54756.0)</td>\n </tr>\n <tr>\n <th>2</th>\n <td>Total</td>\n <td>9011.0</td>\n <td>60961.0</td>\n <td>0.1523</td>\n <td>(10659.0/69972.0)</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" 0 1 Error Rate\n0 0 6784.0 8432.0 0.5542 (8432.0/15216.0)\n1 1 2227.0 52529.0 0.0407 (2227.0/54756.0)\n2 Total 9011.0 60961.0 0.1523 (10659.0/69972.0)"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nMaximum Metrics: Maximum metrics at their respective thresholds\n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>metric</th>\n <th>threshold</th>\n <th>value</th>\n <th>idx</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>max f1</td>\n <td>0.603684</td>\n <td>0.907887</td>\n <td>245.0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>max f2</td>\n <td>0.438034</td>\n <td>0.952923</td>\n <td>310.0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>max f0point5</td>\n <td>0.742401</td>\n <td>0.899509</td>\n <td>174.0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>max accuracy</td>\n <td>0.639008</td>\n <td>0.848225</td>\n <td>230.0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>max precision</td>\n <td>0.970715</td>\n <td>1.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>5</th>\n <td>max recall</td>\n <td>0.121993</td>\n <td>1.000000</td>\n <td>392.0</td>\n </tr>\n <tr>\n <th>6</th>\n <td>max specificity</td>\n <td>0.970715</td>\n <td>1.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>7</th>\n <td>max absolute_mcc</td>\n <td>0.702468</td>\n <td>0.526670</td>\n <td>198.0</td>\n </tr>\n <tr>\n <th>8</th>\n <td>max min_per_class_accuracy</td>\n <td>0.777815</td>\n <td>0.784043</td>\n <td>152.0</td>\n </tr>\n <tr>\n <th>9</th>\n <td>max mean_per_class_accuracy</td>\n <td>0.783004</td>\n <td>0.785380</td>\n <td>149.0</td>\n </tr>\n <tr>\n <th>10</th>\n <td>max tns</td>\n <td>0.970715</td>\n <td>15216.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>11</th>\n <td>max fns</td>\n <td>0.970715</td>\n <td>54744.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>12</th>\n <td>max fps</td>\n <td>0.041544</td>\n <td>15216.000000</td>\n <td>399.0</td>\n </tr>\n <tr>\n <th>13</th>\n <td>max tps</td>\n <td>0.121993</td>\n <td>54756.000000</td>\n <td>392.0</td>\n </tr>\n <tr>\n <th>14</th>\n <td>max tnr</td>\n <td>0.970715</td>\n <td>1.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>15</th>\n <td>max fnr</td>\n <td>0.970715</td>\n <td>0.999781</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>16</th>\n <td>max fpr</td>\n <td>0.041544</td>\n <td>1.000000</td>\n <td>399.0</td>\n </tr>\n <tr>\n <th>17</th>\n <td>max tpr</td>\n <td>0.121993</td>\n <td>1.000000</td>\n <td>392.0</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" metric threshold value idx\n0 max f1 0.603684 0.907887 245.0\n1 max f2 0.438034 0.952923 310.0\n2 max f0point5 0.742401 0.899509 174.0\n3 max accuracy 0.639008 0.848225 230.0\n4 max precision 0.970715 1.000000 0.0\n5 max recall 0.121993 1.000000 392.0\n6 max specificity 0.970715 1.000000 0.0\n7 max absolute_mcc 0.702468 0.526670 198.0\n8 max min_per_class_accuracy 0.777815 0.784043 152.0\n9 max mean_per_class_accuracy 0.783004 0.785380 149.0\n10 max tns 0.970715 15216.000000 0.0\n11 max fns 0.970715 54744.000000 0.0\n12 max fps 0.041544 15216.000000 399.0\n13 max tps 0.121993 54756.000000 392.0\n14 max tnr 0.970715 1.000000 0.0\n15 max fnr 0.970715 0.999781 0.0\n16 max fpr 0.041544 1.000000 399.0\n17 max tpr 0.121993 1.000000 392.0"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nGains/Lift Table: Avg response rate: 78.25 %, avg score: 78.22 %\n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>group</th>\n <th>cumulative_data_fraction</th>\n <th>lower_threshold</th>\n <th>lift</th>\n <th>cumulative_lift</th>\n <th>response_rate</th>\n <th>score</th>\n <th>cumulative_response_rate</th>\n <th>cumulative_score</th>\n <th>capture_rate</th>\n <th>cumulative_capture_rate</th>\n <th>gain</th>\n <th>cumulative_gain</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>1</td>\n <td>0.010004</td>\n <td>0.950928</td>\n <td>1.274236</td>\n <td>1.274236</td>\n <td>0.997143</td>\n <td>0.957110</td>\n <td>0.997143</td>\n <td>0.957110</td>\n <td>0.012747</td>\n <td>0.012747</td>\n <td>27.423625</td>\n <td>27.423625</td>\n </tr>\n <tr>\n <th>1</th>\n <td></td>\n <td>2</td>\n <td>0.020008</td>\n <td>0.945205</td>\n <td>1.274236</td>\n <td>1.274236</td>\n <td>0.997143</td>\n <td>0.947873</td>\n <td>0.997143</td>\n <td>0.952492</td>\n <td>0.012747</td>\n <td>0.025495</td>\n <td>27.423625</td>\n <td>27.423625</td>\n </tr>\n <tr>\n <th>2</th>\n <td></td>\n <td>3</td>\n <td>0.030012</td>\n <td>0.940951</td>\n <td>1.272411</td>\n <td>1.273628</td>\n <td>0.995714</td>\n <td>0.943057</td>\n <td>0.996667</td>\n <td>0.949347</td>\n <td>0.012729</td>\n <td>0.038224</td>\n <td>27.241069</td>\n <td>27.362773</td>\n </tr>\n <tr>\n <th>3</th>\n <td></td>\n <td>4</td>\n <td>0.040002</td>\n <td>0.937519</td>\n <td>1.268747</td>\n <td>1.272409</td>\n <td>0.992847</td>\n <td>0.939208</td>\n <td>0.995713</td>\n <td>0.946815</td>\n <td>0.012674</td>\n <td>0.050899</td>\n <td>26.874653</td>\n <td>27.240874</td>\n </tr>\n <tr>\n <th>4</th>\n <td></td>\n <td>5</td>\n <td>0.050006</td>\n <td>0.934455</td>\n <td>1.274236</td>\n <td>1.272774</td>\n <td>0.997143</td>\n <td>0.935936</td>\n <td>0.995999</td>\n <td>0.944638</td>\n <td>0.012747</td>\n <td>0.063646</td>\n <td>27.423625</td>\n <td>27.277434</td>\n </tr>\n <tr>\n <th>5</th>\n <td></td>\n <td>6</td>\n <td>0.100011</td>\n <td>0.921381</td>\n <td>1.266931</td>\n <td>1.269853</td>\n <td>0.991426</td>\n <td>0.927596</td>\n <td>0.993712</td>\n <td>0.936117</td>\n <td>0.063354</td>\n <td>0.127000</td>\n <td>26.693090</td>\n <td>26.985262</td>\n </tr>\n <tr>\n <th>6</th>\n <td></td>\n <td>7</td>\n <td>0.150003</td>\n <td>0.911046</td>\n <td>1.253046</td>\n <td>1.264251</td>\n <td>0.980560</td>\n <td>0.916133</td>\n <td>0.989329</td>\n <td>0.929457</td>\n <td>0.062642</td>\n <td>0.189641</td>\n <td>25.304563</td>\n <td>26.425136</td>\n </tr>\n <tr>\n <th>7</th>\n <td></td>\n <td>8</td>\n <td>0.200009</td>\n <td>0.901310</td>\n <td>1.243922</td>\n <td>1.259169</td>\n <td>0.973421</td>\n <td>0.906064</td>\n <td>0.985352</td>\n <td>0.923608</td>\n <td>0.062203</td>\n <td>0.251845</td>\n <td>24.392236</td>\n <td>25.916875</td>\n </tr>\n <tr>\n <th>8</th>\n <td></td>\n <td>9</td>\n <td>0.300006</td>\n <td>0.882085</td>\n <td>1.228211</td>\n <td>1.248850</td>\n <td>0.961126</td>\n <td>0.891748</td>\n <td>0.977277</td>\n <td>0.912989</td>\n <td>0.122818</td>\n <td>0.374662</td>\n <td>22.821101</td>\n <td>24.884999</td>\n </tr>\n <tr>\n <th>9</th>\n <td></td>\n <td>10</td>\n <td>0.400003</td>\n <td>0.860930</td>\n <td>1.206660</td>\n <td>1.238303</td>\n <td>0.944262</td>\n <td>0.871642</td>\n <td>0.969024</td>\n <td>0.902652</td>\n <td>0.120663</td>\n <td>0.495325</td>\n <td>20.666025</td>\n <td>23.830293</td>\n </tr>\n <tr>\n <th>10</th>\n <td></td>\n <td>11</td>\n <td>0.500000</td>\n <td>0.835298</td>\n <td>1.162280</td>\n <td>1.223099</td>\n <td>0.909533</td>\n <td>0.848680</td>\n <td>0.957126</td>\n <td>0.891858</td>\n <td>0.116225</td>\n <td>0.611549</td>\n <td>16.228028</td>\n <td>22.309884</td>\n </tr>\n <tr>\n <th>11</th>\n <td></td>\n <td>12</td>\n <td>0.599997</td>\n <td>0.803029</td>\n <td>1.101829</td>\n <td>1.202888</td>\n <td>0.862227</td>\n <td>0.819804</td>\n <td>0.941310</td>\n <td>0.879849</td>\n <td>0.110180</td>\n <td>0.721729</td>\n <td>10.182856</td>\n <td>20.288761</td>\n </tr>\n <tr>\n <th>12</th>\n <td></td>\n <td>13</td>\n <td>0.699994</td>\n <td>0.758217</td>\n <td>1.007590</td>\n <td>1.174988</td>\n <td>0.788481</td>\n <td>0.781751</td>\n <td>0.919477</td>\n <td>0.865836</td>\n <td>0.100756</td>\n <td>0.822485</td>\n <td>0.758962</td>\n <td>17.498846</td>\n </tr>\n <tr>\n <th>13</th>\n <td></td>\n <td>14</td>\n <td>0.799991</td>\n <td>0.687910</td>\n <td>0.881390</td>\n <td>1.138289</td>\n <td>0.689724</td>\n <td>0.726272</td>\n <td>0.890759</td>\n <td>0.848391</td>\n <td>0.088136</td>\n <td>0.910622</td>\n <td>-11.861021</td>\n <td>13.828928</td>\n </tr>\n <tr>\n <th>14</th>\n <td></td>\n <td>15</td>\n <td>0.899989</td>\n <td>0.555041</td>\n <td>0.637391</td>\n <td>1.082635</td>\n <td>0.498785</td>\n <td>0.629473</td>\n <td>0.847207</td>\n <td>0.824067</td>\n <td>0.063737</td>\n <td>0.974359</td>\n <td>-36.260871</td>\n <td>8.263484</td>\n </tr>\n <tr>\n <th>15</th>\n <td></td>\n <td>16</td>\n <td>1.000000</td>\n <td>0.041544</td>\n <td>0.256381</td>\n <td>1.000000</td>\n <td>0.200629</td>\n <td>0.405392</td>\n <td>0.782542</td>\n <td>0.782194</td>\n <td>0.025641</td>\n <td>1.000000</td>\n <td>-74.361906</td>\n <td>0.000000</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" group cumulative_data_fraction lower_threshold lift \\\n0 1 0.010004 0.950928 1.274236 \n1 2 0.020008 0.945205 1.274236 \n2 3 0.030012 0.940951 1.272411 \n3 4 0.040002 0.937519 1.268747 \n4 5 0.050006 0.934455 1.274236 \n5 6 0.100011 0.921381 1.266931 \n6 7 0.150003 0.911046 1.253046 \n7 8 0.200009 0.901310 1.243922 \n8 9 0.300006 0.882085 1.228211 \n9 10 0.400003 0.860930 1.206660 \n10 11 0.500000 0.835298 1.162280 \n11 12 0.599997 0.803029 1.101829 \n12 13 0.699994 0.758217 1.007590 \n13 14 0.799991 0.687910 0.881390 \n14 15 0.899989 0.555041 0.637391 \n15 16 1.000000 0.041544 0.256381 \n\n cumulative_lift response_rate score cumulative_response_rate \\\n0 1.274236 0.997143 0.957110 0.997143 \n1 1.274236 0.997143 0.947873 0.997143 \n2 1.273628 0.995714 0.943057 0.996667 \n3 1.272409 0.992847 0.939208 0.995713 \n4 1.272774 0.997143 0.935936 0.995999 \n5 1.269853 0.991426 0.927596 0.993712 \n6 1.264251 0.980560 0.916133 0.989329 \n7 1.259169 0.973421 0.906064 0.985352 \n8 1.248850 0.961126 0.891748 0.977277 \n9 1.238303 0.944262 0.871642 0.969024 \n10 1.223099 0.909533 0.848680 0.957126 \n11 1.202888 0.862227 0.819804 0.941310 \n12 1.174988 0.788481 0.781751 0.919477 \n13 1.138289 0.689724 0.726272 0.890759 \n14 1.082635 0.498785 0.629473 0.847207 \n15 1.000000 0.200629 0.405392 0.782542 \n\n cumulative_score capture_rate cumulative_capture_rate gain \\\n0 0.957110 0.012747 0.012747 27.423625 \n1 0.952492 0.012747 0.025495 27.423625 \n2 0.949347 0.012729 0.038224 27.241069 \n3 0.946815 0.012674 0.050899 26.874653 \n4 0.944638 0.012747 0.063646 27.423625 \n5 0.936117 0.063354 0.127000 26.693090 \n6 0.929457 0.062642 0.189641 25.304563 \n7 0.923608 0.062203 0.251845 24.392236 \n8 0.912989 0.122818 0.374662 22.821101 \n9 0.902652 0.120663 0.495325 20.666025 \n10 0.891858 0.116225 0.611549 16.228028 \n11 0.879849 0.110180 0.721729 10.182856 \n12 0.865836 0.100756 0.822485 0.758962 \n13 0.848391 0.088136 0.910622 -11.861021 \n14 0.824067 0.063737 0.974359 -36.260871 \n15 0.782194 0.025641 1.000000 -74.361906 \n\n cumulative_gain \n0 27.423625 \n1 27.423625 \n2 27.362773 \n3 27.240874 \n4 27.277434 \n5 26.985262 \n6 26.425136 \n7 25.916875 \n8 24.884999 \n9 23.830293 \n10 22.309884 \n11 20.288761 \n12 17.498846 \n13 13.828928 \n14 8.263484 \n15 0.000000 "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\n\nModelMetricsBinomial: gbm\n** Reported on validation data. **\n\nMSE: 0.12646804143667775\nRMSE: 0.35562345456490596\nLogLoss: 0.40431468438767093\nMean Per-Class Error: 0.24322534304415488\nAUC: 0.8356918248557774\nAUCPR: 0.941458335659589\nGini: 0.6713836497115548\n\nConfusion Matrix (Act/Pred) for max f1 @ threshold = 0.5467423242599629: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>0</th>\n <th>1</th>\n <th>Error</th>\n <th>Rate</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>1045.0</td>\n <td>2250.0</td>\n <td>0.6829</td>\n <td>(2250.0/3295.0)</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>302.0</td>\n <td>11423.0</td>\n <td>0.0258</td>\n <td>(302.0/11725.0)</td>\n </tr>\n <tr>\n <th>2</th>\n <td>Total</td>\n <td>1347.0</td>\n <td>13673.0</td>\n <td>0.1699</td>\n <td>(2552.0/15020.0)</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" 0 1 Error Rate\n0 0 1045.0 2250.0 0.6829 (2250.0/3295.0)\n1 1 302.0 11423.0 0.0258 (302.0/11725.0)\n2 Total 1347.0 13673.0 0.1699 (2552.0/15020.0)"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nMaximum Metrics: Maximum metrics at their respective thresholds\n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>metric</th>\n <th>threshold</th>\n <th>value</th>\n <th>idx</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>max f1</td>\n <td>0.546742</td>\n <td>0.899520</td>\n <td>274.0</td>\n </tr>\n <tr>\n <th>1</th>\n <td>max f2</td>\n <td>0.400301</td>\n <td>0.949257</td>\n <td>326.0</td>\n </tr>\n <tr>\n <th>2</th>\n <td>max f0point5</td>\n <td>0.741179</td>\n <td>0.884006</td>\n <td>177.0</td>\n </tr>\n <tr>\n <th>3</th>\n <td>max accuracy</td>\n <td>0.605872</td>\n <td>0.833489</td>\n <td>250.0</td>\n </tr>\n <tr>\n <th>4</th>\n <td>max precision</td>\n <td>0.966835</td>\n <td>1.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>5</th>\n <td>max recall</td>\n <td>0.171955</td>\n <td>1.000000</td>\n <td>385.0</td>\n </tr>\n <tr>\n <th>6</th>\n <td>max specificity</td>\n <td>0.966835</td>\n <td>1.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>7</th>\n <td>max absolute_mcc</td>\n <td>0.728986</td>\n <td>0.460782</td>\n <td>184.0</td>\n </tr>\n <tr>\n <th>8</th>\n <td>max min_per_class_accuracy</td>\n <td>0.783641</td>\n <td>0.753263</td>\n <td>149.0</td>\n </tr>\n <tr>\n <th>9</th>\n <td>max mean_per_class_accuracy</td>\n <td>0.788654</td>\n <td>0.756775</td>\n <td>145.0</td>\n </tr>\n <tr>\n <th>10</th>\n <td>max tns</td>\n <td>0.966835</td>\n <td>3295.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>11</th>\n <td>max fns</td>\n <td>0.966835</td>\n <td>11715.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>12</th>\n <td>max fps</td>\n <td>0.064508</td>\n <td>3295.000000</td>\n <td>399.0</td>\n </tr>\n <tr>\n <th>13</th>\n <td>max tps</td>\n <td>0.171955</td>\n <td>11725.000000</td>\n <td>385.0</td>\n </tr>\n <tr>\n <th>14</th>\n <td>max tnr</td>\n <td>0.966835</td>\n <td>1.000000</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>15</th>\n <td>max fnr</td>\n <td>0.966835</td>\n <td>0.999147</td>\n <td>0.0</td>\n </tr>\n <tr>\n <th>16</th>\n <td>max fpr</td>\n <td>0.064508</td>\n <td>1.000000</td>\n <td>399.0</td>\n </tr>\n <tr>\n <th>17</th>\n <td>max tpr</td>\n <td>0.171955</td>\n <td>1.000000</td>\n <td>385.0</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" metric threshold value idx\n0 max f1 0.546742 0.899520 274.0\n1 max f2 0.400301 0.949257 326.0\n2 max f0point5 0.741179 0.884006 177.0\n3 max accuracy 0.605872 0.833489 250.0\n4 max precision 0.966835 1.000000 0.0\n5 max recall 0.171955 1.000000 385.0\n6 max specificity 0.966835 1.000000 0.0\n7 max absolute_mcc 0.728986 0.460782 184.0\n8 max min_per_class_accuracy 0.783641 0.753263 149.0\n9 max mean_per_class_accuracy 0.788654 0.756775 145.0\n10 max tns 0.966835 3295.000000 0.0\n11 max fns 0.966835 11715.000000 0.0\n12 max fps 0.064508 3295.000000 399.0\n13 max tps 0.171955 11725.000000 385.0\n14 max tnr 0.966835 1.000000 0.0\n15 max fnr 0.966835 0.999147 0.0\n16 max fpr 0.064508 1.000000 399.0\n17 max tpr 0.171955 1.000000 385.0"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nGains/Lift Table: Avg response rate: 78.06 %, avg score: 78.21 %\n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>group</th>\n <th>cumulative_data_fraction</th>\n <th>lower_threshold</th>\n <th>lift</th>\n <th>cumulative_lift</th>\n <th>response_rate</th>\n <th>score</th>\n <th>cumulative_response_rate</th>\n <th>cumulative_score</th>\n <th>capture_rate</th>\n <th>cumulative_capture_rate</th>\n <th>gain</th>\n <th>cumulative_gain</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>1</td>\n <td>0.010053</td>\n <td>0.951514</td>\n <td>1.281023</td>\n <td>1.281023</td>\n <td>1.000000</td>\n <td>0.957157</td>\n <td>1.000000</td>\n <td>0.957157</td>\n <td>0.012878</td>\n <td>0.012878</td>\n <td>28.102345</td>\n <td>28.102345</td>\n </tr>\n <tr>\n <th>1</th>\n <td></td>\n <td>2</td>\n <td>0.020040</td>\n <td>0.944583</td>\n <td>1.272483</td>\n <td>1.276768</td>\n <td>0.993333</td>\n <td>0.947860</td>\n <td>0.996678</td>\n <td>0.952524</td>\n <td>0.012708</td>\n <td>0.025586</td>\n <td>27.248330</td>\n <td>27.676756</td>\n </tr>\n <tr>\n <th>2</th>\n <td></td>\n <td>3</td>\n <td>0.030027</td>\n <td>0.939599</td>\n <td>1.281023</td>\n <td>1.278183</td>\n <td>1.000000</td>\n <td>0.942052</td>\n <td>0.997783</td>\n <td>0.949041</td>\n <td>0.012793</td>\n <td>0.038380</td>\n <td>28.102345</td>\n <td>27.818305</td>\n </tr>\n <tr>\n <th>3</th>\n <td></td>\n <td>4</td>\n <td>0.040013</td>\n <td>0.936074</td>\n <td>1.272483</td>\n <td>1.276760</td>\n <td>0.993333</td>\n <td>0.937724</td>\n <td>0.996672</td>\n <td>0.946216</td>\n <td>0.012708</td>\n <td>0.051087</td>\n <td>27.248330</td>\n <td>27.676048</td>\n </tr>\n <tr>\n <th>4</th>\n <td></td>\n <td>5</td>\n <td>0.050000</td>\n <td>0.933012</td>\n <td>1.263943</td>\n <td>1.274200</td>\n <td>0.986667</td>\n <td>0.934541</td>\n <td>0.994674</td>\n <td>0.943884</td>\n <td>0.012623</td>\n <td>0.063710</td>\n <td>26.394314</td>\n <td>27.420043</td>\n </tr>\n <tr>\n <th>5</th>\n <td></td>\n <td>6</td>\n <td>0.100000</td>\n <td>0.920021</td>\n <td>1.255437</td>\n <td>1.264819</td>\n <td>0.980027</td>\n <td>0.926271</td>\n <td>0.987350</td>\n <td>0.935078</td>\n <td>0.062772</td>\n <td>0.126482</td>\n <td>25.543710</td>\n <td>26.481876</td>\n </tr>\n <tr>\n <th>6</th>\n <td></td>\n <td>7</td>\n <td>0.150000</td>\n <td>0.909208</td>\n <td>1.248614</td>\n <td>1.259417</td>\n <td>0.974700</td>\n <td>0.914476</td>\n <td>0.983134</td>\n <td>0.928211</td>\n <td>0.062431</td>\n <td>0.188913</td>\n <td>24.861407</td>\n <td>25.941720</td>\n </tr>\n <tr>\n <th>7</th>\n <td></td>\n <td>8</td>\n <td>0.200000</td>\n <td>0.899211</td>\n <td>1.229851</td>\n <td>1.252026</td>\n <td>0.960053</td>\n <td>0.904328</td>\n <td>0.977364</td>\n <td>0.922240</td>\n <td>0.061493</td>\n <td>0.250405</td>\n <td>22.985075</td>\n <td>25.202559</td>\n </tr>\n <tr>\n <th>8</th>\n <td></td>\n <td>9</td>\n <td>0.300000</td>\n <td>0.878844</td>\n <td>1.212793</td>\n <td>1.238948</td>\n <td>0.946738</td>\n <td>0.888916</td>\n <td>0.967155</td>\n <td>0.911132</td>\n <td>0.121279</td>\n <td>0.371684</td>\n <td>21.279318</td>\n <td>23.894812</td>\n </tr>\n <tr>\n <th>9</th>\n <td></td>\n <td>10</td>\n <td>0.400000</td>\n <td>0.857422</td>\n <td>1.171002</td>\n <td>1.221962</td>\n <td>0.914115</td>\n <td>0.868480</td>\n <td>0.953895</td>\n <td>0.900469</td>\n <td>0.117100</td>\n <td>0.488785</td>\n <td>17.100213</td>\n <td>22.196162</td>\n </tr>\n <tr>\n <th>10</th>\n <td></td>\n <td>11</td>\n <td>0.500000</td>\n <td>0.832167</td>\n <td>1.135181</td>\n <td>1.204606</td>\n <td>0.886152</td>\n <td>0.845024</td>\n <td>0.940346</td>\n <td>0.889380</td>\n <td>0.113518</td>\n <td>0.602303</td>\n <td>13.518124</td>\n <td>20.460554</td>\n </tr>\n <tr>\n <th>11</th>\n <td></td>\n <td>12</td>\n <td>0.600000</td>\n <td>0.800473</td>\n <td>1.084861</td>\n <td>1.184648</td>\n <td>0.846871</td>\n <td>0.817035</td>\n <td>0.924767</td>\n <td>0.877322</td>\n <td>0.108486</td>\n <td>0.710789</td>\n <td>8.486141</td>\n <td>18.464819</td>\n </tr>\n <tr>\n <th>12</th>\n <td></td>\n <td>13</td>\n <td>0.700000</td>\n <td>0.757131</td>\n <td>0.992751</td>\n <td>1.157234</td>\n <td>0.774967</td>\n <td>0.780013</td>\n <td>0.903367</td>\n <td>0.863421</td>\n <td>0.099275</td>\n <td>0.810064</td>\n <td>-0.724947</td>\n <td>15.723424</td>\n </tr>\n <tr>\n <th>13</th>\n <td></td>\n <td>14</td>\n <td>0.800000</td>\n <td>0.691098</td>\n <td>0.865672</td>\n <td>1.120789</td>\n <td>0.675766</td>\n <td>0.727475</td>\n <td>0.874917</td>\n <td>0.846428</td>\n <td>0.086567</td>\n <td>0.896631</td>\n <td>-13.432836</td>\n <td>12.078891</td>\n </tr>\n <tr>\n <th>14</th>\n <td></td>\n <td>15</td>\n <td>0.900000</td>\n <td>0.565262</td>\n <td>0.709595</td>\n <td>1.075101</td>\n <td>0.553928</td>\n <td>0.636082</td>\n <td>0.839251</td>\n <td>0.823056</td>\n <td>0.070959</td>\n <td>0.967591</td>\n <td>-29.040512</td>\n <td>7.510069</td>\n </tr>\n <tr>\n <th>15</th>\n <td></td>\n <td>16</td>\n <td>1.000000</td>\n <td>0.064508</td>\n <td>0.324094</td>\n <td>1.000000</td>\n <td>0.252996</td>\n <td>0.413434</td>\n <td>0.780626</td>\n <td>0.782094</td>\n <td>0.032409</td>\n <td>1.000000</td>\n <td>-67.590618</td>\n <td>0.000000</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" group cumulative_data_fraction lower_threshold lift \\\n0 1 0.010053 0.951514 1.281023 \n1 2 0.020040 0.944583 1.272483 \n2 3 0.030027 0.939599 1.281023 \n3 4 0.040013 0.936074 1.272483 \n4 5 0.050000 0.933012 1.263943 \n5 6 0.100000 0.920021 1.255437 \n6 7 0.150000 0.909208 1.248614 \n7 8 0.200000 0.899211 1.229851 \n8 9 0.300000 0.878844 1.212793 \n9 10 0.400000 0.857422 1.171002 \n10 11 0.500000 0.832167 1.135181 \n11 12 0.600000 0.800473 1.084861 \n12 13 0.700000 0.757131 0.992751 \n13 14 0.800000 0.691098 0.865672 \n14 15 0.900000 0.565262 0.709595 \n15 16 1.000000 0.064508 0.324094 \n\n cumulative_lift response_rate score cumulative_response_rate \\\n0 1.281023 1.000000 0.957157 1.000000 \n1 1.276768 0.993333 0.947860 0.996678 \n2 1.278183 1.000000 0.942052 0.997783 \n3 1.276760 0.993333 0.937724 0.996672 \n4 1.274200 0.986667 0.934541 0.994674 \n5 1.264819 0.980027 0.926271 0.987350 \n6 1.259417 0.974700 0.914476 0.983134 \n7 1.252026 0.960053 0.904328 0.977364 \n8 1.238948 0.946738 0.888916 0.967155 \n9 1.221962 0.914115 0.868480 0.953895 \n10 1.204606 0.886152 0.845024 0.940346 \n11 1.184648 0.846871 0.817035 0.924767 \n12 1.157234 0.774967 0.780013 0.903367 \n13 1.120789 0.675766 0.727475 0.874917 \n14 1.075101 0.553928 0.636082 0.839251 \n15 1.000000 0.252996 0.413434 0.780626 \n\n cumulative_score capture_rate cumulative_capture_rate gain \\\n0 0.957157 0.012878 0.012878 28.102345 \n1 0.952524 0.012708 0.025586 27.248330 \n2 0.949041 0.012793 0.038380 28.102345 \n3 0.946216 0.012708 0.051087 27.248330 \n4 0.943884 0.012623 0.063710 26.394314 \n5 0.935078 0.062772 0.126482 25.543710 \n6 0.928211 0.062431 0.188913 24.861407 \n7 0.922240 0.061493 0.250405 22.985075 \n8 0.911132 0.121279 0.371684 21.279318 \n9 0.900469 0.117100 0.488785 17.100213 \n10 0.889380 0.113518 0.602303 13.518124 \n11 0.877322 0.108486 0.710789 8.486141 \n12 0.863421 0.099275 0.810064 -0.724947 \n13 0.846428 0.086567 0.896631 -13.432836 \n14 0.823056 0.070959 0.967591 -29.040512 \n15 0.782094 0.032409 1.000000 -67.590618 \n\n cumulative_gain \n0 28.102345 \n1 27.676756 \n2 27.818305 \n3 27.676048 \n4 27.420043 \n5 26.481876 \n6 25.941720 \n7 25.202559 \n8 23.894812 \n9 22.196162 \n10 20.460554 \n11 18.464819 \n12 15.723424 \n13 12.078891 \n14 7.510069 \n15 0.000000 "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\n\nScoring History: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>timestamp</th>\n <th>duration</th>\n <th>number_of_trees</th>\n <th>training_rmse</th>\n <th>training_logloss</th>\n <th>training_auc</th>\n <th>training_pr_auc</th>\n <th>training_lift</th>\n <th>training_classification_error</th>\n <th>validation_rmse</th>\n <th>validation_logloss</th>\n <th>validation_auc</th>\n <th>validation_pr_auc</th>\n <th>validation_lift</th>\n <th>validation_classification_error</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>2020-06-03 20:06:33</td>\n <td>0.131 sec</td>\n <td>0.0</td>\n <td>0.412517</td>\n <td>0.523672</td>\n <td>0.500000</td>\n <td>0.000000</td>\n <td>1.000000</td>\n <td>0.217458</td>\n <td>0.413827</td>\n <td>0.526125</td>\n <td>0.500000</td>\n <td>0.000000</td>\n <td>1.000000</td>\n <td>0.219374</td>\n </tr>\n <tr>\n <th>1</th>\n <td></td>\n <td>2020-06-03 20:06:34</td>\n <td>1.147 sec</td>\n <td>10.0</td>\n <td>0.382665</td>\n <td>0.460842</td>\n <td>0.778901</td>\n <td>0.916229</td>\n <td>1.255981</td>\n <td>0.185903</td>\n <td>0.385980</td>\n <td>0.466955</td>\n <td>0.768938</td>\n <td>0.911758</td>\n <td>1.272596</td>\n <td>0.191478</td>\n </tr>\n <tr>\n <th>2</th>\n <td></td>\n <td>2020-06-03 20:06:35</td>\n <td>2.106 sec</td>\n <td>20.0</td>\n <td>0.368603</td>\n <td>0.432135</td>\n <td>0.815712</td>\n <td>0.933750</td>\n <td>1.268760</td>\n <td>0.174441</td>\n <td>0.374044</td>\n <td>0.442140</td>\n <td>0.799287</td>\n <td>0.926887</td>\n <td>1.281023</td>\n <td>0.181625</td>\n </tr>\n <tr>\n <th>3</th>\n <td></td>\n <td>2020-06-03 20:06:36</td>\n <td>3.032 sec</td>\n <td>30.0</td>\n <td>0.358795</td>\n <td>0.412357</td>\n <td>0.838837</td>\n <td>0.943557</td>\n <td>1.268773</td>\n <td>0.167667</td>\n <td>0.367004</td>\n <td>0.427451</td>\n <td>0.814227</td>\n <td>0.932822</td>\n <td>1.281023</td>\n <td>0.176964</td>\n </tr>\n <tr>\n <th>4</th>\n <td></td>\n <td>2020-06-03 20:06:36</td>\n <td>3.928 sec</td>\n <td>40.0</td>\n <td>0.350731</td>\n <td>0.396337</td>\n <td>0.854237</td>\n <td>0.949766</td>\n <td>1.274236</td>\n <td>0.159421</td>\n <td>0.361087</td>\n <td>0.415420</td>\n <td>0.824577</td>\n <td>0.936954</td>\n <td>1.281023</td>\n <td>0.174501</td>\n </tr>\n <tr>\n <th>5</th>\n <td></td>\n <td>2020-06-03 20:06:37</td>\n <td>4.830 sec</td>\n <td>50.0</td>\n <td>0.342974</td>\n <td>0.381394</td>\n <td>0.868354</td>\n <td>0.955011</td>\n <td>1.274236</td>\n <td>0.152332</td>\n <td>0.355623</td>\n <td>0.404315</td>\n <td>0.835692</td>\n <td>0.941458</td>\n <td>1.281023</td>\n <td>0.169907</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" timestamp duration number_of_trees training_rmse \\\n0 2020-06-03 20:06:33 0.131 sec 0.0 0.412517 \n1 2020-06-03 20:06:34 1.147 sec 10.0 0.382665 \n2 2020-06-03 20:06:35 2.106 sec 20.0 0.368603 \n3 2020-06-03 20:06:36 3.032 sec 30.0 0.358795 \n4 2020-06-03 20:06:36 3.928 sec 40.0 0.350731 \n5 2020-06-03 20:06:37 4.830 sec 50.0 0.342974 \n\n training_logloss training_auc training_pr_auc training_lift \\\n0 0.523672 0.500000 0.000000 1.000000 \n1 0.460842 0.778901 0.916229 1.255981 \n2 0.432135 0.815712 0.933750 1.268760 \n3 0.412357 0.838837 0.943557 1.268773 \n4 0.396337 0.854237 0.949766 1.274236 \n5 0.381394 0.868354 0.955011 1.274236 \n\n training_classification_error validation_rmse validation_logloss \\\n0 0.217458 0.413827 0.526125 \n1 0.185903 0.385980 0.466955 \n2 0.174441 0.374044 0.442140 \n3 0.167667 0.367004 0.427451 \n4 0.159421 0.361087 0.415420 \n5 0.152332 0.355623 0.404315 \n\n validation_auc validation_pr_auc validation_lift \\\n0 0.500000 0.000000 1.000000 \n1 0.768938 0.911758 1.272596 \n2 0.799287 0.926887 1.281023 \n3 0.814227 0.932822 1.281023 \n4 0.824577 0.936954 1.281023 \n5 0.835692 0.941458 1.281023 \n\n validation_classification_error \n0 0.219374 \n1 0.191478 \n2 0.181625 \n3 0.176964 \n4 0.174501 \n5 0.169907 "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nVariable Importances: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>variable</th>\n <th>relative_importance</th>\n <th>scaled_importance</th>\n <th>percentage</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>HelpfulnessDenominator</td>\n <td>3420.752686</td>\n <td>1.000000</td>\n <td>0.187125</td>\n </tr>\n <tr>\n <th>1</th>\n <td>HelpfulnessNumerator</td>\n <td>1690.508789</td>\n <td>0.494192</td>\n <td>0.092476</td>\n </tr>\n <tr>\n <th>2</th>\n <td>C99</td>\n <td>1359.424683</td>\n <td>0.397405</td>\n <td>0.074365</td>\n </tr>\n <tr>\n <th>3</th>\n <td>C62</td>\n <td>1213.377075</td>\n <td>0.354711</td>\n <td>0.066375</td>\n </tr>\n <tr>\n <th>4</th>\n <td>C4</td>\n <td>916.311829</td>\n <td>0.267868</td>\n <td>0.050125</td>\n </tr>\n <tr>\n <th>5</th>\n <td>Time</td>\n <td>797.362915</td>\n <td>0.233096</td>\n <td>0.043618</td>\n </tr>\n <tr>\n <th>6</th>\n <td>C43</td>\n <td>684.979614</td>\n <td>0.200242</td>\n <td>0.037470</td>\n </tr>\n <tr>\n <th>7</th>\n <td>C54</td>\n <td>549.793091</td>\n <td>0.160723</td>\n <td>0.030075</td>\n </tr>\n <tr>\n <th>8</th>\n <td>C16</td>\n <td>497.228485</td>\n <td>0.145356</td>\n <td>0.027200</td>\n </tr>\n <tr>\n <th>9</th>\n <td>C81</td>\n <td>478.059814</td>\n <td>0.139753</td>\n <td>0.026151</td>\n </tr>\n <tr>\n <th>10</th>\n <td>C2</td>\n <td>387.449371</td>\n <td>0.113264</td>\n <td>0.021195</td>\n </tr>\n <tr>\n <th>11</th>\n <td>C73</td>\n <td>348.745483</td>\n <td>0.101950</td>\n <td>0.019077</td>\n </tr>\n <tr>\n <th>12</th>\n <td>C9</td>\n <td>317.079102</td>\n <td>0.092693</td>\n <td>0.017345</td>\n </tr>\n <tr>\n <th>13</th>\n <td>C22</td>\n <td>304.667206</td>\n <td>0.089064</td>\n <td>0.016666</td>\n </tr>\n <tr>\n <th>14</th>\n <td>C100</td>\n <td>290.342468</td>\n <td>0.084877</td>\n <td>0.015883</td>\n </tr>\n <tr>\n <th>15</th>\n <td>C19</td>\n <td>262.545776</td>\n <td>0.076751</td>\n <td>0.014362</td>\n </tr>\n <tr>\n <th>16</th>\n <td>C93</td>\n <td>261.258850</td>\n <td>0.076375</td>\n <td>0.014292</td>\n </tr>\n <tr>\n <th>17</th>\n <td>C60</td>\n <td>259.402893</td>\n <td>0.075832</td>\n <td>0.014190</td>\n </tr>\n <tr>\n <th>18</th>\n <td>C86</td>\n <td>245.558838</td>\n <td>0.071785</td>\n <td>0.013433</td>\n </tr>\n <tr>\n <th>19</th>\n <td>C90</td>\n <td>237.920685</td>\n <td>0.069552</td>\n <td>0.013015</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" variable relative_importance scaled_importance percentage\n0 HelpfulnessDenominator 3420.752686 1.000000 0.187125\n1 HelpfulnessNumerator 1690.508789 0.494192 0.092476\n2 C99 1359.424683 0.397405 0.074365\n3 C62 1213.377075 0.354711 0.066375\n4 C4 916.311829 0.267868 0.050125\n5 Time 797.362915 0.233096 0.043618\n6 C43 684.979614 0.200242 0.037470\n7 C54 549.793091 0.160723 0.030075\n8 C16 497.228485 0.145356 0.027200\n9 C81 478.059814 0.139753 0.026151\n10 C2 387.449371 0.113264 0.021195\n11 C73 348.745483 0.101950 0.019077\n12 C9 317.079102 0.092693 0.017345\n13 C22 304.667206 0.089064 0.016666\n14 C100 290.342468 0.084877 0.015883\n15 C19 262.545776 0.076751 0.014362\n16 C93 261.258850 0.076375 0.014292\n17 C60 259.402893 0.075832 0.014190\n18 C86 245.558838 0.071785 0.013433\n19 C90 237.920685 0.069552 0.013015"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\nSee the whole table with table.as_data_frame()\n"},{"data":{"text/plain":""},"execution_count":30,"metadata":{},"output_type":"execute_result"}]},{"metadata":{},"cell_type":"markdown","source":"## Just like before, let's log all of our outcomes"},{"metadata":{"trusted":false,"scrolled":false},"cell_type":"code","source":"%matplotlib inline \nfrom IPython.core.display import HTML\n\n# Print and Log Model params\nparams = dict(zip(gbm_embeddings.summary().col_header[1:],\n gbm_embeddings.summary().cell_values[0][1:]))\nprint(gbm_embeddings.summary())\nmlflow.log_params(params)\n\n\n#Plot and Log Scoring history\ngbm_embeddings.plot()\nprint(\"AUC on Validation Data: \" + str(round(gbm_embeddings.auc(valid = True), 3)))\n# Log training and validation metrics over time\nfor step, row in gbm_embeddings.scoring_history().iterrows():\n row_dict = row.to_dict()\n for r in row_dict:\n if 'train' in r or 'valid' in r:\n mlflow.log_metric(r, row_dict[r],step=step)\n\n\n# Print and Log Confusion Matrix\nprint(gbm_embeddings.confusion_matrix(valid = True))\nmlflow.lm('fpr', gbm_embeddings.fpr(valid=True)[0][0])\nmlflow.lm('tpr', gbm_embeddings.tpr(valid=True)[0][0])\nmlflow.lm('fnr', gbm_embeddings.fnr(valid=True)[0][0])\nmlflow.lm('tnr', gbm_embeddings.fnr(valid=True)[0][0])\nmlflow.lm('F0point5', gbm_embeddings.F0point5(valid=True)[0][1])\nmlflow.lm('F1', gbm_embeddings.F1(valid=True)[0][1])\nmlflow.lm('F2', gbm_embeddings.F2(valid=True)[0][1])\nmlflow.lm('auc', gbm_embeddings.auc(valid = True))\nmlflow.lp('threshold', gbm_embeddings.F1(valid=True)[0][0]) # First element is the threshold\n\n\n# Plot and Log Variable Importance\ngbm_embeddings.varimp_plot()\nfor var in gbm_embeddings.varimp():\n mlflow.lm(f'varimp_{var[0]}',var[-1])\n \n \n# Partial Dependence Plot\npdp_helpfulness = gbm_embeddings.partial_plot(ext_train, cols = [\"HelpfulnessNumerator\"])","execution_count":31,"outputs":[{"name":"stdout","output_type":"stream","text":"\nModel Summary: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>number_of_trees</th>\n <th>number_of_internal_trees</th>\n <th>model_size_in_bytes</th>\n <th>min_depth</th>\n <th>max_depth</th>\n <th>mean_depth</th>\n <th>min_leaves</th>\n <th>max_leaves</th>\n <th>mean_leaves</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>50.0</td>\n <td>50.0</td>\n <td>22916.0</td>\n <td>5.0</td>\n <td>5.0</td>\n <td>5.0</td>\n <td>29.0</td>\n <td>32.0</td>\n <td>31.82</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" number_of_trees number_of_internal_trees model_size_in_bytes \\\n0 50.0 50.0 22916.0 \n\n min_depth max_depth mean_depth min_leaves max_leaves mean_leaves \n0 5.0 5.0 5.0 29.0 32.0 31.82 "},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\n"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAYgAAAEXCAYAAAC3c9OwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3jV9fXA8ffJIswESBhJCAHZhAAhBBBEhiKgIkuFqq1apS6sP0dFO7TW3WpxtWqtYq1KURAUFVBRAQcQwt5hZ0BCWIEMMs7vj+8FYgyQwL25yc15PU8ecz/3O843PtxzP1tUFWOMMaYsP28HYIwxpnqyBGGMMaZcliCMMcaUyxKEMcaYclmCMMYYUy5LEMYYY8plCcKY0xCRV0Xkj166d7SIHBURf2/c3xiwBGFqIBEZICLfi8hhETkgIt+JSG9330dVb1PVv7j7uiISIyIqIgFlyqeJyOOue+9W1QaqWnyWa90oIkvcHaMxAAFnP8SY6kNEGgFzgduBGUAQcBFQ4Ob7+J/tw9kXiEiAqhZ5Ow5TPVkNwtQ0HQBU9X1VLVbVPFVdoKprThwgIreKyEYRyRGRDSIS7yrvLCLfiMghEVkvIqNKnTNNRP4pIp+JyDFgcOlv9CIySERSReQ+EckUkQwRuanU+U1F5BMROSIiy0Xk8fP5Zl+2luGqKWx3PdMOEblORDoDrwL9XM1Rh1zHhojIf0QkS0R2icgfRMSv1HW+E5G/i8gB4C+uWli3UvduJiJ5IhJ+rvEb32AJwtQ0W4BiEXlbREaISOPSb4rI1cCjwC+BRsAoIFtEAoFPgAVAM2Ay8K6IdCx1+i+AJ4CGQHkf7i2AECAS+DXwSqn7vwIccx3zK9ePW4hIfeBFYISqNgQuBFap6kbgNuAHV3NUqOuUl1xxtgUuxvlb3FTqkn2A7Th/h8eA6cD1pd6fCHypqlnuegZTM1mCMDWKqh4BBgAK/AvIEpGPRaS565BbgGdVdbk6UlR1F9AXaAA8rarHVXUhTlPVxFKXn6Oq36lqiarml3P7QuAxVS1U1c+Ao0BHV0fyOOARVc1V1Q3A2xV4nP2u2swh17f/X5zh2BIgVkTqqmqGqq4v7yBXLNcCD6lqjqruBJ4Dbih1WLqqvqSqRaqa54r1FydqGa5j36lA/MbHWYIwNY6qblTVG1U1CogFIoCprrdbAdvKOS0C2KOqJaXKduHUBk7Yc5ZbZ5dpr8/FSTrhOP15pc8/27UAwlQ19MQP8F55B6nqMZwP/duADBH5VEQ6ne6aOP0yu0qVnfE5VXUpTu3nYtd12wEfVyB+4+MsQZgaTVU3AdNwEgU4H34XlHNoOtCq1LdkgGggrfTlzjGMLKAIiCpV1uocr1UuVZ2vqpcCLYFNOLUn+HnM+3FqOq1LlVXkOd/GaWa6AfjwNDUoU8tYgjA1ioh0cnUUR7let8JpJvrRdcgbwP0i0ksc7USkNXDiW/LvRCRQRAYBV+K0v58X12inWcCjIlLP9S38l+d73RNEpLmIjHL1RRTgNG2dGGG1D4gSkaBSscwAnhCRhq5nvxf471lu8w4wBidJ/MddsZuazRKEqWlycDpZl7pGG/0IrAPuA1DVD3A6mt9zHTsbaKKqx3E6rEfgfMv+B/BLVw3EHe7C6Rjei/Nh+z7uG3rrh/N86cABnI7nO1zvLQTWA3tFZL+rbDJOMtyO09n+HvDmmW6gqqlAMk7tYrGb4jY1nNiGQca4n4g8A7RQVbeNZvI0EXkTpwP7D96OxVQPNlHOGDdwNSsFAWuB3jjDYG/xalCVICIxwFigp3cjMdWJNTEZ4x4NcfohjuH0ATwHzPFqRBUkIn/Baab7q6ru8HY8pvqwJiZjjDHlshqEMcaYcvlUH0RYWJjGxMRU7iQtgkNrIaA+NOzgkbiMMaa6WrFixX5VLXfdLZ9KEDExMSQlJVX+xK2vwfLbIOEm6HCn+wMzxphqSkR2ne49a2ICaDcJWg6HlQ/Aka3ejsYYY6qFWp8g8o4X8+gnG/gh/BnwD4YffwUltjy+McbU+gTh5weLt2Zx/9z95Pd4Efb/ABv/6u2wjDHG63yqD+Jc1Anw55lxcVz92g88s6EHj0RfDWsfgYiR0Li7t8MzplYqLCwkNTWV/HxbM9BdgoODiYqKIjAwsMLneDRBiMhw4AXAH3hDVZ8u8/4gnMlEJybnzFLVx1wLsP0HZ/OVEuB1VX3BU3EmxDThhr6tmfbDLkbf8iTdMxfBD7+Ey5aBfx1P3dYYcxqpqak0bNiQmJgYRMTb4dR4qkp2djapqam0adOmwud5rInJtXHJKziLo3UBJopIl3IOXayqPVw/j7nKioD7VLUzzkYvd57mXLf53fBOtGwUzAMfp1GY8DocWgNrH/XkLY0xp5Gfn0/Tpk0tObiJiNC0adNK18g82QeRCKSo6nbXSprTgasqcqJrx6xk1+85wEZ+uuGJ2zWoE8ATY7qxZd9RXtnSCS74NWx8FrK+9+RtjTGnYcnBvc7l7+nJBBHJT3euSqX8D/l+IrJaRD4Xka5l33QtItYTZz1/jxrcqRlX9Yjgla9TSIn6M9RrBT/8CoqOefrWxhhT7XgyQZSXrsou/JQMtFbV7jgbrc/+yQVEGgAzgXtcexH//CYik0QkSUSSsrLOf4/1P13RhQZ1Anhgzk6K+0yDo9tg5e/O+7rGmJojOzubHj160KNHD1q0aEFkZOTJ18ePH6/QNW666SY2b958xmNeeeUV3n33XXeE7BEeW6xPRPoBj6rqZa7XDwGo6lNnOGcnkKCq+0UkEGdT+fmq+nxF7pmQkKDnNJO6jNkr07jnf6v40xVduLnuS7D57zB4AbS89LyvbYw5u40bN9K5c2dvhwHAo48+SoMGDbj//vt/Uq6qqCp+fjVntkB5f1cRWaGqCeUd78knWw60F5E2ru0QJ1BmI3QRaSGuhjERSXTFk+0q+zewsaLJwZ2u6hHBoI7h/HX+ZlKjfw+NOsOPN8HxQ1UdijGmGklJSSE2NpbbbruN+Ph4MjIymDRpEgkJCXTt2pXHHnvs5LEDBgxg1apVFBUVERoaypQpU+jevTv9+vUjMzMTgD/84Q9MnTr15PFTpkwhMTGRjh078v33Tv/nsWPHGDduHN27d2fixIkkJCSwatWqKnlejw1zVdUiEbkLmI8zzPVNVV0vIre53n8VGA/cLiJFQB4wQVVVRAbgbJ6+VkRO/CUeVtXPPBVvaSLCE2O6Mez5b3no4xT+c9XbyBf9IOluuNC26zWmKv35k/VsSC+3hfmcdYloxCNX/qzLs0I2bNjAW2+9xauvvgrA008/TZMmTSgqKmLw4MGMHz+eLl1+Oujy8OHDXHzxxTz99NPce++9vPnmm0yZMuVn11ZVli1bxscff8xjjz3GvHnzeOmll2jRogUzZ85k9erVxMfHn1Pc58KjdSNV/UxVO6jqBar6hKvsVVdyQFVfVtWuqtpdVfuq6veu8iWqKqoaV2oIbJUkhxMiQ+vy4IhOLN66n1m7WkLXP8DOd2DPrKoMwxhTzVxwwQX07t375Ov333+f+Ph44uPj2bhxIxs2bPjZOXXr1mXEiBEA9OrVi507d5Z77bFjx/7smCVLljBhwgQAunfvTteu55bYzkWtn0l9Jtf3ac3Hq9L5y6cbuPie+wlLnwvLfgNh/aFuc2+HZ0ytcK7f9D2lfv36J3/funUrL7zwAsuWLSM0NJTrr7++3LkGQUFBJ3/39/enqKj89d7q1Knzs2O8ualbzeld8QI/P+HpcXHkFhTz6Nwt0O8/UJgDy38DthOfMbXekSNHaNiwIY0aNSIjI4P58+e7/R4DBgxgxowZAKxdu7bcGoqnWII4i3bNGjB5SDvmrsngy7Sm0P1JSJ0DO972dmjGGC+Lj4+nS5cuxMbGcuutt9K/f3+332Py5MmkpaURFxfHc889R2xsLCEhIW6/T3l8ak9qdw1zLet4UQmjXl7CodxCFvzfABp9NwwOroKRa6F+tNvvZ0xtV52GuXpbUVERRUVFBAcHs3XrVoYNG8bWrVsJCKh8D0F1GubqM4IC/HhmXByZOfk8M28L9J0GWuIMfdUSb4dnjPFhR48epX///nTv3p1x48bx2muvnVNyOBfWSV1B3VuFcnP/NryxZAejuvelT/zfYdmtsOVl6Hi3t8Mzxvio0NBQVqxY4ZV7Ww2iEu4d1oFWTeoyZdZa8qNvhIjLYdWDcOTM0+mNMaYmsgRRCfWCAnhqTBw79h/jhYUp0Odf4F/P2TvCtik1xvgYSxCVNKB9GFf3iuL1RdtZd6Ae9P4HZC+DDU+f/WRjjKlBLEGcgz9c3oXG9YKYMmsNRVFXQ+sJsPbPcGClt0Mzxhi3sQRxDkLqBfLYVV1Zl3aEN5bsgIRXIDjcaWoqLvB2eMaY8zRo0KCfTXqbOnUqd9xxx2nPadCgAQDp6emMHz/+tNc921D8qVOnkpube/L1yJEjOXTIOwuFWoI4RyNiWzCsS3P+/sUWduTUgcQ34PA6WPMnb4dmjDlPEydOZPr06T8pmz59OhMnTjzruREREXz44YfnfO+yCeKzzz4jNDT0nK93PixBnCMR4S+jYwkK8OOhWWvQiBHQbhJs/CtkLvF2eMaY8zB+/Hjmzp1LQYHTIrBz507S09Pp0aMHQ4cOJT4+nm7dujFnzpyfnbtz505iY2MByMvLY8KECcTFxXHttdeSl5d38rjbb7/95DLhjzzyCAAvvvgi6enpDB48mMGDBwMQExPD/v37AXj++eeJjY0lNjb25DLhO3fupHPnztx666107dqVYcOG/eQ+58PmQZyH5o2C+f3IzkyZtZbpy/cwseffIOML+PFXMGI1BDbwdojG1Hwr7nFWLnCnxj2g19TTvt20aVMSExOZN28eV111FdOnT+faa6+lbt26fPTRRzRq1Ij9+/fTt29fRo0addr9nv/5z39Sr1491qxZw5o1a36yVPcTTzxBkyZNKC4uZujQoaxZs4a7776b559/nq+//pqwsLCfXGvFihW89dZbLF26FFWlT58+XHzxxTRu3JitW7fy/vvv869//YtrrrmGmTNncv3115/3n8lqEOfp2t6t6Ne2KU9+tpF9eYHQ7204ugNWPuDt0Iwx56F0M9OJ5iVV5eGHHyYuLo5LLrmEtLQ09u3bd9prLFq06OQHdVxcHHFxcSffmzFjBvHx8fTs2ZP169efdRG+JUuWMGbMGOrXr0+DBg0YO3YsixcvBqBNmzb06NEDOPNy4pVlNYjzJCI8NbYbl01dxB9nr+O1GwYgne+DjX+DqKsgYri3QzSmZjvDN31PGj16NPfeey/Jycnk5eURHx/PtGnTyMrKYsWKFQQGBhITE1Pu8t6llVe72LFjB3/7299Yvnw5jRs35sYbbzzrdc60bt6JZcLBWSrcXU1MVoNwg5iw+tx7aQcWbNjH5+v2QtxfIKQrLP01HD/o7fCMMeegQYMGDBo0iJtvvvlk5/Thw4dp1qwZgYGBfP311+zateuM1xg4cCDvvvsuAOvWrWPNmjWAs0x4/fr1CQkJYd++fXz++ecnz2nYsCE5OTnlXmv27Nnk5uZy7NgxPvroIy666CJ3PW65LEG4ya8HtKFbZAh/mrOOQwV+zt4R+Zmw/C5vh2aMOUcTJ05k9erVJ3d0u+6660hKSiIhIYF3332XTp06nfH822+/naNHjxIXF8ezzz5LYmIi4OwM17NnT7p27crNN9/8k2XCJ02axIgRI052Up8QHx/PjTfeSGJiIn369OGWW26hZ8+ebn7in7Llvt1offphRr38HWN6RvK3q7vD2sdg7SMwYAZEX+21uIypaWy5b8+w5b69qGtECL8Z2JYPV6SyeGsWdH0ImvSG5bdD3l5vh2eMMZViCcLN7h7anrZh9Xlo1lpyi8Rpaio6BktvtW1KjTE1ikcThIgMF5HNIpIiIlPKeX+QiBwWkVWunz9V9NzqKjjQn6fGdiP1YB7PLdgCIZ2g+1OQPhe2v+nt8IypMXyp+bs6OJe/p8cShIj4A68AI4AuwEQR6VLOoYtVtYfr57FKnlst9WnblOv6RPPWdztYteeQs6FQs0HOhJ+jO70dnjHVXnBwMNnZ2ZYk3ERVyc7OJjg4uFLneXIeRCKQoqrbAURkOnAVcObZIOd/brUwZUQnvtqYyYMfruGTyQMI6jcNPu0GP94IQxeCWOueMacTFRVFamoqWVlZ3g7FZwQHBxMVFVWpczyZICKBPaVepwJ9yjmun4isBtKB+1V1fSXOrbYaBgfy+OhYbvlPEq9+u427h7Z3Jvws/TVsfhE63ePtEI2ptgIDA2nTpo23w6j1PPk1trzFScrWF5OB1qraHXgJmF2Jc50DRSaJSJKIJFW3bxuXdGnOld0jeHlhCimZOdD2Joi8ElZNgcMbvR2eMcackScTRCrQqtTrKJxawkmqekRVj7p+/wwIFJGwipxb6hqvq2qCqiaEh4e7M363eOTKLtSr48+DM9dSokDi684ifj/8EkoKvR2eMcaclicTxHKgvYi0EZEgYALwcekDRKSFuBYqEZFEVzzZFTm3pghrUIc/XdGFFbsO8s6Pu6BuC+j9KhxIgvVPeTs8Y4w5LY8lCFUtAu4C5gMbgRmqul5EbhOR21yHjQfWufogXgQmqKPccz0Vq6eN6RnJwA7hPDNvE6kHcyF6PLT+Baz7CxxY4e3wjDGmXLbURhVJPZjLsL8vondME6bd1BspPASfxkJQKAxfAf6VG35mjDHuYEttVANRjevxwGUd+XZLFrNXpUFQY+jzJhzeAKv/4O3wjDHmZyxBVKFf9ouhZ3Qoj32ygeyjBRBxGbS7DTY9D/u+9XZ4xhjzE5YgqpC/n/DMuDiOFhTx509cc/56/hUatHUm0BX+fA14Y4zxFksQVaxD84bcObgdH69OZ+Gmfc6Q135vw7FdkHyft8MzxpiTLEF4wR2D2tGheQN+/9E6cvILIbw/dH4Atv0L0j71dnjGGANYgvCKoAA/nh4Xx94j+Tw7b7NTGPcYhMTC0lugINu7ARpjDJYgvCY+ujE3XhjDOz/uYvnOA+BfBy58B45nw/I7vR2eMcZYgvCm+4d1JKpxXR6cuYb8wmJo3ANiH4Hd/4Od070dnjGmlrME4UX16wTw5JhubM86xksLtzqFXR6Epn0g6Q7ILXf5KWOMqRKWILxsYIdwxsVH8dq329mQfgT8ApxRTcX5Tn+ED810N8bULJYgqoE/XtGZ0HqBPDhzDUXFJdCoI/R4BjI+h21veDs8Y0wtZQmiGgitF8Sjo7qyNu0wb363wynscCc0HwLJ/wdHt3s3QGNMrWQJopq4vFtLLuncnOe/2MKu7GPOlqR93wLxhx9uhJJib4dojKllLEFUEyLC46NjCfTz46FZa53N2utHQ68XIWsxbJ7q7RCNMbWMJYhqpEVIMFNGduL7bdnMSHJtyd3mlxB1Fax+GA7V2C0xjDE1kCWIamZi72gS2zTh8U83knkkH0Rc25SG2DalxpgqZQmimvHzE54e242CohL+NMdVYwhuBomvwcFkWPe4dwM0xtQaliCqobbhDbjnkvbMW7+XeesynMJWYyDmBlj/BGQv926AxphawRJENXXrRW3p0rIRf5yznsO5rmalhBehbkunqakoz7sBGmN8niWIairQ349nx8dx4NhxnvjMtblQUKizTemRTU6ntTHGeJAliGosNjKEWy9qy4ykVL5L2e8UtrwU2t/pDHvd97V3AzTG+DSPJggRGS4im0UkRUSmnOG43iJSLCLjS5X9n4isF5F1IvK+iAR7Mtbq6p5L2tMmrD4PzVpL3nHXZLmez0CDdvDjTVB4xLsBGmN8lscShIj4A68AI4AuwEQR6XKa454B5pcqiwTuBhJUNRbwByZ4KtbqLDjQn6fGdmP3gVye/8K1uVBAfej3H8jdAyv+z7sBGmN8lidrEIlAiqpuV9XjwHTgqnKOmwzMBDLLlAcAdUUkAKgH1Nq1r/u2bcrExGj+vWQHq/cccgrD+0HnB2H7m5D6iXcDNMb4JE8miEhgT6nXqa6yk1w1hTHAq6XLVTUN+BuwG8gADqvqAg/GWu09NLIT4Q3r8ODMNRQWlziF3R6B0DhYdivk7/dugMYYn+PJBCHllJXd3GAq8KCq/mQlOhFpjFPbaANEAPVF5PpybyIySUSSRCQpKyvLDWFXT42CA/nLVbFs2pvDa99ucwr960C/d+D4AVh+u+0dYYxxK08miFSgVanXUfy8mSgBmC4iO4HxwD9EZDRwCbBDVbNUtRCYBVxY3k1U9XVVTVDVhPDwcHc/Q7UyrGsLLu/Wkhe/SiEl86hT2DgOuj0Gez6EXe97N0BjjE/xZIJYDrQXkTYiEoTTyfxx6QNUtY2qxqhqDPAhcIeqzsZpWuorIvVERIChwEYPxlpjPDqqK3WD/Hlo1hpKSlw1hs4PQFg/WH4n5KZ5N0BjjM/wWIJQ1SLgLpzRSRuBGaq6XkRuE5HbznLuUpyEkQysdcX5uqdirUnCG9bhD5d3ZvnOg7y7dJdT6OcPfd+GkuOw9NfW1GSMcQtRH/owSUhI0KSkJG+H4XGqyi/fXEbyroMsuPdiIkPrOm9s+Qck3Qm9/wntz5iDjTEGABFZoaoJ5b1nM6lrIBHhyTHdKFH4w0euzYUA2t8OLS6F5PsgJ8W7QRpjajxLEDVUqyb1uG9YB77enMXHq119/yLQ903wC4Qfb7RtSo0x58USRA12U/82dG8Vyp8/2cCBY8edwnpRkPAyZH0Hm57zboDGmBrNEkQN5u8nPDsujpz8Qh77pNR2pDHXQauxsOaPcGit9wI0xtRoliBquI4tGnL7oHbMXpXO15tdq5WIQO9XneXBv78Bio97N0hjTI1kCcIH3Dn4Ato1a8DvZ63laEGRUxgc7uxlfWg1rHvMuwEaY2okSxA+oE6AP8+MiyPjSD5/nbfp1BtRV0HbG2HDU7D/R6/FZ4ypmSxB+IherRvzq34x/OfHXazYdeDUG/FToW4U/PArKMr1XoDGmBrHEoQPeeCyjkSE1OV3H64hv9A1xDUoBPq+BTlbYNVp92wyxpifsQThQ+rXCeCJMbFsyzrGK1+XmijXYgh0uBu2vAR7v/JegMaYGsUShI8Z1LEZY3pG8s9vtrExo9R2pD2egoYdnG1Kjx/2XoDGmBrDEoQP+uMVXWhUN5ApM9dQfGLF14B6zjaleWmw4rfeDdAYUyNYgvBBTeoH8ciVXVidepi3vttx6o2wPtDlYdjxNqTO8V6AxpgawRKEjxrVPYIhnZrxtwWb2Z1davRS7B+hcU9YeqvtHWGMOSNLED5KRHh8dCwBfn48XHrFV/8gp6mp6Ch8Ggtb/2mL+hljymUJwodFhNblwRGdWJKynw9WpJ56IzQWhidDk56w/A5Y0A8OJHsvUGNMtWQJwsddlxhNYkwTHp+7gcyc/FNvhHSCIV9Bv/9C7m6Y3xuSJtsIJ2PMSZYgfJyfn/DUuG7kF5Xw6Mfrf/qmCLS5Dq7YBO1uhy2vwNxOsPN927bUGGMJoja4ILwBvx3ans/W7mX++r0/PyAoFHq/DJctg3qR8P0vYOGlcGRz1QdrjKk2Kp0gRMRPRBp5IhjjOZMGtqVzy0b8cfY6DucVln9Q0wQYthQSXoEDSfBZHKz+IxTlVW2wxphqoUIJQkTeE5FGIlIf2ABsFpEHPBuacadAfz+eGdeN/UcLeOqzjac/0M8fOtzhNDtFXw3rH4fPYiH986oL1hhTLVS0BtFFVY8Ao4HPgGjgBo9FZTwiLiqUWy5qy/Tle/h+2/4zH1y3BVz4X6cj2y8QvhkJi8dDbuqZzzPG+IyKJohAEQnESRBzVLUQOGsvpogMF5HNIpIiIqddSlREeotIsYiML1UWKiIfisgmEdkoIv0qGKs5g/+7pAOtm9bjoVlryTtegfkPLYbAiNXQ/QlI/9TpxN74HJScppnKGOMzKpogXgN2AvWBRSLSGjhyphNExB94BRgBdAEmikiX0xz3DDC/zFsvAPNUtRPQHThDu4ipqLpB/jw1phu7snOZ+K8f2bn/2NlP8q8DXR+GyzdAs0Gw8n6Y1wuyvvN4vMYY76lQglDVF1U1UlVHqmMXMPgspyUCKaq6XVWPA9OBq8o5bjIwE8g8UeDqBB8I/Nt1/+OqeqgisZqzu7BdGK/8Ip7tWUcZ+eJiZizfc2qm9Zk0aAMXfwIXfQTHD8EXA2DpLZB/luYqY0yNVNFO6t+6OqlFRP4tIsnAkLOcFgnsKfU61VVW+rqRwBjg1TLntgWygLdEZKWIvOHqIC8vtkkikiQiSVlZWRV5HANcHteSefcMJC4qhN/NXMMd7yZz8Njxs58oAq1GO7WJzg/A9rfh006w7d+gJZ4P3BhTZSraxHSzq5N6GBAO3AQ8fZZzpJyysl9TpwIPqmrZxvAAIB74p6r2BI4B5fZhqOrrqpqgqgnh4eFnCcmUFhFal3dv6cuUEZ34cuM+hr+wiO9SKlgbCGwAPZ+FESuhURenJvHFRXBwjWeDNsZUmYomiBMf9iOBt1R1NeUngNJSgValXkcB6WWOSQCmi8hOYDzwDxEZ7To3VVWXuo77ECdhGDfz9xNuu/gCPrqjP/XrBHDdG0t54tMNFBRVcAG/0Fi45NtT25rOi4fk+6Awx7OBG2M8rqIJYoWILMBJEPNFpCFwtvaE5UB7EWkjIkHABODj0geoahtVjVHVGJwkcIeqzlbVvcAeEenoOnQozvwL4yGxkSF8Ovkiru8bzb8W72D0K9+zZV8FP+RFoO2NztyJtjfDpudhbmfYPdOW7DCmBqtogvg1ThNPb1XNBYJwmplOS1WLgLtwRidtBGao6noRuU1EbqvAPScD74rIGqAH8GQFYzXnqG6QP4+P7sYbv0wg80g+V760hLe/31mxDmyAOk2hz+sw7AeoEwZLxsM3l8PR7Z4N3BjjEVLRf/wiMgpnZBHAt6r6iceiOkcJCQmalJTk7TB8QmZOPr/7cA3fbM5icMdwnh3fnfCGdSp+gZIi2PIyrPkjaJGzk12X3zlDZo0x1YaIrFDVhPLeq+gopqeB3+I082wA7haRp9wXoqlumjUM5q0be9N91ycAACAASURBVPPnUV35fls2w6cu4quN+yp+Ab8A6HSP0+wUeSWs/ZOzttPerzwXtDHGrSpUgzjRzKPqjGN0TW5bqapxHo6vUqwG4Rlb9uVw9/sr2bQ3h+v7RvP7kV2oG+RfuYukz4ekO+HoNmg9EeKfg7otPROwMabCzrsG4RJa6veQ8wvJ1CQdmjdkzl39ufWiNvz3x91c8dJi1qVVcmOhiMvg8nUQ+wjsmeks2bH5Jdvu1JhqrKIJ4ilgpYhME5G3gRVYp3GtUifAn99f3oX//roPRwuKGPOP73j1220Ul1RilJJ/MMQ9CiPXQdM+sOJumJ8I2cs9Frcx5txVppO6JdAbZ/7DUtdQ1GrFmpiqxsFjx3n4o7V8vm4vfds24flrehARWrdyF1GF3R9A8j2Qtxfa3wbdn3Q2LzLGVJkzNTGdMUGIyBknp6lqtdrp3hJE1VFVPkhK5dFP1hPgJzw5thtXxEVU/kKFR2DNn2DLS87Q2J7PQcx1ztwKY4zHnU+C+PoM11VVPdt6TFXKEkTV27n/GPf8bxWr9hxibHwkfx7VlYbBgZW/0IGVsPx2yF7qrBjb+x8Q0tnt8RpjfuqcE0RNYwnCOwqLS3jpq628/HUKkY3rMvXaHvRq3aTyF9IS2PYGrJoCRUeh0/0Q+wcIqOf+oI0xgBsShIiMLaf4MLBWVTPLec8rLEF4V9LOA9zzv1WkH8pj8pD2TB7SjgD/Sm97DvmZsPJ3sONtqB8DCS9B5BVuj9cY455hrr8G3gCuc/38C7gX+E5EbOtRA0BCTBM+/+1FjO4ZyQtfbeXq135gV3YFNiQqK7gZ9JvmLAIYUA++vRIWjYFju90eszHm9CqaIEqAzqo6TlXH4ewQVwD0AR70VHCm5mkYHMjz1/TgpYk9Sck8ysgXFvNBUgU3JCqr2UAYsQp6PAMZC5wFADc8a9udGlNFKpogYlS19DoLmUAHVT0A2L9W8zNXdo9g3j0DiY0M4YEP13Dne8kcyq3AhkRl+QU6azhdsQFaXgqrHoTPe0LmYvcHbYz5iYomiMUiMldEfiUiv8JZtnuRa5c32wrUlCsytC7v3dqXB4d3YsH6fQyfupjvK7ohUVn1W8PA2TDwY6cD+8uB8MONkG+7CBrjKRXtpBZgLDAAZ6LcEmCmVrMhUNZJXX2tTT3Mb/+3kh37j3HrRW25b1gH6gRUcj2nE4pyYd3jsOlvENAAejwNF9wCcg4d4sbUcufdSe1KBEuAhcCXwKLqlhxM9dYtKoS5kwfwi8RoXl+0nTGvfE9K5jnuOhdQD3o8CSNWQ2h3WPYbWNAfDq5yb9DG1HIVXe77GmAZzrag1wBLRWS8JwMzvqdeUABPjHE2JNp7JJ/LX1zCOz9UYkOiskI6w9CF0O8dOLYd5vWCFfc4s7ONMeetok1Mq4FLT8x5EJFw4EtV7e7h+CrFmphqjsycfB74YA3fbsliSKdmPDMurnIbEpV1/CCs/j1sfdVZRjz+7xB9tS3ZYcxZuGMehF+ZCXHZlTjXmJ9p1jCYaTf15tEru7AkZT/Dpy5i4aZKbEhUVlBjZ3mOYT9CcAv47lr4ejjkpLgvaGNqmYp+yM8TkfkicqOI3Ah8CnzmubBMbSAi3Ni/DXMnDyC8YR1unpbEH2evI+/4eewREZYIly2DXi9B9o/waSyseRSK890WtzG1RWWW+x4H9McZxbRIVT/yZGDnwpqYaq6ComL+Om8zbyzZQbtmDXhhQg+6RpznvlR5GZB8H+x6Hxq0g4SXnY2LjDEn2WJ9psZYsnU/932wigPHjnP/sI7celFb/PzOsx9h75ew/E7I2QKh3SDmBoj5BdSLdE/QxtRg59wHISI5InKknJ8cETnrUBERGS4im0UkRUSmnOG43iJSXHZklIj4i8hKEZl7tnsZ3zCgfRjzfjuQoZ2a89Tnm7j+30vJOJx3fhdtcQmMXOP0UfjXh1W/g9mt4KtLYPvbUHiOw22N8XEeq0GIiD+wBbgUSAWWAxNVdUM5x30B5ANvquqHpd67F0gAGqnqWZfztBqE7yi9IVGgvx9PjunG5XEt3XPxI1th53+dn6Pbwb8uRI2BNjc4ycQvwD33MaYGcMcopnORCKSo6nZVPQ5MB64q57jJwEyc9Z1OEpEo4HKcVWRNLSMiXNO7FZ/efRExYfW5871k7v9gNUcLis7/4o3aQ9yf4coUuPQ7aPMryPgcvhkBs6Ngxf/BgWRnW1RjajFPJohIYE+p16muspNEJBIYA7xazvlTgd/hrCR7WiIySUSSRCQpK8vW5fE1bcLq8+Ft/bh7SDtmJacy8oXFrNh10D0XF4HwCyHxnzAmAy6aBWEXwtZXnEl3n8XC+qdsmXFTa3kyQZTXs1j2K9lU4EFV/cm4RhG5AshU1RVnu4mqvq6qCaqaEB4efu7Rmmor0N+Pe4d15H+/6UeJKte89gNTv9xCUfEZvztUjn8daDUGBs6CMXuh96vO3IrVD8OcGPhyMGx702Zpm1rFk30Q/YBHVfUy1+uHAFT1qVLH7OBUIgkDcoFJOPtM3AAUAcFAI2CWql5/pntaH4TvO5JfyCNz1vPRyjTio0OZem1Popt6cEvSo9thx7uw8x3I2Qr+wRB5FbS5Hlpe5ixHbkwN5pVhriISgNNJPRRIw+mk/oWqrj/N8dOAuaU7qV3lg4D7rZPalPbx6nR+/9FaVOHRUV0ZFx+JeHJZDVXIXgY73oHd06EgG+qEQ+sJTud2kwRb1sPUSF7ppFbVIuAuYD6wEZihqutF5DYRuc1T9zW1wyjXhkRdIhpx/werueu9lRzO9eDeVSIQ1gd6vwyj02HgHGg+CFJeh/mJ8GlnZwnyozs9F4MxVcwmypkarbhEeW3RNp5fsIXwhnV47pruXHhBWNUFcPwQ7P7QaYLKXOSUhV/k1Cqixzv9GMZUYzaT2vi8tamH+e30lezIPsakgW2579KOBAVU8XqSR3fCrvecZqgjm8AvCCKvdJJFyxHgH1S18RhTAZYgTK2Qe7yIxz/dyHtLd9M1ohEvTOhBu2YNqz4QVTiwwkkUu96Hgiyo0xSir3WSRdM+1l9hqg1LEKZWWbB+L1NmrSX3eBG/v7wL1/eJ9mwH9pmUFELGF04TVOpsZ1XZBu2cUVAx10PDC7wTlzEuliBMrZOZk8/9H6xh0ZYshnZqxjPj4whrcB4bErlD4RHYPdNJFvu+AdSZmNfmeqd2UaeJd+MztZIlCFMrlZQob/+wk6c+30Sj4AD+Or47gzs183ZYjmN7TvVXHF7vzKeIuNxpgoq43Jm4Z0wVsARharXNe3P47fSVbNqbw+CO4Uwe2p746GoyukgVDq5y9Ve8B/n7nJFP0dc4TVDh/a2/wniUJQhT6+UXFvPvJTt4Y/F2DuYWMqBdGJOHtKNP26beDu2UkiLY+5XTBLXnIyjOhfptXP0VNziLDBrjZpYgjHE5VlDEu0t38fqiHew/WkBimybcPaQ9/ds19V5HdnkKj0LqR07NYt9XoCXO6KeY653Z28FVONfD+DRLEMaUkV9YzPvLdvPat9vZeySfntGh3D2kPYM6hlevRAGQm36qv+LQGpAAiBjh9FdEXumsD2XMObIEYcxpFBQV80FSKv/8Zhtph/LoFhnCXUPacWnn5ue/1aknHFzj2uzoXchLh8BGEH210wTV7CKQKp4caGo8SxDGnEVhcQkfJafxyjcp7MrOpVOLhtw1pB0jYlviXx0TRUkxZH7t1Cr2zISiY1C/NcRc5ySLkE7ejtDUEJYgjKmgouISPlmTzssLU9iWdYwLwutz15B2XBkXQYB/Nf12XnQMUuc4yWLvAqe/okkvJ1G0ngB1m3s7QlONWYIwppKKS5TP12Xw8sIUNu3NoXXTetw5qB1j4iMJrK6JAiBvr7O8x4534OBKp8kp7EKIvAIiroCQLjZs1vyEJQhjzlFJifLFxn28tHAr69KOEBlal9sHXcDVCVHUCfD3dnhndmg97P4fpH0KB5Odsvoxp5JF84utg9tYgjDmfKkq32zO4sWFW1m5+xAtGgXzm4vbMjExmuDAap4oAHLTIP0zSJsLe7+A4jwIqA8tLnUljJFQt6W3ozReYAnCGDdRVb5LyebFhVtZtuMAYQ3qMGlgG67r05r6dQK8HV7FFOVB5jdOskj7BHL3OOVNEpxkEXkFNO5pI6JqCUsQxnjA0u3ZvLQwhSUp+2lcL5BfD2jDLy+MoVFwDdqnWhUOrYX0uU7C2P8joE5tIuJyJ1m0uMSpbRifZAnCGA9K3n2QlxemsHBTJo2CA7ixfxtu7h9DaL0auEFQfhakf+4kjIz5zgq0fnWg+eBTtYv6rb0dpXEjSxDGVIF1aYd5aeFW5q/fR4M6AdzQrzW3DGhDU28vM36uio9D1pJTTVFHU5zykNhTyaJpX/CrAX0w5rQsQRhThTbtPcLLC1P4dG0GwQH+XNcnmkkD29KsUQ0fMXRki5Ms0udC5mLQImenvJYjnGTR8jIICvV2lKaSLEEY4wUpmUf5x9cpzFmdjr+fMLF3K35z8QVEhNb1dmjn7/ghyFjgJIyMz6AgG8Qfwi86Vbto1NHbUZoK8FqCEJHhwAuAP/CGqj59muN6Az8C16rqhyLSCvgP0AIoAV5X1RfOdj9LEKY62pV9jH98vY2ZyamIwPherbhj0AW0alLP26G5R0kxZC89Vbs4tNYpb9jemW8ReQWEDwD/GtgnUwt4JUGIiD+wBbgUSAWWAxNVdUM5x30B5ANvuhJES6ClqiaLSENgBTC67LllWYIw1VnqwVxe/XYbM5anUqzKmJ6R3Dm4HW3CfGyE0LFdzuS8tLmwbyGUFDiLCra8zEkYESMgONzbURoXbyWIfsCjqnqZ6/VDAKr6VJnj7gEKgd7AXFX9sJxrzQFeVtUvznRPSxCmJth7OJ/XFm3jvaW7KSwu4cruEdw1uB3tmzf0dmjuV3TM2QTpRO0iLwMQCOt7akZ3aDdb/sOLvJUgxgPDVfUW1+sbgD6qelepYyKB94AhwL8pJ0GISAywCIhV1SPl3GcSMAkgOjq6165duzzyPMa4W1ZOAW8s3s47P+4ir7CYEbEtuGtwe7pENPJ2aJ6hJc72qmmuORcHljvl9VqVWv5jMAT4QB9NDeKtBHE1cFmZBJGoqpNLHfMB8Jyq/igi0yiTIESkAfAt8ISqzjrbPa0GYWqiA8eO8+aSHbz9/U5yCoq4pHNzJg9pR/dWPj4iKG9vqeU/Fji1Df+6zsS8yCuciXr1Ir0dpc+rtk1MIrIDOFG3DANygUmqOltEAoG5wHxVfb4i97QEYWqyw3mFTPtuJ29+t4PDeYUM7BDO3UPakRDTxNuheV5xAWR+e2rOxbGdTnnjnqdqF00TbPkPD/BWggjA6aQeCqThdFL/QlXXn+b4abhqEOLs+fg2cEBV76noPS1BGF+Qk1/IOz/u4o3FOzhw7Dj92jZl8tB29GtbzfbN9hRVOLLxVFPU/u+c5qng5s6igpFXOIsMBvpgn40XeHOY60hgKs4w1zdV9QkRuQ1AVV8tc+w0TiWIAcBiYC3OMFeAh1X1szPdzxKE8SW5x4t4b+luXlu0naycAhJaN2by0PYMbB9WOxLFCQXZzrIfaXOdZUAKD4FfIDQbdGrORYO23o6yxrKJcsbUYPmFxcxI2sM/v9lGxuF8urcKZfLgdgzt3Kx2JQqAkiLY/73TDJU2F45scspDupyacxHWD/xqyMq61YAlCGN8QEFRMTNXpPGPb1JIPZhHl5aNmDykHZd1bYFfddw3uyrkpDhzLtLnOn0YJYUQGAothjrzLloOs8UFz8IShDE+pLC4hDmr0nnl6xR27D9Gh+YNuHNwO66Ii8C/tiYKcFaezfjCGRm1dwHkpjrlDTs4iaLFMGg+yPouyrAEYYwPKi5R5q5J5+WFKWzNPErbsPrcMbgdV/WIqN77ZlcFVaf5KWOBkyz2fQPFuU7fRdiFpxJGk/haPzLKEoQxPqykRJm/fi8vLkxhY8YRWjWpyx2D2jEuPoqggNr94XdScYHTd5GxwPk5sUd3nabQ/BInYbQcBvWivBunF1iCMKYWUFW+2pjJSwu3sjr1MC1DgvnNwLaM6xVFw5q0y11VyM+EvV+eqmHkZTjlIV2cmkXLYdBsYK3YSc8ShDG1iKqyaOt+XvpqK0m7DlIvyJ8r4yKYkNiKHq1Ca9/Ip7NRhcPrXbWL+ZC1CIrzwS/IWYX2RGd3aJxPNkdZgjCmFlJVVu45xPRlu/lkdQZ5hcV0atGQiYnRjO4RSUg9q1WUqyjP2Ulvr6s56tAapzy4mTNBr8UwaHmps2+3D7AEYUwtl5NfyMer03l/2W7WpR2hToAfl3dryYTEaHrHNLZaxZnkZTijo/YugL1fOM1T4NQoTnR2hw+osYsMWoIwxpy0Lu0w7y/bzZxV6RwtKOKC8PpMTIxmbHwUTerbpj5npCVOjeJEZ3fWYig5Dv7BED7wVGd3SGyNWcLcEoQx5mdyjxcxd3UG7y/fzcrdhwjy92NY1+ZMTIymX9umtXfyXWUUHYPMRac6uw+79jSr2/JUZ3eLS5zmqWrKEoQx5ow2783h/WW7+WhlGofzCmndtB7X9m7F+F5RNGsY7O3wao7c1J82RxVkO+WNe5ZqjuoP/nW8G2cpliCMMRWSX1jM5+syeH/ZHpbtOECAnzC0czMmJkZzUfvw2j1Tu7K0BA4kn+rszvoOtAj86zkzuk/UMBp18mpzlCUIY0ylbcs6yv+W7+HDFakcOHacyNC6XJPQimt6R9EypGZ2yHpVYY6zXtSJ4bQ5W5zyelGuZHGZs4ZUnaZVGpYlCGPMOSsoKuaLDfuYvmwPS1L24ycwuGMzJiRGM7hjOAG1fVmPc3V0p9MMlbHAmbRXeAgQaJJwqrO7aV/w9+zAAUsQxhi32J2dy/+SdjMjKZWsnAKaN6rD1b1acW3vVrRqUs/b4dVcJcVwIMmpWexdAPt/BC2GgAbQfMip/ouG7dzeHGUJwhjjVoXFJSzclMn0Zbv5dksWCgxoF8bExGgu6dzc1oA6X8cPw76vXf0X8+Hodqe8fsypZNFiKASd/77lliCMMR6TfiiPGUl7mLF8D+mH82laP4jxvaK4tncr2oY38HZ4viFn26nO7r1fQVGOs+xH0z6nOrvD+p7TUiCWIIwxHldcoizaksX7y3bz1aZMikuUvm2bMDExmsu6tiA40N/bIfqGkkLIXnZqst6BZRDUBMbuswRxJpYgjKkeMo/k88GKVKYv382eA3mE1gtkTM9IJiZG06G5bdjjVgUHIGcrhPU5p9MtQRhjvKKkRPl+WzbvL9/NgvV7KSxWerVuzITerbgiLoK6QVar8DZLEMYYr8s+WsDM5FSmL9vD9v3HaFgngNE9I5mQ2IquESHeDq/W8lqCEJHhwAuAP/CGqj59muN6Az8C16rqh5U5tzRLEMZUf6rKsh0HmL58D5+uzeB4UQlxUSFM6B3NqB4RNKgT4O0QaxWvJAgR8Qe2AJcCqcByYKKqbijnuC+AfOBNVf2woueWZQnCmJrlUO5xPlqZxvRle9i8L4d6Qf6M6h7BhMRoukeF2DLkVeBMCcKTqToRSFHV7a4gpgNXAWU/5CcDM4He53CuMaYGC60XxE3923DjhTEnNzeasyqd6cv3nNrcqGckIXVtcyNv8ORslkhgT6nXqa6yk0QkEhgDvFrZc0tdY5KIJIlIUlZW1nkHbYypeiJCfHRjnh3fnWW/H8rjo2MJ8Bce+Xg9fZ78kntnrGL5zgP4Up9pTeDJGkR5dcOy/3enAg+qanGZqmRFznUKVV8HXgeniekc4jTGVCMNgwO5vm9rru/b+iebG81KTqNdswZM6N3KNjeqIp5MEKlAq1Kvo4D0MsckANNdySEMGCkiRRU81xjj42IjQ3hiTDceHtmZT9c4mxs9/ulGnp23mctiWzCxdyv62uZGHuPJTuoAnI7moUAaTkfzL1R1/WmOnwbMdXVSV+rcE6yT2hjft2nvEaYv28Os5FSO5BfZ5kbnyZvDXEfiNCP544xQekJEbgNQ1VfLHDsNV4I43blnu58lCGNqj/I2N7qkc3MmJLayzY0qwSbKGWN8WkrmUf63fDczk9M4cOw4zRrWYUzPSMbGR9GxhS3tcSaWIIwxtUJBUTELN2YyMzmNbzZnUlSidI1oxLj4KEb1iCCsQfXZC7q6sARhjKl19h8t4JPVzuintWmH8fcTBnUIZ1yvKIZ0amary7pYgjDG1Gqb9+Ywa2Uqs1emse9IAY2CA7iyewRj46OIjw6t1TO2LUEYYwzOnhXfpexnVnIq89bvJb+whDZh9RnbM5Ix8ZFENa5926ZagjDGmDJy8gv5fN1eZq5IZemOAwD0bduEsfFRjOzWstYsGmgJwhhjzmDPgVxmr0xjZnIqO7NzCQ70Y3jXFozrFcWFF4T59JBZSxDGGFMBqkry7kPMTE5l7up0juQX0bxRHUb3jGR8fBTtfXA3PEsQxhhTSfmFxXy1MZNZyal8syWL4hKlW2QIY+MjGdU9gqY+MmTWEoQxxpyH/UcLXAsGprI+/QgBfsKgjs0Y3yuSwZ2aUSeg5g6ZtQRhjDFusmnvEWYlp/HRyjSycgoIrRfIlXERjI2PpEermjdk1hKEMca4WVFxCUtS9jMrOY356/dSUFRC2/D6jIuPYnTPSCJD63o7xAqxBGGMMR50JL+Qz9dmMDM5jWU7DiAC/do2ZWx8FCNiW1C/Gg+ZtQRhjDFVZHd2Lh+tTGPWylR2ZedSN9CfEbEtGBsfRb8Lmla7IbOWIIwxpoqpKit2HXSGzK7JICe/iJYhwYzuGcm4+EjaNaseQ2YtQRhjjBflFxbz5cZ9zFyRyqKt+ykuUbpHhTA2PopR3SNo7MXtUy1BGGNMNZGZk8/Hq9KZmZzGxowjBPoLgzs2Y1yvKAZ3bEZQgF+VxmMJwhhjqqEN6UeYlZzK7FXp7D9aQON6gVzZPYJx8VHERYVUyZBZSxDGGFONFRWXsDhlPzNXpLJgwz6OF5XQrlkDxsZHMqZnJC1DPDdk1hKEMcbUEIfzCvlsbQazklNZvvMgItD/gjDGxkcyPLYF9YLcO2TWEoQxxtRAu7KPMSvZGTK750Ae9YL8GRHbknHxkfRt2xQ/NwyZtQRhjDE1WEmJkrTrILOSU/l0TQY5BUVEhAQzJj6SsfFRXBDe4JyvbQnCGGN8RH5hMQs27GNWciqLtmRRohAfHcr0Sf3OaQRUrUkQIpIF7DrH08OA/W4MpyawZ/Z9te15wZ65slqranh5b/hUgjgfIpJ0uizqq+yZfV9te16wZ3anqp2RYYwxpsawBGGMMaZcliBOed3bAXiBPbPvq23PC/bMbmN9EMYYY8plNQhjjDHlsgRhjDGmXLU+QYjIcBHZLCIpIjLF2/F4goi8KSKZIrKuVFkTEflCRLa6/tvYmzG6m4i0EpGvRWSjiKwXkd+6yn32uUUkWESWichq1zP/2VXus88MICL+IrJSROa6Xvv08wKIyE4RWSsiq0QkyVXm9ueu1QlCRPyBV4ARQBdgooh08W5UHjENGF6mbArwlaq2B75yvfYlRcB9qtoZ6Avc6fp/68vPXQAMUdXuQA9guIj0xbefGeC3wMZSr339eU8YrKo9Ss1/cPtz1+oEASQCKaq6XVWPA9OBq7wck9up6iLgQJniq4C3Xb+/DYyu0qA8TFUzVDXZ9XsOzgdIJD783Oo46noZ6PpRfPiZRSQKuBx4o1Sxzz7vWbj9uWt7gogE9pR6neoqqw2aq2oGOB+mQDMvx+MxIhID9ASW4uPP7WpuWQVkAl+oqq8/81Tgd0BJqTJfft4TFFggIitEZJKrzO3P7d6FxWue8tbKtXG/PkREGgAzgXtU9UhV7NDlTapaDPQQkVDgIxGJ9XZMniIiVwCZqrpCRAZ5O54q1l9V00WkGfCFiGzyxE1qew0iFWhV6nUUkO6lWKraPhFpCeD6b6aX43E7EQnESQ7vquosV7HPPzeAqh4CvsHpe/LVZ+4PjBKRnTjNw0NE5L/47vOepKrprv9mAh/hNJe7/blre4JYDrQXkTYiEgRMAD72ckxV5WPgV67ffwXM8WIsbidOVeHfwEZVfb7UWz773CIS7qo5ICJ1gUuATfjoM6vqQ6oapaoxOP92F6rq9fjo854gIvVFpOGJ34FhwDo88Ny1fia1iIzEacf0B95U1Se8HJLbicj7wCCcJYH3AY8As4EZQDSwG7haVct2ZNdYIjIAWAys5VT79MM4/RA++dwiEofTOemP8+Vvhqo+JiJN8dFnPsHVxHS/ql7h688rIm1xag3gdBO8p6pPeOK5a32CMMYYU77a3sRkjDHmNCxBGGOMKZclCGOMMeWyBGGMMaZcliCMMcaUyxKEMcaYclmCMOY0ROQbEUk4+5Fuu99fXct0//U074/20dWGTTVV29diMsYjRCRAVYsqedpvgHBVLTjN+6OBucAGN93PmDOyGoSp8UQkxrUx0L9c38AXiEjd0jUAEQlzrdmDiNwoIrNF5BMR2SHy/+3dT0hUURTH8e+vNoYUEbSNQIT+WtIf0aQ20bIgCwmiRX+gwAqiPwuXQZsCqVWrFkX//0G2UcJIwkoJzKSIolrWKsQiWtRpcc/Qa3jOqBjUdD6r9+67982ZEefOuxfOUZukQ1505rGkOZnbb5fUJ2lY0mofX+1FmAZ8zKbMfa9L6gS6x4hV/qQw7AVfWr39DlANPCm0FY1rAjYCJ71ITI2/vxOSHgAHPdXGTY9rQNKaMvEuViowNChpSFLtVPw9QuWIJ4hQKWqBbWa2R9I1oKVM/yWkFOBVwBvgmJnVS+oAdpDSrwBUm1mTpLXAOR/XTsr7s9NzH/VLuuf9G4G6EikONpOK+SwjpT4ZkNRrZhslfTaz5XmDzKzPJ5G7ZnYDwDPTzjazdX5+Cegws4eS5gFde58PHAAAAbVJREFUwMIS8e4FTpvZRc9FNr3MZxb+MzFBhErxzswG/fgpML9M//teSGhU0gjQ6e3PgbpMv8uQii5JmuVfsBtIWUQPe58qUv4bSDUYSuW/aQYue1ruj/7rfxWTTxJ5NXO8HlikXynNZ3lSt7HifQS0KxXduWVmrycZQ6hQMUGESpFdt/8OzCCVHS0so1aV6P8jc/6D3/8vipOVGamOSIuZvcpekNQAfCkT51QXpMi+3jSg0cy+FsWVGy/wUtITUkW2Lkm7zaxniuML/7DYgwiV7D2wwo+3TPIehT2CZmDEzEZISzf7/YsXSfUTuF8v0KpU+W0usBboH+fYUWBmievdQFvhRFJhuSo3Xs8K+tbMzpCeYOoIISMmiFDJTgH7JPWR1vsn45OPPwvs8rbjpHrPQ5KG/Xy8bgNDwDOgBzhqZh/GOfYKcMQ3mmtyrh8AVvqG8wvSHkOpeFuBYaUSpQuA8xN4H+E/EOm+Qwgh5IoniBBCCLlikzqEP0DSUuBCUfM3M2sYx9h2YGtR8/VKrHYY/m6xxBRCCCFXLDGFEELIFRNECCGEXDFBhBBCyBUTRAghhFw/AVzg7r/8oEMpAAAAAElFTkSuQmCC\n","text/plain":"<Figure size 432x288 with 1 Axes>"},"metadata":{"needs_background":"light"},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"AUC on Validation Data: 0.836\n\nConfusion Matrix (Act/Pred) for max f1 @ threshold = 0.5467423242599629: \n"},{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>0</th>\n <th>1</th>\n <th>Error</th>\n <th>Rate</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>0</td>\n <td>1045.0</td>\n <td>2250.0</td>\n <td>0.6829</td>\n <td>(2250.0/3295.0)</td>\n </tr>\n <tr>\n <th>1</th>\n <td>1</td>\n <td>302.0</td>\n <td>11423.0</td>\n <td>0.0258</td>\n <td>(302.0/11725.0)</td>\n </tr>\n <tr>\n <th>2</th>\n <td>Total</td>\n <td>1347.0</td>\n <td>13673.0</td>\n <td>0.1699</td>\n <td>(2552.0/15020.0)</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" 0 1 Error Rate\n0 0 1045.0 2250.0 0.6829 (2250.0/3295.0)\n1 1 302.0 11423.0 0.0258 (302.0/11725.0)\n2 Total 1347.0 13673.0 0.1699 (2552.0/15020.0)"},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"\n"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAA54AAAJTCAYAAABgjsk5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeZhlVXkv4N8HrShBMREEg0NHkKABRGmcEAGHaNLG60ziEIkRVGJMjBNOCUluIonJNQ4RghN4Y8QRLwoaVAQxGrVRsAXiAGkTwNnYDiBCs+4fe5cci1PV1dCri6p+3+c5zzln7bX3/vY5Vf30r9bae1drLQAAANDLNotdAAAAAMub4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAJtdVX2iqq7ZDNu5tKq+ugn996iqVlVvvLH7BgA2H8ETYBmoqn8ZA9ezFtD3w2PfR22J2pabMVS3qnrAYtfS26YG/61BVT1k/P4/Mk+fmT+AfHVW+x2q6o+q6kNVta6qrqqq71bVGRv7fayqm1XV08ff329V1U/H5zOq6mlVteJGHNODq+qtVfWVqvrhuO1vVNVHqupFVbXblHVmfg8mH9dU1Ter6gNV9bAp66yY6LuhqlbOU9M5E32ffEOPDbjpuMH/SAFwk3JCkt9JckSS4+bqNP5H78FJvp7kAx3reWKSW3bcPixFf5zkeUkuSXJmkm8mWZnk0UkeWlWvbK29cPZKVXWnJKcmuUeSb2T43f1Gkl2T/GaShyY5qqoe2Vq7fKHFVNVtkpyU5JFJfprk4+O2r0iyc5L7JDk2yZ9X1b1ba1+Yspm3JPmv8fUtk9wtyW8kWV1Vv99ae/OUda7J8H/QpyX50yl17ZXkARP9gGXALzPAMtBaO6uqvpzknlV1r9ba5+bo+vtJKslbWms3eirsPPX818Z7wVbn35M8sLV2zmRjVe2d5JNJXlBVb2utnT+xbIckH8oQ6N6c5NmttSsnlv9CkuOTPDnJ6VV1v8nlcxlHSN+b5NAMIfiprbVLp/TbO8lfJLn1HJt6c2vtE7PWOSzJyUleMtY822VJ/ifJ06rqz1trG2YtP2J8/kASMzNgmTDVFmD5eMP4fMS0hVW1bZLfS9KSvHGifbeq+rOq+uQ4ve6nVXVZVb1tHHmYvZ2fnUdZVb9aVe+qqm9X1bUz00+nneNZVdtV1R9W1Qer6mvjNMPvjVMHrzctb9a6t6mq11fV5VX1k6q6oKr+oKpqoR9OVf1CVb2kqs6vqh9X1Y/GYz5sodvYyPYvraqvVtWtq+rV4/srq+rzVfXIsc+Kqnr5OKXxJ2P/602PnpjO+bKqOrCqPlpVPxgfH6yqe81Rw22q6m+q6svj9r9Xw7TOB21kH/etqtPH/q2qnlxVLcluSXafNZ1y8mfnMePPyVcmPtM1VfXsqrre/zGq6p/Hbdyxqo6qqi+OdX6jqo6vqqnhZuz/2onP7btV9ZmqeukcfV9fVZfUdVNZ/19V7T/f97cltNbePTt0ju1fTPLu8e0hsxY/P0PoPCfJ02eHytbaj5McnuTTGUZEn7PAcp6aIXT+R5LfmhY6Z2prrT0mQ2heqDPG553n6fOGDD9fvzHZWFU3T/K7GUZfv7QJ+wRu4gRPgOXjpAzT5Z5YVdtPWf4bGf6j95HW2n9OtB+a5IVJvpfkPUn+IclnkjwhyWfGEY9p9hz73SHJP2f4j+QP56lv53HbOyT5cJL/k2H64P5JPlhVh8+x3nYZRmQekuRfxv3cNsnrxu1tVFX9YpJ/S/JXSa7OMApzUpJdkpxcVccsZDsLsF2SjyR5WJL3Zfhc7prkvVV1SIbP98gkH0vypgyjSK+vqsfOsb37j32vzHC8/5rk15N8oqruP+sYfynJpzJ8l/+T4bM5JcmBST5SVU+fYx8PyPCf/JuPNb01yVeS/HmG7/N/xtczj1Mn1v3bJPtlCCWvTfJ/x2N67bitufx9hu/i80n+McO00Wdk+Hx+TlXdJ8n5SZ6d5NIkr07y9iQ/yqxpmlW1Ksl5SZ6ZIVC9Jsn7M4S5T1bVr8/qP3POYbfR/01w9fg8u5aZPyT9ZWutTVtxHDH86/HtkQvc38zPw9+21q7YWOdNnCHxkPF5zTx93pZhSu/sn8tHJ9kp1/0hDVguWmseHh4eHsvkkeQdGUY0D5+y7P+Nyx43q32XJDtM6X/PJD9O8v5Z7XuM22lJ/mKOOj6R5JpZbbdIstuUvrdJclGSbyfZbtayS8f9nJ3k5hPtOyX5z3HZ/afU9sZZ2/nnsf1PZrXfMkMIvjbJPgv8jD8xbusBc9T6vsnjyBDsW4Zg/+9JdpxYdtcMgeOzs7b1kInP+Jmzlj12bP+PJDXR/qax/fWz+u+VIUD+JMkd59jH789xrJcm+eo8n8XuU9q2yRAqWpL95/ge/jPJHSbab5ZhqmlLcq+J9u0ynD/Ykjxhyr5mb+OSDCF99ndzhwznNV866+doxbjta+Y6xin7nPncLklyzByP14x95vzspvwOfDvJhiR3nWj/lXE7P82s340p29hhXL8l2XUjfW8+/ty1JHde6LHP8Xvw5onj/pvx5/+nSdYm2WvWOjOf97rx/YljHbef6PORDL8rt8hwfmlL8uQbUqOHh8dN67HoBXh4eHh4bL5HhgsHtSSfmNV++/E/eN9IcrNN2N7pGUYltp1omwl3l03+J37WetcLnhvZzwszK0SO7TNh7n5T1nn6uOwNU2p740Tb7cb/kH9qjn3vP67z1wusdWPB885T1pkJTw+csuycJFcl2WaibSbcXJSJcDlrnZbkwPH9dhkC1/okt5nS/xVj/5dM2cdn5znWeYPnPOvde/b+xvaZ4Hn4lHWOyKygneSwse09C9jnTCB/xRzLnzcu//VZ7Xsl+dVNOLbJwL6xx0Y/uwznXL937P/qWcvuP7ZfusDavpNZ4X2Ofr88UeOKKcsflOuH6UfO8Xsw7fGdJC/OrH8fcv3geeDkz0mSu2T4I9BrxveCp4fHMnq4uBDA8nJmkouTHFhVd2utXTS2/16G//Sd2Fq7evZK4zmIz8gQwm6b61987pcyjMhMOq+19tNNKa6q9knyggzTO385Q2CadL3bNmQYPZl2ftlZ4/M9N7Lbe2cYhas5ptTO1HC3jWxnIb7TWvvalPbLk9wxybSLPl2WYQRq5wxXOZ10TmutTVnn7Ayf4T0zTCG+e4YRok+31r4/pf+ZSY7O9M/qM1PaFqSqdsrwff5mhtG5X5jVZdr3mUyfgvnf4/MvTrTdd3z+4ALKud/4/CtzfM+/Oj7fLdedg5jW2n8sYNvTfLS19pBpC6pqjwzTlRfi1Rmml56V4bP8uU2Nz9N+BqbueoH9N3Zu9IOSzD5/9k35+WnWMw5q48WFxvMzVyZ5boapv79eVQ9urV07bSettX+rqguT/H5VvSLDHx8qptnCsiR4AiwjrbWZi7+8IsOI4PPGC/A8LbMuKjSjqv4kwzl338swze1rGUbPWpLHJNkn1w+IyTB6umBVdeC4/W2SfDTD1N8fZhjhuFeS35pjP9+aI3zN7H/Hjez6tuPzfcbHXHbYyHYWYv0c7dck2dBa+9Ecy5Jhquhss4PojNnHPvP89Tn6z7TfZp5tbZLxnNI1Se6c4cI2b83wM3RNhj9U/GGmf59JMi0cz3wO2060zdR72QJKmvmeN3axqM3xPW8WVfWqDJ/TxzJc4Gf2H3JmvrfbVdV2rbWr5tnWL+S6z2uun4MZM9N6t83wB6Cfuwp1a+1lSV42bvfhWVjwz1j/l5M8q6rumeHc2scmedc8q70xw/neD8t4kaTW2tqF7A9YWgRPgOXnLRluf/C7VfXiJAcl2T3Jma212Te0v1mGaXSXZ5ie981Zyw+aZz8LHYWZ8fIMo3I/GyGZ2M/LMwTPaW5XVTUlfO46Ps8V9jJr+dR7JN7E7TJH++xjXz+rfbbbz+o3aVO/xxlHZgidL2+t/e/JBePPzR/ewO1Omgmoc42cTpo5ttWttdM3w767Gf8Y9OoMn9FHMkxjvd4tUFprl1TV1zN8fw/McD7yXB6U4Y86l7TW5v1jQmvtp1X12Qwjyg/O8G/G5vbpDH/ouXfmD55vzfCHsjdk+Pm93pWKgeXBVW0BlpkxPJ6a4QI8j8p1V408YUr3XZLcKsM5obND562z8Wmsm2KPDKOXn5iy7OB51rt5rptyOemQ8fnzG9nvpzOEq/lC9E3VQWNImW3m85o59gszXDzonnPckuTQ8Xmu+7vOZWZUbJo9xufrXYk283+fm2JmivVvzNvr5/vepL/n8fs8PkPo/FCGkc757rs5M0vhpXP8LGS8dc1LxrfTfs/n2+4LquoWC1xnU8xMmZ73/5qtte9mOMf1DhlmQLyjQy3ATYDgCbA8zZwj9bwM5499J8OtNWb7eobAcsA4VS/Jz87Vem1+/ny7G2tdkp2r6tcmG6vqGRlGXeZz7FjTzDo75bqRkXlHa1prX89wM/v7VtWLa7if6c+p4d6kd974IWxxe2U49/ZnxluvPCDDPQ4/mSTjFMy3Z5hy+xez+t81w61Ifprh4j6b4rsZp3lOWbZufD5k1v5WJXnRJu5nLu/LcO7nY6rqCbMXVtUdJt6eMtb0nJrjvrBVdf/ZIauq9qqqX53Wf3MbA+KbMowWfyDJo1prP9nIaq/M8F0fnOSfptS/fYYry943w21nXrPAck7KcF7p3ZK8v6rmGlWeNj17XlV1lyT/a3x71gJWeXGGf6ce3ob7kgLLkKm2AMvTGRluWXHv8f3rpl0IqLW2oapel+Em9Wur6tQM5+U9KEOIOTubb/TqVRkC5ier6p1JfjDWd78Mo2Zz3cvy0gyjsl+cqO9xGablvaa19skF7PtZGUbo/jrJ4VX1iQznud0+w4V5ViV5fIbzW29KPpjkNVW1OsPtKe6a4bzbKzPcAmVymuzMRZv+qKruneG72znD/Vh3SPKs1trPncu3AB/NMOr9oao6J0N4/Xxr7bQMt8J4XpLXVtVDknw1w71dH5Hh+9zYuZYb1Vq7qqoen2Fk8B1V9cwMF0O6ZYbA9MAM07dn+j5m7Puhqvq3DPf0vDLJnZIckOECSDtn+GNLqmpFhisHb8iW+T/Rn2e40NcVSb6Q5MVTBjE/11r72UV8Wms/HM+zPDXDxXceUVUfzHBu7q5JVmeYufC5bHz09Gdaa9dU1aMz3Hv1EUkuqaqzk1ww1rdzkr0z/H5elWHmwDRPG7//ZDhPeWWGmRbbJ3lfa+39C6jla7np/e4Bm5ngCbAMjRcZelOSmXPv5rtK5IuTfCvDBYiekeG8ug9nGFF8xWas6bSq+l/jdn87w8VkPpNhxGyvzB08r8oQhF+R5IkZLiJzcZK/SvKPC9z3+vG8w2ck+Z0MwXW7DBfv+UqSP85w5debmk9mOM6/zHXnTH44yUtba+dOdmytfbeq7pNhyuWjk/xJhgDxqQznt37kBuz/z5PcOkMwOSjDtNs3JTmttXbp+JkemyEAPjxDiHtGko9nMwTPJGmtfbqq9svwc/rwDLfg+GGGoHvMrL6fr6p9Mxz7IzL8TF+bYWT/3AznGf/P5qjrBvqV8Xn7XDc1drbrXT22tbZuHEk+PMPn+sgMI5HfzxCuX5rkpNbaNdkE4xWQf6uqHprkdzOEzAMzBMjvZQihL07yf1trc13g6fcmN5nhXNtzM5y7+eZNqQdY3mr6hQIBgMUyjiB9OFMu3AMAS5FzPAEAAOhK8AQAAKArwRMAAICunOMJAABAV65qy4KcdNJJ7alPfepilwEAANx0Xe8eUTNMtWVBfvxj93MGAABuGMETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoKsVi10AS8Pay9Zn5dGnLXYZAABAknXHrl7sEjaJEU8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArjYaPKvqR7PeH15Vr9vIOhvtM/Z7e1V9oaqeO0+fQ6rqAxvb1o011vztqvp8VX2lqv61qu7fe7/z1PPMqvrdG7juyqp64uauCQAA4IZYsVg7rqpdk9y/tXbnxaphine01p6dJFV1aJL3VtWhrbWLtnQhrbXjb8TqK5M8Mcm/LHSFqlrRWrvmRuwTAABgqhs11baqdq6q91TVZ8fHgVP6nFhVx1fVOVX15ap6xLjojCS3q6rzquqgqjqrqlaN6+xUVeumbOuYqnrz2PeSqnrOxLInV9Vnxu39U1VtOz5OrKovVtXamZHVqnpOVV04jraePO3YWmsfS3JCkiPHdXavqg9V1bnjsew1cXyvqapPjjU9bmyvqnrlxL4PG9sPqaqzq+qd4+dxbFU9aax9bVXtPnGszx9fn1VVfzP2+XJVHTS2rxxr+dz4mBmhPTbJQeNn8dyqukVVvWXc/ufHUD0zyvuuqnr/+H0AAABsdgsZ8bxlVZ038f6Xkpw6vn51kle11j5RVXdK8q9J7jZlGyuTHJxk9yQfq6o9kjwyyQdaa/slSVUttOa9khya5FZJvlRVxyXZI8lhSQ5srV1dVa9P8qQkFyTZrbW297iP24zbODrJr7TWrppom+ZzSZ4xvj4hyTNba1+pqvskeX2SB43Lbp/kAWNtpyZ5d5LHJNkvyT2S7JTks1X18bH/PTJ8Tt9LckmSN7bW7l1Vf5TkD5P88ZRaVox9fjPJnyV5SJJvJXloa+0nVXXXJG9Psmo8vue31h4xHvfzkqS1ts8YmM+oqj3H7d4vyb6tte/N3mFVHZkxeB/xxy9KtpvnkwIAAJjDQoLnlTPhMBlGyTKEm2QIP3efCI23rqpbTdnGO1tr1yb5SlVdkiGgff8G1nxaa+2qJFdV1beS7JLkwUn2zxDukuSWGULZ+5Pcpapem+S0XDeq94Ukb6uq9yV53zz7qiSpqh2S3D/JuyaOdTKGvW88vgurapex7QFJ3t5a25Dkm1V1dpIDkvwgyWdba18ft33xRF1rM4Tqad47Pp+bIcgnyc2SvK6q9kuyIcmeU9abqeW1SdJa+4+q+tpE3w9PC51j3xMyBO4866WvaNkwx9YBAADmcWPP8dwmyf1aa1dONk4ZvWwbeZ8k1+S6qb+3mGefV0283pDhGCrJSa21F8/uXFX3SPKwJH+Q5AlJnpZkdZIHZhh1fXlV/doc+7pnkovGur4/GcDnqalmPW+s/7UT76/N3N/JTJ8NE32em+SbGUZQt0nykznWna+WH8+zDAAA4Ea7sbdTOSPJs2fejCNv0zy+qrYZz1+8S5IvTemzLsOoZZI8bhPr+GiSx1XV7cY6fqmq7lxVOyXZprX2niQvT3KvqtomyR3HczhfmOQ2SXaYvcGqOjjDNNM3tNZ+kOQ/q+rx47IaA+18Pp7ksPE8050zBN3PbOJxbcyOSb4+jrY+Jcm2Y/sPM0xFnqzlSUkyTrG9U6Z/BwAAAJvdjR3xfE6Sf6yqL4zb+niSZ07p96UkZ2eYFvvM8ZzE2X3+Lsk7q+opSc7clCJaaxdW1csynLu4TZKrM4xwXpnkLWNbkrw4Qzj756raMcNI4Ktaa98f6zmsqh6QZPsk/5nksRNXtH1SkuPG/dwsyclJzp+nrFMynD95foYR3he21r4xc1GizeT1Sd4zBuKP5brRyy8kuaaqzk9y4tjv+Kpam2Fk+fDx/NbNWAoAAMB01dq0Wa+bcQdVJ2a4iNC7u+6Irp710le0D27Yd7HLAAAAkqw7dvVilzDNnCNbN3aqLQAAAMzrxk613ajW2uG99wEAAMBNlxFPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgqxWLXQBLwz677Zjjjlq92GUAAABLkBFPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6WrHYBbA0rL1sfVYefdpilwHAJlp37OrFLgEAjHgCAADQl+AJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdLXZg2dV/WjW+8Or6nUbWWejfcZ+b6+qL1TVc+fpc0hVfWDhFd8wY83XVtW+E21frKqVvfe9EFX1qKq6+2LXAQAAsGRGPKtq1yT3b63t21p71WLXM7o0yUsXa+dVte08ix+VZJOCZ1WtuHEVAQAAXN8WDZ5VtXNVvaeqPjs+DpzS58SqOr6qzqmqL1fVI8ZFZyS5XVWdV1UHVdVZVbVqXGenqlo3ZVvHVNWbx76XVNVzJpY9uao+M27vn6pq2/Fx4jhyuXZmZLWqnlNVF46jrSdP7OIDSX6tqn51yr5/NPH6cVV14sTxHVdVHxtrOnis8aKZPmO/X6+qT1XV56rqXVW1w9i+rqr+tKo+keTxVXXE+FmeP36221fV/ZM8Mskrx+Pbvar2q6p/H4/hlKr6xXF7Z1XVX1fV2Un+aEFfJAAAwCboETxvOYad86rqvCR/MbHs1Ule1Vo7IMljk7xxjm2sTHJwktVJjq+qW2QIUhe31vZrrZ2zCfXsleRhSe6d5M+q6mZVdbckhyU5sLW2X5INSZ6UZL8ku7XW9m6t7ZPkLeM2jk5yz9bavkmeObHta5P8bZKXbEI9SfKLSR6U5LlJ3p/kVUl+Lck+Y0DcKcnLkjyktXavJGuS/MnE+j9prT2gtXZykve21g5ord0jyUVJfr+19skkpyZ5wfh5XZzkrUleNB7D2iR/NrG927TWDm6t/f1kkVV1ZFWtqao1G65Yv4mHCAAAMOgRPK8cw85+Y6j704llD0nyujGQnprk1lV1qynbeGdr7drW2leSXJIhPN5Qp7XWrmqtfSfJt5LskuTBSfZP8tmxlgcnucu4r7tU1Wur6uFJfjBu4wtJ3lZVT05yzazt/0uS+1bVr2xCTe9vrbUMAfCbrbW1rbVrk1yQIXTfN8M02X8b63tqkjtPrP+Oidd7j6PDazOE51+bvbOq2jFDuDx7bDopyQPn2N7PtNZOaK2taq2t2nb7HTfh8AAAAK6zpc/p2ybJ/VprV042VtXsfm0j75MhAM4E51vMs8+rJl5vyHDMleSk1tqLZ3euqntkGCH9gyRPSPK0DCOvD8ww6vryqvpZuGutXVNVf5/kRfPUPLu+mZqunVXftWN9G5J8uLX2O3Mc048nXp+Y5FGttfOr6vAkh8yxznx+vPEuAAAAN8yWvrjQGUmePfOmqvabo9/jq2qbqto9w0jkl6b0WZdh1DJJHreJdXw0yeOq6nZjHb9UVXcep7hu01p7T5KXJ7lXVW2T5I6ttY8leWGS2yTZYdb2TswwmrvzRNs3q+pu4/qP3sT6/j3JgVW1x1jf9lW15xx9b5Xk61V1swwjnjN+OC5La219kv+pqoPGZU9JcnYAAAC2gC094vmcJP9YVV8Y9/3x/Pw5kzO+lCEY7ZLkma21n0wZFf27JO+sqqckOXNTimitXVhVL0tyxhgMr84wwnllkreMbUny4iTbJvnncbpqZThH9fuT9bTWflpVr8lwDuuMozNcfOi/k3wx1w+r89X37XH08u1Vtd3Y/LIkX57S/eVJPp3kaxmm7s5MXT45yRvGCyo9LsN03eOravsMU4p/b6H1AAAA3Bg1nGp40zFe2fUDrbV3L3YtXOdZL31F++CGfTfeEYCblHXHrl7sEgDYelxvtHDGkrmPJwAAAEvTlp5qu1GttcMXuwYAAAA2HyOeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXa1Y7AJYGvbZbcccd9TqxS4DAABYgox4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQ1YrFLoClYe1l67Py6NMWuwyA61l37OrFLgEA2AgjngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgucyUFW7VtXJVXVxVV1YVadX1Z5V9TdV9cXxcdhE/wdV1efG9pOqasVi1g8AACxvgucSV1WV5JQkZ7XWdm+t3T3JS5L8TpJ7JdkvyX2SvKCqbl1V2yQ5Kclvt9b2TvK1JE9dnOoBAICtgeC59B2a5OrW2vEzDa2185JckeTs1to1rbUfJzk/ycOT3DbJVa21L4/dP5zksVu4ZgAAYCsieC59eyc5d0r7+Ul+o6q2r6qdMgTUOyb5TpKbVdWqsd/jxvbrqaojq2pNVa3ZcMX6DqUDAABbA8FzmWqtnZHk9CSfTPL2JJ9Kck1rrSX57SSvqqrPJPlhkmvm2MYJrbVVrbVV226/4xaqHAAAWG4Ez6XvgiT7T1vQWvur1tp+rbWHJqkkXxnbP9VaO6i1du8kH59pBwAA6EHwXPrOTLJdVR0x01BVB1TVwVV12/H9vkn2TXLG+P524/N2SV6U5PjrbRUAAGAzcRuNJa611qrq0Un+oaqOTvKTJOuSHJ3knOGit/lBkie31mam1L6gqh6R4Q8Px7XWztzylQMAAFsLwXMZaK1dnuQJUxbdfY7+L0jygq5FAQAAjEy1BQAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSg0Hnw8AABeKSURBVPAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK5WLHYBLA377LZjjjtq9WKXAQAALEFGPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6GrFYhfA0rD2svVZefRpi10GcBO07tjVi10CAHATZ8QTAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBcxmoql2r6uSquriqLqyq06tqz6q6U1WdUVUXje0rx/5vq6ovVdUXq+rNVXWzxT0CAABgORM8l7iqqiSnJDmrtbZ7a+3uSV6SZJckb03yytba3ZLcO8m3xtXelmSvJPskuWWSp2/xwgEAgK2G4Ln0HZrk6tba8TMNrbXzknw3yYrW2ofHth+11q4YX5/eRkk+k+QOi1A3AACwlRA8l769k5w7pX3PJN+vqvdW1eer6pVVte1kh3GK7VOSfGjahqvqyKpaU1VrNlyxfrMXDgAAbB0Ez+VrRZKDkjw/yQFJ7pLk8Fl9Xp/k4621c6ZtoLV2QmttVWtt1bbb79izVgAAYBkTPJe+C5LsP6X90iSfb61d0lq7Jsn7ktxrZmFV/VmSnZP8yRapEgAA2GoJnkvfmUm2q6ojZhqq6oAk2yX5xaraeWx+UJILx+VPT/KwJL/TWrt2C9cLAABsZQTPJW68QNCjkzx0vJ3KBUmOSXJ5hmm2H62qtUkqyRvG1Y7PcNXbT1XVeVX1p1u+cgAAYGuxYrEL4MZrrV2e5AlTFn0lyb5T+vveAQCALcaIJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHS1YrELYGnYZ7cdc9xRqxe7DAAAYAky4gkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFcrFrsAloa1l63PyqNPW+wy4CZn3bGrF7sEAICbPCOeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeC5zJWVbtW1clVdXFVXVhVp1fVnuOyW1fVZVX1usWuEwAAWN4Ez2WqqirJKUnOaq3t3lq7e5KXJNll7PKXSc5erPoAAICtx4rFLoBuDk1ydWvt+JmG1tp5SVJV+2cIoB9KsmpxygMAALYWRjyXr72TnDu7saq2SfL3SV6wsQ1U1ZFVtaaq1my4Yn2HEgEAgK2B4Ln1OSrJ6a21/95Yx9baCa21Va21Vdtuv+MWKA0AAFiOTLVdvi5I8rgp7fdLclBVHZVkhyQ3r6oftdaO3qLVAQAAWw0jnsvXmUm2q6ojZhqq6oAkJ7TW7tRaW5nk+UneKnQCAAA9CZ7LVGutJXl0koeOt1O5IMkxSS5f1MIAAICtjqm2y1hr7fIkT5hn+YlJTtxS9QAAAFsnI54AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQ1YrFLoClYZ/ddsxxR61e7DIAAIAlyIgnAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdrVjsAlga1l62PiuPPm2xy4AFW3fs6sUuAQCAkRFPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK5WLHYBLExV3TbJR8e3uybZkOTb4/srWmv3X5TCAAAANkLwXCJaa99Nsl+SVNUxSX7UWvu7RS0KAABgAUy1XQaq6kfj8yFVdXZVvbOqvlxVx1bVk6rqM1W1tqp2H/vtXFXvqarPjo8DF/cIAACA5UzwXH7ukeSPkuyT5ClJ9myt3TvJG5P84djn1Ule1Vo7IMljx2XXU1VHVtWaqlqz4Yr1/SsHAACWJcFz+flsa+3rrbWrklyc5IyxfW2SlePrhyR5XVWdl+TUJLeuqlvN3lBr7YTW2qrW2qptt99xC5QOAAAsR87xXH6umnh97cT7a3Pd971Nkvu11q7ckoUBAABbJyOeW6czkjx75k1V7beItQAAAMuc4Ll1ek6SVVX1haq6MMkzF7sgAABg+TLVdglqrR0z6/0O4/NZSc6aaD9k4vXPlrXWvpPksM5lAgAAJDHiCQAAQGeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFcrFrsAloZ9dtsxxx21erHLAAAAliAjngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdLVisQtgaVh72fqsPPq0xS6DZWTdsasXuwQAALYQI54AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmey0BV7VpVJ1fVxVV1YVWdXlV7jstuXVWXVdXrJvq/qarOr6ovVNW7q2qHxaseAABY7gTPJa6qKskpSc5qre3eWrt7kpck2WXs8pdJzp612nNba/dore2b5L+SPHuLFQwAAGx1Vix2Adxohya5urV2/ExDa+28JKmq/TME0A8lWTWx/Afj8kpyyyRtSxYMAABsXYx4Ln17Jzl3dmNVbZPk75O8YNpKVfWWJN9IsleS187R58iqWlNVazZcsX7zVQwAAGxVBM/l66gkp7fW/nvawtba7yX55SQXJTlsjj4ntNZWtdZWbbv9jv0qBQAAljXBc+m7IMn+U9rvl+TZVbUuyd8l+d2qOnayQ2ttQ5J3JHls7yIBAICtl+C59J2ZZLuqOmKmoaoOSHJCa+1OrbWVSZ6f5K2ttaNrsMfYr5L8VpL/WIS6AQCArYTgucS11lqSRyd56Hg7lQuSHJPk8jlWqSQnVdXaJGuT3D7JX2yJWgEAgK2Tq9ouA621y5M8YZ7lJyY5cXx9bZIDt0hhAAAAMeIJAABAZ4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXa1Y7AJYGvbZbcccd9TqxS4DAABYgox4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQ1YrFLoClYe1l67Py6NMWuww6WXfs6sUuAQCAZcyIJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgucyUFW7VtXJVXVxVV1YVadX1Z5VtaGqzhsfp05Z77VV9aPFqBkAANh6rFjsArhxqqqSnJLkpNbab49t+yXZJcmVrbX95lhvVZLbbLFCAQCArZbgufQdmuTq1trxMw2ttfOSZMik11dV2yZ5ZZInJnn0FqgRAADYiplqu/TtneTcOZbdoqrWVNW/V9WjJtqfneTU1trX59twVR05rr9mwxXrN1e9AADAVkbwXN7u1FpblWFk8x+qaveq+uUkj0/y2o2t3Fo7obW2qrW2atvtd+xdKwAAsEwJnkvfBUn2n7agtXb5+HxJkrOS3HN87JHkq1W1Lsn2VfXVLVIpAACwVRI8l74zk2xXVUfMNFTVAVV1cFVtN77fKcmBSS5srZ3WWtu1tbaytbYyyRWttT0WpXIAAGCrIHguca21luECQQ8db6dyQZJjxsVrqur8JB9Lcmxr7cJFKhMAANiKuartMjBOqX3ClEX7LGDdHTZ/RQAAANcx4gkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXKxa7AJaGfXbbMccdtXqxywAAAJYgI54AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHS1YrELYGlYe9n6rDz6tMUugwVad+zqxS4BAAB+xognAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeC5zJQVbtW1clVdXFVXVhVp1fVnlX1oar6flV9YFb/qqq/qqovV9VFVfWcxaodAABY/lYsdgHcOFVVSU5JclJr7bfHtv2S7JLklUm2T/KMWasdnuSOSfZqrV1bVbfbchUDAABbG8Fz6Ts0ydWtteNnGlpr5828rqpDpqzzrCRPbK1dO/b/Vu8iAQCArZeptkvf3knO3cR1dk9yWFWtqaoPVtVdp3WqqiPHPms2XLH+RhcKAABsnQTPrdN2SX7SWluV5A1J3jytU2vthNbaqtbaqm2333GLFggAACwfgufSd0GS/TdxnUuTvGd8fUqSfTdrRQAAABMEz6XvzCTbVdURMw1VdUBVHTzPOu9L8qDx9cFJvtyxPgAAYCsneC5xrbWW5NFJHjreTuWCJMckubyqzknyriQPrqpLq+ph42rHJnlsVa1N8ookT1+E0gEAgK2Eq9ouA621y5M8Ycqig+bo//0kq7sWBQAAMDLiCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF2tWOwCWBr22W3HHHfU6sUuAwAAWIKMeAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0NWKxS6ApWHtZeuz8ujTFrsM5rDu2NWLXQIAAMzJiCcAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleC5DFTVrlV1clVdXFUXVtXpVbVnVf1tVV1QVRdV1Wuqqsb+z66qr1ZVq6qdFrt+AABgeRM8l7gxTJ6S5KzW2u6ttbsneUmS2yc5MMm+SfZOckCSg8fV/i3JQ5J8bctXDAAAbG0Ez6Xv0CRXt9aOn2lorZ2X5KdJbpHk5km2S3KzJN8cl3++tbZuy5cKAABsjQTPpW/vJOfObmytfSrJx5J8fXz8a2vtok3ZcFUdWVVrqmrNhivWb5ZiAQCArY/guUxV1R5J7pbkDsn/b+/uQi2ryzCAPy/OFMmERkLEWChhH0YO5FheVJoRqUEiVPRBgkQf2NfldFWBXtRFEFIqIiLd5EVJKWQRRBnYUAk6NokxKdhgIFNRjV3E6NvFPiOn4czMGjr/vWcdfz+Yi73XYnguHvZez1n77JOdSa6oqnedyv/R3bd39+7u3n3GmWeNiAkAALwIGJ7ztz/JxRs8f22Svd19uLsPJ7k/yaVLTQYAABDDcyv4eZKXVtWnjj5RVZckOTPJZVW1raq2Z/HFQqf0UVsAAIDNYHjOXHd3Fnc337v251T2J/lakruT/CnJo0keSfJId9+XJFX1xao6mMXHcPdV1R0rCQ8AALwobFt1AP5/3f10kg9vcOgzxzn/5iQ3Dw0FAACwxh1PAAAAhjI8AQAAGMrwBAAAYCjDEwAAgKEMTwAAAIYyPAEAABjK8AQAAGAowxMAAIChDE8AAACGMjwBAAAYyvAEAABgKMMTAACAoQxPAAAAhjI8AQAAGGrbqgMwD2/ZeVZuveH9q44BAADMkDueAAAADGV4AgAAMJThCQAAwFCGJwAAAEMZngAAAAxleAIAADCU4QkAAMBQhicAAABDGZ4AAAAMZXgCAAAwlOEJAADAUIYnAAAAQxmeAAAADGV4AgAAMJThCQAAwFCGJwAAAEMZngAAAAxleAIAADCU4QkAAMBQhicAAABDGZ4AAAAMZXgCAAAwlOEJAADAUIYnAAAAQxmeAAAADGV4AgAAMJThCQAAwFCGJwAAAEMZngAAAAxV3b3qDMzAnj17/rV9+/bHV52DrePw4cPn7Nix49Cqc7B16BSbSZ/YbDrFZjtNO3XopptuunKjA4Ynk1TV77p796pzsHXoFJtNp9hM+sRm0yk229w65aO2AAAADGV4AgAAMJThyVS3rzoAW45Osdl0is2kT2w2nWKzzapTfscTAACAodzxBAAAYCjDEwAAgKEMT/5HVV1ZVY9X1YGq+vIGx6uqbl47vq+q3rqKnMzHhE59fK1L+6rqwaratYqczMPJ+rTuvEuq6rmq+uAy8zE/UzpVVZdX1cNVtb+qfrnsjMzLhPe9s6rqvqp6ZK1T168iJ/NQVXdW1TNV9fvjHJ/NtbnhyQuq6owk30lyVZILk3y0qi485rSrklyw9u/TSW5dakhmZWKnnkxyWXdflOTGzOwX5VmeiX06et43kvx0uQmZmymdqqqzk9yS5APd/eYkH1p6UGZj4uvU55L8obt3Jbk8yTer6iVLDcqc3JXkyhMcn821ueHJem9LcqC7n+ju/yS5O8k1x5xzTZLv9sLeJGdX1auXHZTZOGmnuvvB7v772sO9Sc5dckbmY8prVJJ8IckPkjyzzHDM0pROfSzJPd39VJJ0t15xIlM61UleXlWVZEeSvyU5styYzEV3P5BFR45nNtfmhifr7Uzy53WPD649d6rnwFGn2pdPJrl/aCLm7KR9qqqdSa5NctsSczFfU16jXp/kFVX1i6p6qKquW1o65mhKp76d5E1Jnk7yaJIvdffzy4nHFjSba/Ntqw7AaaU2eO7Yv7cz5Rw4anJfqurdWQzPdwxNxJxN6dO3kuzp7ucWNxPghKZ0aluSi5O8J8nLkvy6qvZ29x9Hh2OWpnTqfUkeTnJFktcl+VlV/aq7/zk6HFvSbK7NDU/WO5jkNesen5vFT+NO9Rw4alJfquqiJHckuaq7/7qkbMzPlD7tTnL32ug8J8nVVXWku3+4nIjMzNT3vUPd/WySZ6vqgSS7khiebGRKp65P8vXu7iQHqurJJG9M8pvlRGSLmc21uY/ast5vk1xQVeev/ZL7R5Lce8w59ya5bu0btC5N8o/u/suygzIbJ+1UVb02yT1JPuEOAidx0j519/ndfV53n5fk+0luMDo5gSnvez9K8s6q2lZVZyZ5e5LHlpyT+ZjSqaeyuIOeqnpVkjckeWKpKdlKZnNt7o4nL+juI1X1+Sy+CfKMJHd29/6q+uza8duS/DjJ1UkOJPl3Fj+1gw1N7NRXkrwyyS1rd6mOdPfuVWXm9DWxTzDZlE5192NV9ZMk+5I8n+SO7t7wzxrAxNepG5PcVVWPZvExyT3dfWhloTmtVdX3svj243Oq6mCSrybZnszv2rwWd/kBAABgDB+1BQAAYCjDEwAAgKEMTwAAAIYyPAEAABjK8AQAAGAowxMAAIChDE8AAACG+i/gJt9d6D5/IwAAAABJRU5ErkJggg==\n","text/plain":"<Figure size 1008x720 with 1 Axes>"},"metadata":{"needs_background":"light"},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":"PartialDependencePlot progress: |█████████████████████████████████████████| 100%\n"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAf4AAALWCAYAAACnePHjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeZwcdZ3/8deHJASSAAETzrCAEMJGuSQGXUQmggIei+66CrqsoCxeeP7wxAPUddVVXF1FFg/wxnghAppBlgAusCCXJBxmwk04RRISCJDk+/ujakxnpifphKmpqfm+no9HP7q6vtXVn293zby7zo6UEpIkKQ8b1V2AJEkaOga/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfz1hEfDQivtXhtGdFxGeqrmkoRMQdEXFI3XV0YiS972sTETtHRIqI0R1O/5qIuDsilkbEvuuYtisi7hmcSqX6GPwZKAPqifKf2wMRcWZETNjAefX755dS+mxK6bhBqPOYiFhZ1rk0Im4va939mc47BxExNyKWl+/dwxHxi4jYbgPmkyJit7W09/2clkbE155Z9X+d98kR8YP1rekZ+CJwQkppQkrpugrmPyhavtCc32f8DyLi5JrKWi8VfoZaTwZ/Pl6VUpoAPA94PvCx9Z1Bp2tRz9AVZZ1bAIcATwDXRMRzh+C1R4ITyvdvd2Ai8OWKXueKMix7byes7wyGaHlal52A+XUXsR5eEBEH1F1EX1V/lsNkWRkxDP7MpJTuBX4DPBcgIo6NiJsj4rGIuC0i3to7be/afUR8KCLuB35cPnf7ljW97fuupUXETyPi/ohYHBGXRsRzNqDOlSmlhSmldwCXACe3zP8FEXF5RDwaETdERFdL29yI+PeIuKp8/V9FxFbr8dxPR8T/lu9Hd0RMamk/OiLujIg/R8RJrfVGxEYR8eGIWFi2z+593Za1tTdFxF3l2vhJLc8dVe4uWVi+7jURsWPZtkdEXBgRj0TErRHxug7fv0eAn1N+zn1FxL9GRE8533MjYvty/KXlJDeUn+/rO3m9lvluERHfi4iHyvfqYxGxUdl2TPnefjkiHqHlM13P1xjwvW4zbdvlISLGRsRSYFTZ14Xl9GuslcZadpFEsSXtxIj4Yznvn0TEJi3tr4yI68tl7fKI2Kul7UMRcW/5ed8aEQeX42dGxB8iYkkUW+dO7fOyXwAGqueYiPh9n3F/7U/Zl9Mi4jflZ/u/EbFtRPxnRPwlIm6Jlt0dUfxt/7z8LG+PiHe3tJ0cET+LYovDEuCYsvYryv7eFxFfi4iNy+nbLlcDLYcttb8zIhYAC9r1WRvG4M9MGSgvB3o3az4IvBLYHDgW+HJEPK/lKdsCW1GsGf0LcDiwqGVNb1Gbl/kNMBXYGrgW+OEzLPsXwIFl/TsA51P889sKOBH4eURMbpn+X4A3A9sDK4Cvrsdz30DxPmwNbFxOQ0RMB74BHF3O91nAlJbnvRt4NXBQ2f4X4Ot9+vEiYBpwMPCJiPjbcvz7gaMoPpfNy9ofj4jxwIXAj8p6jgJOiw6+SEXxheUfWf05t7a9BPh34HXAdsCdwNkAKaUXl5PtXX6+P1nXa/XxXxRba55N8V78C8X72Wt/4LayP/+2nvPu1cl73arf8pBSerLcMgJFX3fdwFpeBxwG7ALsBRwDUP4NfQd4K8Wy8t/AueUXjmnACcDzU0qbAYcCd5Tz+wrwlZTS5sCuwOw+r/d1YPfY8GNLXkextW8S8CRwBcXf6CTgZ8CpZf0bAb8GbgB2oFhm3xsRh7bM64jyORMp/sZXAu8r5/XC8jnvgPbL1dqWwxavplhmpm9gf9VOSsnbCL9R/FNZCjxK8cd1GrDpANOeA7ynHO4CngI2aWnvAu7p85yTgR8MML+JQAK2KB+fBXxmgGmPAX7fZvxhwNPl8IeA7/dpnwO8qRyeC3yupW162YdRHT73Yy1t7wB+Ww5/Aji7pW18Od9Dysc3Awe3tG8HPA2MBnYu34MpLe1XAUeWw7cCR7Tp9+uBy/qM+2/gkwO8f3OBx8vP+V6Kf8aT+77vwLeBL7Q8b0JZ687l4wTstpbl6RiKAH205faC8j1+EpjeMu1bgbktz7trHcvqyeX7+mif219r6vC9Hr2u5aFdX9s8bn3fumhZ9in+rv655fEXgNPL4W8An+7Tt1spvqzsRvGF+xBgTJ9pLgVOASb1Gf/XflEsl1eW438AnDzQ30+f9+0s4Jstbe8Cbm55vCfwaDm8f9/PCvgIcGbL53TpOj7L9wK/XMt728ly+JK1vYa3Dbu5xp+PV6eUJqaUdkopvSOl9ARARBweEVeWm9oepVjrnNTyvIdSSss7fZEoNlt/rtwMu4TVazKT1vK0ddkBeKQc3gn4p3Jz4qNlzS+i+Off6+6W4TuBMeXrd/Lc+1uGH6f4ZwTF2uJf55tSWgb8uWXanYBftsz3Zoo1oG06mPeOwMI2/d4J2L9PvW+k2AozkHeXn/MOKaU3ppQeajPN9hTvS29flpZ92WEt8+3ryvJ1em9XUrzHG7fOuxxunW/rZzOQ2X3mPbFPeyfvdauBlofBMNBnuhPw//p8djsC26eUeihC8WTgwYg4u2UT91sojs+4JSKujohXtnnNbwLbRMSrNqDeB1qGn2jzuLX+7fvU/1HWfI/X+CwjYveIOC+K3XxLgM+y9ve5k+Wwk+VF68ngz1hEjKXYD/xFYJvyH+wFQLRM1vfnG9f1c45voNgEeAjFJt+de1/uGZT6GuCycvhuirX21mAYn1L6XMv0O7YM/w3FWsTDHT53IPe1zjcixlFswu11N3B4n3lvkopjKtblborNuu3GX9JnnhNSSm/vYJ5rs4jiHzsA5S6FZ1FsJXgmHqZ4r3dqGfc3feY7GD8Hur7v9UDLQzuPA+NaHq/tS9a6avy3PjWOSyn9GCCl9KOU0oso3qsEfL4cvyCldBTFrpDPAz8rP5+/Sik9TbFV4NOs+Xe1rLX2iNjQ2nvrv71P/ZullF7eWkqf53wDuAWYmopdFR9l7X/3nSyH/nxsBQz+vG0MjAUeAlZExOHAy9bxnAeAZ0XEFgO0b0axuffPFP+EPrshhZVbDnaJiP+i2MR6Stn0A+BVEXFoOc0mURyE2Lq//Z8jYnoZzp8CfpZSWtnhcwfyM+CVEfGi8oClT7Hm38/pwL9FxE5l/ZMj4ogOu/st4NMRMTUKe0XEs4DzKPbnHh0RY8rb81uODdhQPwKOjYh9yi9/nwX+L6V0R9n+AMU++vVSvsezKd6Hzcr34v0U7/tgWt/3eqDloZ3rgTeUy8dhFJvmN8Q3gbdFxP7lZzo+Il5Rvi/TIuIl5Xu/nGJNe2XZl3+OiMkppVUUuzjobevj+xR/u4e1jLsBeE75uW7CBh48WboKWBLFQYiblu/HcyPi+Wt5zmbAEmBpROwB9P2C2ne5WtdyqIoY/BlLKT1GcaDUbIoDpN4AnLuO59xCcXT/beUmwO37TPI9is139wI3AVeuZ1kvjOJo6yUU+2c3pzgI6sby9e+m2KLwUYovLHcDH2DNZfn7FPsz7wc2KfvY6XPbSinNB95J8c/qPor3q/V6Bl+heO+6I+Kxst/7d9jnUyk+g+6y39+mOAbjMYovYkdSrB3dT7EWOLbD+Q7Ul4uAj1Ns7bmPYmvDkS2TnAx8t/x8OzqLoMW7KNY8bwN+T/F+feeZ1NvG+r7XbZeHAbwHeBVF6L6R4piX9ZZS+gPwr8DXKJaVHsoD/yg+v89RbHW4n2Lt/qNl22HA/PJv4CsUx4H029VWfnH5JMVBqr3j/kTxxeZ3FEfB/77v89aj/pUU78M+wO1lrd+i2Io3kBMp/oc8RvHFp++BoSfTslx1sByqIpGSW1I0ckTEXIoDDTu6kqBGNpcHqT/X+CVJyojBL0lSRtzUL0lSRlzjlyQpIwa/JEkZadwvHk2cODHtttvI+mXHZcuWMX78+HVP2CD2qRnsUzPYp2YYTn265pprHk4pTW7X1rjg32abbfjDH/5QdxmDau7cuXR1ddVdxqCyT81gn5rBPjXDcOpTRNw5UJub+iVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUkcqCPyK+ExEPRsS8AdojIr4aET0R8ceIeF5VtUiSpEKVa/xnAYetpf1wYGp5Ox74RoW1SJIkKgz+lNKlwCNrmeQI4HupcCUwMSK2q6oeSZJU7z7+HYC7Wx7fU46TJEkViZRSdTOP2Bk4L6X03DZt5wP/nlL6ffn4IuCDKaVr2kx7PMXuACZPnrzf7NmzK6u5DkuXLmXChAl1lzGo7FMz2KdmsE/NMJz6NGvWrGtSSjPatY0e6mJa3APs2PJ4CrCo3YQppTOAMwCmTZuWurq6Ki9uKM2dOxf7NPzZp2awT81gn+pT56b+c4F/KY/ufwGwOKV0X431SJI04lW2xh8RPwa6gEkRcQ/wSWAMQErpdOAC4OVAD/A4cGxVtUiSpEJlwZ9SOmod7Ql4Z1WvL0mS+vPKfZIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlJHRdRcwnDz5JNxyC6xatXrc1lvDDjvAypXwxz/2f86228J228FTT8H8+f3bd9ihmMfy5XDzzf3bd9yxuF+2DP70p/7tO+8MW24Jjz0GPT3925/9bNhiC3j0Ubj99v7tU6fChAnwyCNw553926dNg3Hj4KGH4J57+rdPnw5jx8IDD8CiRf3bn/tcGDMG7rsP7r9/9fgFCyawxRaw114walQx74ce6v/8ffct7u+6C/785zXbNtoI9t67GL799qKPrcaMKV4fYOFCWLJkzfaxY4v6oXhvly1bs33cuKL/UHzuTzyxZvuECcX7B8Vn29unXltsUbz/ADfeCCtWrPn8LbcsPj+AG25Yc7kCmDSp+PxTguuvpx+XPZc9cNlz2VuzfX2WvQGllBp123333dNgWrGiuF+8OKXNN0+p+FNYfTvxxKL90Uf7t0FKp5xStN99d/v2U08t2m+6qX37N7+Z0sUXX5yuuqp9+9lnF8+/6KL27eedV7Sfc0779ksuKdq///327ddeW7Sfdlr79gULivYvfKF9+/33F+0f/3j79qVLi/b3vrd/W8Tqz+Etb+nfvvnmq9tf97r+7TvssLr95S/v377HHqvbDzywf/uMGavb99mnf/usWavbd921f/vf//3q9q237t/+xjeubh83rn/7295WtK1c2f69c9lz2XPZc9nb0GUP+ENK7XM0uzX+lIpvqL/+dXGbNAnOPRc23xw++EHYdVfYdNPV0++2W3E/bhycc07/+e2xR3H/rGe1b+/9ZjZlSvv2ffYpvtXttlv79hkzivs992zfvt9+xf3Mme3be7/5HXRQ+/ZddinuDzusffu22xb3r3417L57//betZCjjlpdC8CNN97InnvuydixxeM3vxm6uvo/v9c73wmvetWa40a3LJ0nnghveMOa7Ztssnr44x+H449fs33ChNXDn/1s/2/WEyeuHv7yl2Hx4jXbJ01aPXz66XDllUWfevW+NwDf/W6xxajVlCmrh3/yk2LtqVXvGllE+/feZa+4d9lz2evLZa+wrmVvQAN9Ixiut2eyxn/qqSntvPPqb0YzZqT0uc9t8OwGzcUXX1x3CYPOPjWDfWoG+9QMw6lP5LjG/+CDcP758JvfwJlnwvjxxbfePfeEj34UXvEK2H77uquUJGlojajgX7SoCPlf/xquuqpYr58ypTgAYq+9is0mJ55Yd5WSJNWn0afzLV8Ov/1tcdQqFEdgfuxjReCfcgpcd11x1ORee9VbpyRJw0Xj1vhXrgy+/W047zy48MLidJC3vrU4AGaffYrTK1oPfJEkSas1LvjvvHM8xx1XnAf6pjfBK18Js2YVbRGGviRJa9O44N9mm+Wcf36x+T6i7mokSWqWxgX/+PEr/npVI0mStH4afXCfJElaPwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGKg3+iDgsIm6NiJ6I+HCb9i0j4pcR8ceIuCoinltlPZIk5a6y4I+IUcDXgcOB6cBRETG9z2QfBa5PKe0F/AvwlarqkSRJ1a7xzwR6Ukq3pZSeAs4GjugzzXTgIoCU0i3AzhGxTYU1SZKUtSqDfwfg7pbH95TjWt0A/ANARMwEdgKmVFiTJElZi5RSNTOO+Cfg0JTSceXjo4GZKaV3tUyzOcXm/X2BG4E9gONSSjf0mdfxwPEAkydP3m/27NmV1FyXpUuXMmHChLrLGFT2qRnsUzPYp2YYTn2aNWvWNSmlGe3aRlf4uvcAO7Y8ngIsap0gpbQEOBYgIgK4vbzRZ7ozgDMApk2blrq6uqqpuCZz587FPg1/9qkZ7FMz2Kf6VLmp/2pgakTsEhEbA0cC57ZOEBETyzaA44BLyy8DkiSpApWt8aeUVkTECcAcYBTwnZTS/Ih4W9l+OvC3wPciYiVwE/CWquqRJEnVbuonpXQBcEGfcae3DF8BTK2yBkmStJpX7pMkKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjo+suYFh59FH4wQ/g6aeH9GWn9PTAddcN6WtWzT41g31qBvvUDE3pk8Hf6rTT4KSThvxldxvyV6yefWoG+9QM9qkZmtIng79VdzfsvTdccsmQvuxll13GgQceOKSvWTX71Az2qRnsUzMMqz5NnDhgk8Hf67HH4PLL4f3vhy22GNKXXjlhwpC/ZtXsUzPYp2awT83QlD55cF+vSy4p9u2/7GV1VyJJUmUqDf6IOCwibo2Inoj4cJv2LSLi1xFxQ0TMj4hjq6xnrebMgU03hQMOqK0ESZKqVlnwR8Qo4OvA4cB04KiImN5nsncCN6WU9ga6gC9FxMZV1bRW3d3Q1QVjx9by8pIkDYUq1/hnAj0ppdtSSk8BZwNH9JkmAZtFRAATgEeAFRXW1N4dd8Cf/uRmfknSiBcppWpmHPFa4LCU0nHl46OB/VNKJ7RMsxlwLrAHsBnw+pTS+W3mdTxwPMDkyZP3mz179qDWut155zHtS1/iqjPP5PGddx7UeXdi6dKlTJgwYchft0r2qRnsUzPYp2YYTn2aNWvWNSmlGe3aqjyqP9qM6/st41DgeuAlwK7AhRFxWUppyRpPSukM4AyAadOmpa6ursGt9OtfhylTmPmmN0G0K7tac+fOZdD7VDP71Az2qRnsUzM0pU9Vbuq/B9ix5fEUYFGfaY4FfpEKPcDtFGv/Q2flSvjd74rN/DWEviRJQ6nK4L8amBoRu5QH7B1JsVm/1V3AwQARsQ0wDbitwpraVHl1cale9+9LkjJQ2ab+lNKKiDgBmAOMAr6TUpofEW8r208HPg2cFRE3Uuwa+FBK6eGqamqru7tY0z/44CF9WUmS6lDplftSShcAF/QZd3rL8CKg3lXt7m7Ybz+YNKnWMiRJGgp5X7lv8WK48ko380uSspF38F98cXFw36GH1l2JJElDIu/g7+6GCRPgBS+ouxJJkoZE3sE/Zw7MmgUb13OVYEmShlq+wb9wIdx2m/v3JUlZyTf4u7uLe4NfkpSRvIN/p51g6tS6K5EkacjkGfxPPw3/8z/F0fxepleSlJE8g/+qq2DJEjfzS5Kyk2fwz5kDG20EL3lJ3ZVIkjSkOgr+iNg0IqZVXcyQ6e6GmTNhyy3rrkSSpCG1zuCPiFcB1wO/LR/vExF9f2WvOR55pPhFPjfzS5Iy1Mka/8nATOBRgJTS9cDO1ZVUsf/5H1i1yuCXJGWpk+BfkVJaXHklQ6W7GzbfHPbfv+5KJEkacp38LO+8iHgDMCoipgLvBi6vtqyKpFQE/8EHw+hKf5FYkqRhqZM1/ncBzwGeBH4MLAHeW2VRlfnTn+DOO93ML0nK1jpXe1NKjwMnASdFxChgfEppeeWVVcHL9EqSMtfJUf0/iojNI2I8MB+4NSI+UH1pFejuhl13hWc/u+5KJEmqRSeb+qenlJYArwYuAP4GOLrSqqrw1FNw8cWu7UuSstZJ8I+JiDEUwf+rlNLTQKq2rApccQUsW1Zcn1+SpEx1Evz/DdwBjAcujYidKA7wa5bubhg1CmbNqrsSSZJq08nBfV8Fvtoy6s6IaF56zpkDL3xhcQ6/JEmZWmfwR8RY4B8prtbXOv2nKqpp8D30EFx7LZxySt2VSJJUq06uYvMrYDFwDcW5/M1z0UXFxXs8sE+SlLlOgn9KSumwyiupUnd38Ut8M2bUXYkkSbXq5OC+yyNiz8orqUrvZXoPOaQ4uE+SpIx1ssb/IuCYiLidYlN/ACmltFellQ2Wm2+Ge+91M78kSXQW/IdXXkWV5swp7l/60nrrkCRpGFjnpv6U0p3AROBV5W1iOa4Zurth2jTYaae6K5EkqXadXKv/PcAPga3L2w8i4l1VFzYoli+HSy5xM78kSaVONvW/Bdg/pbQMICI+D1wB/FeVhQ2K//1feOIJg1+SpFInR/UHsLLl8cpy3PDX3Q1jxkBXV92VSJI0LHSyxn8m8H8R8UuKwD8C+HalVQ2W7m444ACYMKHuSiRJGhY6ObjvVOBY4BHgz8CxKaX/rLqwZ+yBB+D6693ML0lSi0429fcKip/jbcZm/gsvLO4NfkmS/qqTo/o/AXwX2BKYBJwZER+rurBnrLsbnvUs2HffuiuRJGnY6GQf/1HAviml5QAR8TngWuAzVRb2jPRepvelL4WN1mejhiRJI1snqXgHsEnL47HAwkqqGSw33ljs4z/00LorkSRpWOlkjf9JYH5EXEixj/+lwO8j4qsAKaV3V1jfhunuLu69TK8kSWvoJPh/Wd56za2mlEE0Zw485zmwww51VyJJ0rCyzuBPKX23dzgitgR2TCn9sdKqnonHH4fLLoN3vKPuSiRJGnY6Oap/bkRsHhFbATdQHNV/avWlbaDLLoMnn/Q0PkmS2ujk4L4tUkpLgH8Azkwp7QccUm1Zz0B3N4wdCy9+cd2VSJI07HQS/KMjYjvgdcB5FdfzzHV3w4EHwrhxdVciSdKw00nwfwqYAyxMKV0dEc8GFlRb1gZatAjmzXMzvyRJA+jk4L6fAj9teXwb8I9VFrXBek/jM/glSWqrk4P7do+IiyJiXvl4r2F7yd7ubthmG9hzz7orkSRpWOpkU/83gY8ATwOUp/IdWWVRG2TVquKHebxMryRJA+okIcellK7qM25FFcU8I9dfDw8/7GZ+SZLWopPgfzgidqW4XC8R8Vrgvkqr2hBepleSpHXq5JK97wTOAPaIiHuB24E3VlrVhujuhr33hm23rbsSSZKGrbUGf0RsBMxIKR0SEeOBjVJKjw1Naeth2TL4/e/hve+tuxJJkoa1tW7qTymtAk4oh5cNy9AHmDsXnn7a/fuSJK1DJ/v4L4yIEyNix4jYqvdWeWXro7sbNtkEXvSiuiuRJGlY62Qf/5vL+3e2jEvAswe/nA3U3Q0HHVSEvyRJGlAnV+7bZW3tEfHSlNKFg1fSerrrLrjlFjj++NpKkCSpKQbjSjefH4R5bLgLy+8c7t+XJGmdBiP4YxDmseG6u2H77WH69FrLkCSpCQYj+Eh3z9oAAB2hSURBVNMgzGPDrFxZrPG/7GUQ9X7/kCSpCZp9UftrroG//MXN/JIkdWgwgv+OQZjHhunuLtb0DzmkthIkSWqSTk7nIyL+Dti5dfqU0vfK+3+opLJOdHfD854HkyfXVoIkSU2yzuCPiO8DuwLXAyvL0Qn4XoV1rduSJXDFFfCBD9RahiRJTdLJGv8MYHpKqb6D+NqZOxdWrHD/viRJ66GTffzzgOH3k3dz5sD48fDCF9ZdiSRJjdHJGv8k4KaIuAp4sndkSunvK6uqE93d0NUFY8fWWoYkSU3SSfCfXHUR6+2226CnB971rrorkSSpUTq5Vv8lQ1HIeum9TO+hh9ZbhyRJDbPOffwR8YKIuDoilkbEUxGxMiKWDEVxA+ruhr/5G9h991rLkCSpaTo5uO9rwFHAAmBT4LhyXH0uusjL9EqStAE6uoBPSqknIkallFYCZ0bE5RXXNaBRTzwBixd7Gp8kSRugk+B/PCI2Bq6PiC8A9wHjqy1rYKMff7xY0z/44LpKkCSpsTrZ1H90Od0JwDJgR+AfqyxqbUYtWwbPfz5stVVdJUiS1FidHNV/Z0RsCmyXUjplCGpaq1HLl3s0vyRJG6iTo/pfRXGd/t+Wj/eJiHOrLmyt3L8vSdIG6WRT/8nATOBRgJTS9RS/1FeLtNFGsP/+db28JEmN1knwr0gpLa68kg49MWUKjBlTdxmSJDVSJ0f1z4uINwCjImIq8G6gttP5Vm6ySV0vLUlS43Wyxv8u4DkUP9DzI2Ax8J4qi5IkSdXoJPinl7fRwCbAEcDVVRYlSZKq0cmm/h8CJwLzgFXVliNJkqrUSfA/lFL6deWVSJKkynUS/J+MiG8BF1Hs5wcgpfSLyqqSJEmV6CT4jwX2AMawelN/Agx+SZIappPg3zultGfllUiSpMp1clT/lRExvfJKJElS5TpZ438R8KaIuJ1iH38AKaW0V6WVSZKkQddJ8B9WeRWSJGlIdPSzvENRiCRJql4n+/glSdIIYfBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjlQZ/RBwWEbdGRE9EfLhN+wci4vryNi8iVkbEVlXWJElSzioL/ogYBXwdOByYDhwVEdNbp0kp/UdKaZ+U0j7AR4BLUkqPVFWTJEm5q3KNfybQk1K6LaX0FHA2cMRapj8K+HGF9UiSlL1IKVUz44jXAoellI4rHx8N7J9SOqHNtOOAe4Dd2q3xR8TxwPEAkydP3m/27NmV1FyXpUuXMmHChLrLGFT2qRnsUzPYp2YYTn2aNWvWNSmlGe3aRlf4utFm3EDfMl4F/O9Am/lTSmcAZwBMmzYtdXV1DUqBw8XcuXOxT8OffWoG+9QM9qk+VW7qvwfYseXxFGDRANMeiZv5JUmqXJXBfzUwNSJ2iYiNKcL93L4TRcQWwEHAryqsRZIkUeGm/pTSiog4AZgDjAK+k1KaHxFvK9tPLyd9DdCdUlpWVS2SJKlQ5T5+UkoXABf0GXd6n8dnAWdVWYckSSp45T5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlZHTdBQwnCxbARz4CTz8NZ54JW20FZ58NP/5x/2nPPhs23bSY7pxz+refcw5EwGmnwZw5a7Ztsgn85CfF8Je+BL/4xXOZNGl1+5ZbwllnFcOf+QxcffWaz99+e/jGN4rhk06CefPWbN91Vzj11GL4/e+HhQvXbH/Oc+Czny2G3/52WLRozfYZM+DjHy+GjzkG/vKXNdsPPBBOPLEYfv3rYfnyNdtf9rLiNQCOOIJ+jjgC3vxmeOIJOPLI/u2vfz284Q3wyCNw7LH92485Bl7zGrj3XnjHO/q3v/3tcNhh0NMD/+//9W9///vhoIOK9+2kk/q3n3QSzJxZvO+f+czq8Q8/XHxOn/kM7LknXHpp8fn19aUvwW67FZ/7aaf1bz/tNNhhh2IZOfPM/u0ue8Wwy97q8S57LnvPdNlrZfC3+MUv4Oc/h733hhUrinGLF8Ndd/WfdtWq4v4vf2nf3uuRR/q3b7rp6uGHH4YHHxzL44+vHtc6/NBDa5//gw/2bx8/fvXwAw/0b2/9Y7vvvv7tO+20enjRoqKGVg8/vHr4nnvWrBeKPvdqV3vvH1RK7dsXLy7uV65s3/7YY8X9ihXt25cuLe6feqp9e2+9Tz7Zvv2JJ1bft7YvXVp8Tr1/8I8/3v75Tz65uo527U8/vbof7dpd9goue63zddlr5bLXv31dy16rSCmtfYphZtq0aenWW2+tZN7HHQe//nWx0AyluXPn0tXVNbQvWjH71Az2qRnsUzMMpz5FxDUppRnt2tzH36KnB6ZOrbsKSZKq46b+FhMnws47112FJEnVMfhbtDtYRZKkkcRN/ZIkZcTgL51/Puy3H9xxR92VSJJUHYO/NG8eXHttcQ6rJEkjlcFfWrAAtt4aNt+87kokSaqOwV/yVD5JUg4M/tKCBQa/JGnkM/gpLpF4wAHFtZglSRrJPI8fGDUKZs+uuwpJkqrnGj+rf3hCkqSRzuAH/uM/YJttVv8yliRJI5XBT3FgX8SaPxspSdJIZPBTBP9uu9VdhSRJ1TP48Rx+SVI+sg/+Zctg0SLX+CVJecg++J96Ct73PjjooLorkSSpetmfx7/llnDqqXVXIUnS0Mh+jf+RR2D58rqrkCRpaGQf/B/8IOyyS91VSJI0NLIP/gULYNdd665CkqShkX3weyqfJCknWQd/76l8Br8kKRdZB//ChcW95/BLknKRdfBPnlycyrf//nVXIknS0Mj6PP7ttisu3iNJUi6yXuOfPx/uuKPuKiRJGjpZr/G/4x2wahVcdlndlUiSNDSyXuPv6fHAPklSXrINfk/lkyTlKNvg7+kp7l3jlyTlJPvgd41fkpSTbIP/gAPgpz+FadPqrkSSpKGT7VH9224Lr31t3VVIkjS0sl3j//Wv4brr6q5CkqShlW3wv/Wt8NWv1l2FJElDK8vgX7YM7rvPA/skSfnJMvg9lU+SlKusg981fklSbrIM/gULinvX+CVJucky+N/+drjqKthss7orkSRpaGUZ/FtsAc9/ft1VSJI09LIM/i98Aa64ou4qJEkaetkF/7Jl8KEPwcUX112JJElDL7vg94h+SVLOsgt+j+iXJOUsu+D34j2SpJxlF/y33w7bbOOpfJKkPGUX/KefDvPm1V2FJEn1yC74I2DSpLqrkCSpHlkF/7Jl8OY3w+WX112JJEn1yCr4e3rgzDPh3nvrrkSSpHpkFfyeyidJyl1Wwe+pfJKk3GUV/AsWwLbbeiqfJClfWQX/8uXwt39bdxWSJNVndN0FDKUf/hBSqrsKSZLqk9UaPxTn8UuSlKtsgv/mm+GlL4Vrr627EkmS6pNN8M+fD7/7HWyUTY8lSeovmxjsPZVv113rrUOSpDplE/yeyidJUmbB74V7JEm5y+Z0vu23h512qrsKSZLqlU3wn3123RVIklS/bDb1S5KkTIL/V78qLtV7++11VyJJUr2yCP6bboJbboFJk+quRJKkemUR/D09nsonSRJkEvyeyidJUiGL4O/pgalT665CkqT6jfjgX7UKDj0UXvKSuiuRJKl+I/48/o02gjPPrLsKSZKGhxG/xv/005BS3VVIkjQ8jPjg/+IXYfPN4fHH665EkqT6jfjg7+mB8eNh3Li6K5EkqX4jPvgXLPCIfkmSelUa/BFxWETcGhE9EfHhAabpiojrI2J+RFwy2DV4Dr8kSatVdlR/RIwCvg68FLgHuDoizk0p3dQyzUTgNOCwlNJdEbH1YNawdCncf79r/JIk9apyjX8m0JNSui2l9BRwNnBEn2neAPwipXQXQErpwcEsYMUK+NjHYNaswZyrJEnNVeV5/DsAd7c8vgfYv880uwNjImIusBnwlZTS9wargIkT4dOfHqy5SZLUfJEqOsk9Iv4JODSldFz5+GhgZkrpXS3TfA2YARwMbApcAbwipfSnPvM6HjgeYPLkyfvNnj27oxoeeWRjxoxZxWabrRiEHlVn6dKlTJgwoe4yBpV9agb71Az2qRmGU59mzZp1TUppRru2Ktf47wF2bHk8BVjUZpqHU0rLgGURcSmwN7BG8KeUzgDOAJg2bVrq6urqqIA3vxkuuKDYzz+czZ07l0771BT2qRnsUzPYp2ZoSp+q3Md/NTA1InaJiI2BI4Fz+0zzK+DAiBgdEeModgXcPFgF+OM8kiStqbLgTymtAE4A5lCE+eyU0vyIeFtEvK2c5mbgt8AfgauAb6WU5g1WDZ7KJ0nSmir9kZ6U0gXABX3Gnd7n8X8A/zHYr+2pfJIk9Tdir9zX01PcG/ySJK02YoN/u+3g9NPhBS+ouxJJkoaPSjf112mbbeCtb627CkmShpcRu8Z/7bVw86CdHyBJ0sgwYtf43/c+WLUKLrus7kokSRo+Ruwavz/HK0lSfyMy+Jctg/vu8xx+SZL6GpHB76l8kiS1NyKDf8GC4t41fkmS1jQig7+rC84/H/bYo+5KJEkaXkbkUf2TJsHLX153FZIkDT8jco3/pz+FK6+suwpJkoafERn873kPnHFG3VVIkjT8jLjgX7rUU/kkSRrIiAv+hQuLe0/lkySpvxEX/J7KJ0nSwEZc8PdevMfglySpvxEX/O9+N8ybB5ttVnclkiQNPyMu+MeNg+c8p+4qJEkankZc8J9yClxySd1VSJI0PI2o4F+6FE4+GS6/vO5KJEkankZU8PeeyueBfZIktTeigr/3VD7P4Zckqb0RGfy77lpvHZIkDVcjKvjvvRe23dZT+SRJGsiICv6vfW31Wr8kSepvRAU/wIQJdVcgSdLwNWKCf+lSOPJIuPTSuiuRJGn4GjHB39MDP/kJPPBA3ZVIkjR8jajgB0/lkyRpbUZM8HsqnyRJ6zZigr+nx1P5JElalxET/AD77lt3BZIkDW+j6y5gsHz723VXIEnS8Dei1vglSdLajYjgnz8fDjgArr667kokSRreRkTw33wzXH45jBlTdyWSJA1vIyL4e0/l2223euuQJGm4GxHB33sqn9fplyRp7UZE8C9Y4BX7JEnqxIg4nW/XXWHKlLqrkCRp+BsRwX/mmXVXIElSM4yITf2SJKkzjQ/+n/8cdtoJFi6suxJJkoa/xgf/n/4Ed90F22xTdyWSJA1/jQ/+BQs8lU+SpE41Pvh7ejyVT5KkTjU++Bcs8Ip9kiR1qtGn861aBf/wD3DggXVXIklSMzQ6+DfaCL7+9bqrkCSpORq9qf/xx2HFirqrkCSpORod/F/5CowbV3wBkCRJ69bo4F+wACZNKsJfkiStW6ODv6fHI/olSVofjQ5+f45XkqT109jgf+wxuP9+1/glSVofjQ3+Vavgs5+FQw6puxJJkpqjsefxb7EFfOQjdVchSVKzNHaN/+67YdGiuquQJKlZGhv8n/wkzJhRdxWSJDVLY4PfH+eRJGn9NTb4/TleSZLWXyOD31P5JEnaMI0M/oULi3vX+CVJWj+NDP4pU+B734O/+7u6K5EkqVkaeR7/pElw9NF1VyFJUvM0co3/iivguuvqrkKSpOZp5Br/Bz8IEXDppXVXIklSszRyjd9T+SRJ2jCNC/5Vq4L77zf4JUnaEI0L/qefLkr2HH5JktZf44L/qacCcI1fkqQN0bjgHz9+JRdfDNOm1V2JJEnN07ij+jfaKNHVVXcVkiQ1U+PW+JcsGcMll9RdhSRJzdS44H/oobF873t1VyFJUjM1LvhXrgwP7JMkaQM1LvjBU/kkSdpQjQx+1/glSdowjQx+1/glSdowjQv+XXZZxvjxdVchSVIzNS74x4xZVXcJkiQ1VuOCX5IkbTiDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSMGvyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjlQZ/RBwWEbdGRE9EfLhNe1dELI6I68vbJ6qsR5Kk3I2uasYRMQr4OvBS4B7g6og4N6V0U59JL0spvbKqOiRJ0mpVrvHPBHpSSrellJ4CzgaOqPD1JEnSOlQZ/DsAd7c8vqcc19cLI+KGiPhNRDynwnokScpepJSqmXHEPwGHppSOKx8fDcxMKb2rZZrNgVUppaUR8XLgKymlqW3mdTxwPMDkyZP3mz17diU112Xp0qVMmDCh7jIGlX1qBvvUDPapGYZTn2bNmnVNSmlGu7bK9vFTrOHv2PJ4CrCodYKU0pKW4Qsi4rSImJRSerjPdGcAZwBMmzYtdXV1VVZ0HebOnYt9Gv7sUzPYp2awT/WpclP/1cDUiNglIjYGjgTObZ0gIraNiCiHZ5b1/LnCmiRJylpla/wppRURcQIwBxgFfCelND8i3la2nw68Fnh7RKwAngCOTFXte5AkSZVu6ieldAFwQZ9xp7cMfw34WpU1SJKk1bxynyRJGTH4JUnKiMEvSVJGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXJCkjBr8kSRkx+CVJyojBL0lSRgx+SZIyYvBLkpQRg1+SpIwY/JIkZcTglyQpIwa/JEkZMfglScqIwS9JUkYMfkmSMmLwS5KUEYNfkqSMGPySJGXE4JckKSORUqq7hvUSEY8Bt9ZdxyCbBDxcdxGDzD41g31qBvvUDMOpTzullCa3axg91JUMgltTSjPqLmIwRcQf7NPwZ5+awT41g32qj5v6JUnKiMEvSVJGmhj8Z9RdQAXsUzPYp2awT81gn2rSuIP7JEnShmviGr8kSdpAjQr+iDgsIm6NiJ6I+HDd9XQqIr4TEQ9GxLyWcVtFxIURsaC837Kl7SNlH2+NiEPrqXpgEbFjRFwcETdHxPyIeE85vsl92iQiroqIG8o+nVKOb2yfekXEqIi4LiLOKx83uk8RcUdE3BgR10fEH8pxTe/TxIj4WUTcUv5dvbDJfYqIaeXn03tbEhHvbXKfACLifeX/h3kR8ePy/0bz+pRSasQNGAUsBJ4NbAzcAEyvu64Oa38x8DxgXsu4LwAfLoc/DHy+HJ5e9m0ssEvZ51F196FPf7YDnlcObwb8qay7yX0KYEI5PAb4P+AFTe5TS9/eD/wIOK/py15Z5x3ApD7jmt6n7wLHlcMbAxOb3qeWvo0C7gd2anKfgB2A24FNy8ezgWOa2KcmrfHPBHpSSrellJ4CzgaOqLmmjqSULgUe6TP6CIo/dsr7V7eMPzul9GRK6Xagh6Lvw0ZK6b6U0rXl8GPAzRR/FE3uU0opLS0fjilviQb3CSAipgCvAL7VMrrRfRpAY/sUEZtTrBx8GyCl9FRK6VEa3Kc+DgYWppTupPl9Gg1sGhGjgXHAIhrYpyYF/w7A3S2P7ynHNdU2KaX7oAhSYOtyfKP6GRE7A/tSrCE3uk/lJvHrgQeBC1NKje8T8J/AB4FVLeOa3qcEdEfENRFxfDmuyX16NvAQcGa5S+ZbETGeZvep1ZHAj8vhxvYppXQv8EXgLuA+YHFKqZsG9qlJwR9txo3EUxIa08+ImAD8HHhvSmnJ2iZtM27Y9SmltDKltA8wBZgZEc9dy+TDvk8R8UrgwZTSNZ0+pc24YdWn0gEppecBhwPvjIgXr2XaJvRpNMWuwG+klPYFllFsMh5IE/oEQERsDPw98NN1Tdpm3LDqU7nv/giKzfbbA+Mj4p/X9pQ244ZFn5oU/PcAO7Y8nkKxmaWpHoiI7QDK+wfL8Y3oZ0SMoQj9H6aUflGObnSfepWbWecCh9HsPh0A/H1E3EGxa+wlEfEDmt0nUkqLyvsHgV9SbD5tcp/uAe4ptzAB/Izii0CT+9TrcODalNID5eMm9+kQ4PaU0kMppaeBXwB/RwP71KTgvxqYGhG7lN8ijwTOrbmmZ+Jc4E3l8JuAX7WMPzIixkbELsBU4Koa6htQRATF/sibU0qntjQ1uU+TI2JiObwpxR/5LTS4Tymlj6SUpqSUdqb4e/mflNI/0+A+RcT4iNisdxh4GTCPBvcppXQ/cHdETCtHHQzcRIP71OIoVm/mh2b36S7gBRExrvwfeDDF8U3N61PdRxeuzw14OcUR5AuBk+quZz3q/jHFPqGnKb4FvgV4FnARsKC836pl+pPKPt4KHF53/W368yKKTVZ/BK4vby9veJ/2Aq4r+zQP+EQ5vrF96tO/LlYf1d/YPlHsD7+hvM3v/T/Q5D6VNe4D/KFc/s4BthwBfRoH/BnYomVc0/t0CsUKwTzg+xRH7DeuT165T5KkjDRpU78kSXqGDH5JkjJi8EuSlBGDX5KkjBj8kiRlxOCXhkhE7Bwtv9DYwfQnR8SJ65hmbET8rvwFtNevZbpjIuJr61PvSFP+Oty4uuuQ6mbwS822LzAmpbRPSukndRdTpyis7X/aeynOLV+feY56ZlVJw4/BLw2tURHxzfI3vbsjYtOI2DUiflv+6MxlEbFH3ydFxNyI+M+IuLz8LfCZEbE18ANgn3KNf9cofqt+UvmcGRExt828zoqIr5bzui0iXtvS9oGIuDoi/hgRp5TjxkfE+RFxQ/nary/Hfy4ibiqn/eLa5h0RXRFxXsvrfC0ijimH74iIz0bEFRHxh4h4XkTMiYiFEfG2ddS2cxS/X38acC2wY0R8o5zP/Jbp3k1xffWLI+LictxREXFj2afPt7zO0oj4VET8H/DCDfiMpWFtdN0FSJmZChyVUvrXiJgN/CNwLPC2lNKCiNgfOA14SZvnjk8p/V0UP0rznZTScyPiOODElNIrAYoriXZkO4orMO5BcWnRn0XEy8r6ZlL8wMi55WtNBhallF5RvsYWEbEV8Bpgj5RS6r3c8UDz7qCeu1NKL4yILwNnUfzOwCYUV+c7fS213QVMA45NKb2jrO+klNIj5dr6RRGxV0rpqxHxfmBWSunhiNge+DywH/AXil/7e3VK6RxgPDAvpfSJTt9MqUkMfmlo3Z5Sur4cvgbYmeKHPn7aEtpjB3jujwFSSpdGxOZ9wnZ9nZNSWgXcFBHblONeVt6uKx9PoAjby4AvlmvF56WULovi98iXA9+KiPOB89Yx73Xp/d2NG4EJKaXHgMciYnnZz4Fquwu4M6V0Zcu8XhfFz/WOpvgSMp3iUritng/MTSk9BBARPwReTHG53JUUP0AljUgGvzS0nmwZXglsAzyaip8DXpe+19dud73tFazehbdJh3VEy/2/p5T+u+/EEbEfxe8x/HtEdKeUPhURMyl+qORI4ARWb6VoN+/WutrV1vucVX2ev4ri/1Tb2iJiZ4qfse19vAtwIvD8lNJfIuKsNq/VWlc7y1NKK9fSLjWa+/ilei0Bbo+If4K/HqC29wDT9u5bfxGwOKW0uM00d1BsvoZiN8L6mAO8OSImlK+zQ0RsXW4Wfzyl9APgi8Dzymm2SCldQHHQ3Lq+uNwJTI/iLIQtKL4wPOPa2ky3OcUXgcXl1obDW9oeAzYrh/8POCgiJpW7BI4CLlnPmqRGco1fqt8bgW9ExMeAMcDZFL8+19dfIuL/t3e3OAoEQRTH33McBMMxuAcKhyRZAw7NDTbhEBhIMKhlDckEDoBBcAHsUogeQUiWD0Um9f/Z6XTKvVRPJbVRCbf+P3dNJM1sj1XC7WURsbLdkfRb/3Y4S+pJakua2r6obJgcqATo3HZLpXsePrn7WM807FW2mFWPzr9R29/duZ3tSmU24CDp5+bzt6Sl7VNEdG2PJK3r+hcRMReQANv5gAaop/O/ImL76VoANBtP/QAAJELHDwBAInT8AAAkQvADAJAIwQ8AQCIEPwAAiRD8AAAkQvADAJDIFUcBpeboJIX8AAAAAElFTkSuQmCC\n","text/plain":"<Figure size 504x720 with 1 Axes>"},"metadata":{"needs_background":"light"},"output_type":"display_data"}]},{"metadata":{"trusted":false},"cell_type":"code","source":"old_run = cur_run\ncur_run = mlflow.current_run_id()\ncur_exp = mlflow.current_exp_id()\nlink = f'/mlflow/#/metric/training_auc?runs=[\"{cur_run}\",\"{old_run}\"]&experiment={cur_exp}&plot_metric_keys=[\\\"training_logloss\\\",\\\"validation_logloss\\\",\\\"training_rmse\\\",\\\"validation_rmse\\\"]'\nHTML(f'<font size=\"+1\">Compare your 2 runs <a href={link}>here</a></font>') ","execution_count":32,"outputs":[{"data":{"text/html":"<font size=\"+1\">Compare your 2 runs <a href=/mlflow/#/metric/training_auc?runs=[\"acbb4ff7f942\",\"a92e93f393df\"]&experiment=1&plot_metric_keys=[\"training_logloss\",\"validation_logloss\",\"training_rmse\",\"validation_rmse\"]>here</a></font>","text/plain":"<IPython.core.display.HTML object>"},"execution_count":32,"metadata":{},"output_type":"execute_result"}]},{"metadata":{"trusted":false},"cell_type":"code","source":"gbm_embeddings.scoring_history()","execution_count":33,"outputs":[{"data":{"text/html":"<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th></th>\n <th>timestamp</th>\n <th>duration</th>\n <th>number_of_trees</th>\n <th>training_rmse</th>\n <th>training_logloss</th>\n <th>training_auc</th>\n <th>training_pr_auc</th>\n <th>training_lift</th>\n <th>training_classification_error</th>\n <th>validation_rmse</th>\n <th>validation_logloss</th>\n <th>validation_auc</th>\n <th>validation_pr_auc</th>\n <th>validation_lift</th>\n <th>validation_classification_error</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td></td>\n <td>2020-06-03 20:06:33</td>\n <td>0.131 sec</td>\n <td>0.0</td>\n <td>0.412517</td>\n <td>0.523672</td>\n <td>0.500000</td>\n <td>0.000000</td>\n <td>1.000000</td>\n <td>0.217458</td>\n <td>0.413827</td>\n <td>0.526125</td>\n <td>0.500000</td>\n <td>0.000000</td>\n <td>1.000000</td>\n <td>0.219374</td>\n </tr>\n <tr>\n <th>1</th>\n <td></td>\n <td>2020-06-03 20:06:34</td>\n <td>1.147 sec</td>\n <td>10.0</td>\n <td>0.382665</td>\n <td>0.460842</td>\n <td>0.778901</td>\n <td>0.916229</td>\n <td>1.255981</td>\n <td>0.185903</td>\n <td>0.385980</td>\n <td>0.466955</td>\n <td>0.768938</td>\n <td>0.911758</td>\n <td>1.272596</td>\n <td>0.191478</td>\n </tr>\n <tr>\n <th>2</th>\n <td></td>\n <td>2020-06-03 20:06:35</td>\n <td>2.106 sec</td>\n <td>20.0</td>\n <td>0.368603</td>\n <td>0.432135</td>\n <td>0.815712</td>\n <td>0.933750</td>\n <td>1.268760</td>\n <td>0.174441</td>\n <td>0.374044</td>\n <td>0.442140</td>\n <td>0.799287</td>\n <td>0.926887</td>\n <td>1.281023</td>\n <td>0.181625</td>\n </tr>\n <tr>\n <th>3</th>\n <td></td>\n <td>2020-06-03 20:06:36</td>\n <td>3.032 sec</td>\n <td>30.0</td>\n <td>0.358795</td>\n <td>0.412357</td>\n <td>0.838837</td>\n <td>0.943557</td>\n <td>1.268773</td>\n <td>0.167667</td>\n <td>0.367004</td>\n <td>0.427451</td>\n <td>0.814227</td>\n <td>0.932822</td>\n <td>1.281023</td>\n <td>0.176964</td>\n </tr>\n <tr>\n <th>4</th>\n <td></td>\n <td>2020-06-03 20:06:36</td>\n <td>3.928 sec</td>\n <td>40.0</td>\n <td>0.350731</td>\n <td>0.396337</td>\n <td>0.854237</td>\n <td>0.949766</td>\n <td>1.274236</td>\n <td>0.159421</td>\n <td>0.361087</td>\n <td>0.415420</td>\n <td>0.824577</td>\n <td>0.936954</td>\n <td>1.281023</td>\n <td>0.174501</td>\n </tr>\n <tr>\n <th>5</th>\n <td></td>\n <td>2020-06-03 20:06:37</td>\n <td>4.830 sec</td>\n <td>50.0</td>\n <td>0.342974</td>\n <td>0.381394</td>\n <td>0.868354</td>\n <td>0.955011</td>\n <td>1.274236</td>\n <td>0.152332</td>\n <td>0.355623</td>\n <td>0.404315</td>\n <td>0.835692</td>\n <td>0.941458</td>\n <td>1.281023</td>\n <td>0.169907</td>\n </tr>\n </tbody>\n</table>\n</div>","text/plain":" timestamp duration number_of_trees training_rmse \\\n0 2020-06-03 20:06:33 0.131 sec 0.0 0.412517 \n1 2020-06-03 20:06:34 1.147 sec 10.0 0.382665 \n2 2020-06-03 20:06:35 2.106 sec 20.0 0.368603 \n3 2020-06-03 20:06:36 3.032 sec 30.0 0.358795 \n4 2020-06-03 20:06:36 3.928 sec 40.0 0.350731 \n5 2020-06-03 20:06:37 4.830 sec 50.0 0.342974 \n\n training_logloss training_auc training_pr_auc training_lift \\\n0 0.523672 0.500000 0.000000 1.000000 \n1 0.460842 0.778901 0.916229 1.255981 \n2 0.432135 0.815712 0.933750 1.268760 \n3 0.412357 0.838837 0.943557 1.268773 \n4 0.396337 0.854237 0.949766 1.274236 \n5 0.381394 0.868354 0.955011 1.274236 \n\n training_classification_error validation_rmse validation_logloss \\\n0 0.217458 0.413827 0.526125 \n1 0.185903 0.385980 0.466955 \n2 0.174441 0.374044 0.442140 \n3 0.167667 0.367004 0.427451 \n4 0.159421 0.361087 0.415420 \n5 0.152332 0.355623 0.404315 \n\n validation_auc validation_pr_auc validation_lift \\\n0 0.500000 0.000000 1.000000 \n1 0.768938 0.911758 1.272596 \n2 0.799287 0.926887 1.281023 \n3 0.814227 0.932822 1.281023 \n4 0.824577 0.936954 1.281023 \n5 0.835692 0.941458 1.281023 \n\n validation_classification_error \n0 0.219374 \n1 0.191478 \n2 0.181625 \n3 0.176964 \n4 0.174501 \n5 0.169907 "},"execution_count":33,"metadata":{},"output_type":"execute_result"}]},{"metadata":{"trusted":false},"cell_type":"code","source":"mlflow.end_run()","execution_count":34,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Baseline AUC: \" + str(round(gbm_baseline.auc(valid = True), 3)))\nprint(\"With Embeddings AUC: \" + str(round(gbm_embeddings.auc(valid = True), 3)))\nlink = f'/mlflow/#/metric/training_auc?runs=[\"{cur_run}\",\"{old_run}\"]&experiment={cur_exp}&plot_metric_keys=[\\\"training_auc\\\",\\\"validation_auc\\\"]'\nHTML(f'<font size=\"+1\">See a metrics comparison <a href={link}>here</a></font>')","execution_count":78,"outputs":[{"output_type":"stream","text":"Baseline AUC: 0.732\nWith Embeddings AUC: 0.836\n","name":"stdout"},{"output_type":"execute_result","execution_count":78,"data":{"text/plain":"<IPython.core.display.HTML object>","text/html":"<font size=\"+1\">See a metrics comparison <a href=/mlflow/#/metric/training_auc?runs=[\"acbb4ff7f942\",\"a92e93f393df\"]&experiment=1&plot_metric_keys=[\"training_auc\",\"validation_auc\"]>here</a></font>"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"# That's some great imrpovement! So what's next?\n<blockquote>We included the customer reviews and developed a better model. We've logged everything to MLFlow for detailed comparisons. Now what?<br>\n Let's deploy our models to production so we can utilize what we've built. First, we'll deploy our word vectorizer model, and then deploy our GBM model. Finally, we'll create a feed from the first model to the second, so we can see the final predictions!\n </i></br><footer>Splice Machine</footer></blockquote><br>\n<img src=https://splice-demo.s3.amazonaws.com/H2O+Demo+Model+Diagram.png>"},{"metadata":{},"cell_type":"markdown","source":"## Step 1: Deploy Word2Vec Model"},{"metadata":{"trusted":false},"cell_type":"code","source":"help(mlflow.deploy_db)","execution_count":165,"outputs":[{"name":"stdout","output_type":"stream","text":"Help on function _deploy_db in module splicemachine.mlflow_support.mlflow_support:\n\n_deploy_db(fittedModel, df, db_schema_name, db_table_name, primary_key, run_id: str = None, classes=None, sklearn_args={}, verbose=False, pred_threshold=None, replace=False) -> None\n Function to deploy a trained (currently Spark, Sklearn or H2O) model to the Database.\n This creates 2 tables: One with the features of the model, and one with the prediction and metadata.\n They are linked with a column called MOMENT_ID\n \n :param fittedModel: (ML pipeline or model) The fitted pipeline to deploy\n :param df: (Spark DF) The dataframe used to train the model\n NOTE: this dataframe should NOT be transformed by the model. The columns in this df are the ones\n that will be used to create the table.\n :param db_schema_name: (str) the schema name to deploy to. If None, the currently set schema will be used.\n :param db_table_name: (str) the table name to deploy to. If none, the run_id will be used for the table name(s)\n :param primary_key: (List[Tuple[str, str]]) List of column + SQL datatype to use for the primary/composite key\n :param run_id: (str) The active run_id\n :param classes: (List[str]) The classes (prediction labels) for the model being deployed.\n NOTE: If not supplied, the table will have default column names for each class\n :param sklearn_args: (dict{str: str}) Prediction options for sklearn models\n Available key value options:\n 'predict_call': 'predict', 'predict_proba', or 'transform'\n - Determines the function call for the model\n If blank, predict will be used\n (or transform if model doesn't have predict)\n 'predict_args': 'return_std' or 'return_cov' - For Bayesian and Gaussian models\n Only one can be specified\n If the model does not have the option specified, it will be ignored.\n :param verbose: (bool) Whether or not to print out the queries being created. Helpful for debugging\n :param pred_threshold: (double) A prediction threshold for *Keras* binary classification models\n If the model type isn't Keras, this parameter will be ignored\n NOTE: If the model type is Keras, the output layer has 1 node, and pred_threshold is None,\n you will NOT receive a class prediction, only the output of the final layer (like model.predict()).\n If you want a class prediction\n for your binary classification problem, you MUST pass in a threshold.\n :param replace: (bool) whether or not to replace a currently existing model. This param does not yet work\n \n \n This function creates the following:\n * Table (default called DATA_{run_id}) where run_id is the run_id of the mlflow run associated to that model.\n This will have a column for each feature in the feature vector as well as a MOMENT_ID as primary key\n * Table (default called DATA_{run_id}_PREDS) That will have the columns:\n USER which is the current user who made the request\n EVAL_TIME which is the CURRENT_TIMESTAMP\n MOMENT_ID same as the DATA table to link predictions to rows in the table\n PREDICTION. The prediction of the model. If the :classes: param is not filled in, this will be default values for classification models\n A column for each class of the predictor with the value being the probability/confidence of the model if applicable\n * A trigger that runs on (after) insertion to the data table that runs an INSERT into the prediction table,\n calling the PREDICT function, passing in the row of data as well as the schema of the dataset, and the run_id of the model to run\n * A trigger that runs on (after) insertion to the prediction table that calls an UPDATE to the row inserted,\n parsing the prediction probabilities and filling in proper column values\n\n"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Get the run_id from the name. Note that multiple runs can have the same name, so this returns a list\nrun_id = mlflow.get_run_ids_by_name('review_tokenizer')[0]\n# Get the model from that run\nw2v_model = mlflow.load_model(run_id=run_id, name='word2vec_model')\ndeploy_df = hc.asSparkFrame(reviews[['Text']])\nschema = 'REPLACE_ME_DBSCHEMA'\nschema='splice'\nsplice._dropTableIfExists(f'{schema}.word_vec_model')\nsplice._dropTableIfExists(f'{schema}.word_vec_model_preds')\nmlflow.deploy_db(w2v_model, deploy_df, schema, 'word_vec_model', [('REVIEW_ID', 'INT')], run_id=run_id, classes=[f'C{i+1}' for i in range(100)], verbose=True)","execution_count":111,"outputs":[{"output_type":"stream","text":"Droping table splice.word_vec_model\nDroping table splice.word_vec_model_preds\nWARN: A model with this ID already exists in the table. We are NOT replacing it. We will use the currently existing model.\nTo replace, use a new run_id\nPrediction labels found. Using ['C1', 'C2', 'C3', 'C4', 'C5', 'C6', 'C7', 'C8', 'C9', 'C10', 'C11', 'C12', 'C13', 'C14', 'C15', 'C16', 'C17', 'C18', 'C19', 'C20', 'C21', 'C22', 'C23', 'C24', 'C25', 'C26', 'C27', 'C28', 'C29', 'C30', 'C31', 'C32', 'C33', 'C34', 'C35', 'C36', 'C37', 'C38', 'C39', 'C40', 'C41', 'C42', 'C43', 'C44', 'C45', 'C46', 'C47', 'C48', 'C49', 'C50', 'C51', 'C52', 'C53', 'C54', 'C55', 'C56', 'C57', 'C58', 'C59', 'C60', 'C61', 'C62', 'C63', 'C64', 'C65', 'C66', 'C67', 'C68', 'C69', 'C70', 'C71', 'C72', 'C73', 'C74', 'C75', 'C76', 'C77', 'C78', 'C79', 'C80', 'C81', 'C82', 'C83', 'C84', 'C85', 'C86', 'C87', 'C88', 'C89', 'C90', 'C91', 'C92', 'C93', 'C94', 'C95', 'C96', 'C97', 'C98', 'C99', 'C100'] as labels for predictions [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99] respectively\nDeploying model 53f5a36065ad to table splice.word_vec_model\nCreating data table ... \n CREATE TABLE splice.word_vec_model (\n\tText VARCHAR(5000),\tREVIEW_ID INT,\n\tPRIMARY KEY(REVIEW_ID)\n)\n\nDone.\nCreating prediction table ... \nCREATE TABLE splice.word_vec_model_PREDS (\n \tCUR_USER VARCHAR(50) DEFAULT CURRENT_USER,\n \tEVAL_TIME TIMESTAMP DEFAULT CURRENT_TIMESTAMP,\n \tRUN_ID VARCHAR(50) DEFAULT '53f5a36065ad',\n \tREVIEW_ID INT,\n\t\"C1\" DOUBLE,\n\t\"C2\" DOUBLE,\n\t\"C3\" DOUBLE,\n\t\"C4\" DOUBLE,\n\t\"C5\" DOUBLE,\n\t\"C6\" DOUBLE,\n\t\"C7\" DOUBLE,\n\t\"C8\" DOUBLE,\n\t\"C9\" DOUBLE,\n\t\"C10\" DOUBLE,\n\t\"C11\" DOUBLE,\n\t\"C12\" DOUBLE,\n\t\"C13\" DOUBLE,\n\t\"C14\" DOUBLE,\n\t\"C15\" DOUBLE,\n\t\"C16\" DOUBLE,\n\t\"C17\" DOUBLE,\n\t\"C18\" DOUBLE,\n\t\"C19\" DOUBLE,\n\t\"C20\" DOUBLE,\n\t\"C21\" DOUBLE,\n\t\"C22\" DOUBLE,\n\t\"C23\" DOUBLE,\n\t\"C24\" DOUBLE,\n\t\"C25\" DOUBLE,\n\t\"C26\" DOUBLE,\n\t\"C27\" DOUBLE,\n\t\"C28\" DOUBLE,\n\t\"C29\" DOUBLE,\n\t\"C30\" DOUBLE,\n\t\"C31\" DOUBLE,\n\t\"C32\" DOUBLE,\n\t\"C33\" DOUBLE,\n\t\"C34\" DOUBLE,\n\t\"C35\" DOUBLE,\n\t\"C36\" DOUBLE,\n\t\"C37\" DOUBLE,\n\t\"C38\" DOUBLE,\n\t\"C39\" DOUBLE,\n\t\"C40\" DOUBLE,\n\t\"C41\" DOUBLE,\n\t\"C42\" DOUBLE,\n\t\"C43\" DOUBLE,\n\t\"C44\" DOUBLE,\n\t\"C45\" DOUBLE,\n\t\"C46\" DOUBLE,\n\t\"C47\" DOUBLE,\n\t\"C48\" DOUBLE,\n\t\"C49\" DOUBLE,\n\t\"C50\" DOUBLE,\n\t\"C51\" DOUBLE,\n\t\"C52\" DOUBLE,\n\t\"C53\" DOUBLE,\n\t\"C54\" DOUBLE,\n\t\"C55\" DOUBLE,\n\t\"C56\" DOUBLE,\n\t\"C57\" DOUBLE,\n\t\"C58\" DOUBLE,\n\t\"C59\" DOUBLE,\n\t\"C60\" DOUBLE,\n\t\"C61\" DOUBLE,\n\t\"C62\" DOUBLE,\n\t\"C63\" DOUBLE,\n\t\"C64\" DOUBLE,\n\t\"C65\" DOUBLE,\n\t\"C66\" DOUBLE,\n\t\"C67\" DOUBLE,\n\t\"C68\" DOUBLE,\n\t\"C69\" DOUBLE,\n\t\"C70\" DOUBLE,\n\t\"C71\" DOUBLE,\n\t\"C72\" DOUBLE,\n\t\"C73\" DOUBLE,\n\t\"C74\" DOUBLE,\n\t\"C75\" DOUBLE,\n\t\"C76\" DOUBLE,\n\t\"C77\" DOUBLE,\n\t\"C78\" DOUBLE,\n\t\"C79\" DOUBLE,\n\t\"C80\" DOUBLE,\n\t\"C81\" DOUBLE,\n\t\"C82\" DOUBLE,\n\t\"C83\" DOUBLE,\n\t\"C84\" DOUBLE,\n\t\"C85\" DOUBLE,\n\t\"C86\" DOUBLE,\n\t\"C87\" DOUBLE,\n\t\"C88\" DOUBLE,\n\t\"C89\" DOUBLE,\n\t\"C90\" DOUBLE,\n\t\"C91\" DOUBLE,\n\t\"C92\" DOUBLE,\n\t\"C93\" DOUBLE,\n\t\"C94\" DOUBLE,\n\t\"C95\" DOUBLE,\n\t\"C96\" DOUBLE,\n\t\"C97\" DOUBLE,\n\t\"C98\" DOUBLE,\n\t\"C99\" DOUBLE,\n\t\"C100\" DOUBLE,\n\tPRIMARY KEY(REVIEW_ID)\n)\n\nDone.\nCreating model prediction trigger ... \nCREATE TRIGGER runModel_splice_word_vec_model_53f5a36065ad\n \tAFTER INSERT\n \tON splice.word_vec_model\n \tREFERENCING NEW AS NEWROW\n \tFOR EACH ROW\n \t\tINSERT INTO splice.word_vec_model_PREDS(REVIEW_ID,\"C1\",\"C2\",\"C3\",\"C4\",\"C5\",\"C6\",\"C7\",\"C8\",\"C9\",\"C10\",\"C11\",\"C12\",\"C13\",\"C14\",\"C15\",\"C16\",\"C17\",\"C18\",\"C19\",\"C20\",\"C21\",\"C22\",\"C23\",\"C24\",\"C25\",\"C26\",\"C27\",\"C28\",\"C29\",\"C30\",\"C31\",\"C32\",\"C33\",\"C34\",\"C35\",\"C36\",\"C37\",\"C38\",\"C39\",\"C40\",\"C41\",\"C42\",\"C43\",\"C44\",\"C45\",\"C46\",\"C47\",\"C48\",\"C49\",\"C50\",\"C51\",\"C52\",\"C53\",\"C54\",\"C55\",\"C56\",\"C57\",\"C58\",\"C59\",\"C60\",\"C61\",\"C62\",\"C63\",\"C64\",\"C65\",\"C66\",\"C67\",\"C68\",\"C69\",\"C70\",\"C71\",\"C72\",\"C73\",\"C74\",\"C75\",\"C76\",\"C77\",\"C78\",\"C79\",\"C80\",\"C81\",\"C82\",\"C83\",\"C84\",\"C85\",\"C86\",\"C87\",\"C88\",\"C89\",\"C90\",\"C91\",\"C92\",\"C93\",\"C94\",\"C95\",\"C96\",\"C97\",\"C98\",\"C99\",\"C100\") SELECT \tNEWROW.REVIEW_ID, b.\"C1\",b.\"C2\",b.\"C3\",b.\"C4\",b.\"C5\",b.\"C6\",b.\"C7\",b.\"C8\",b.\"C9\",b.\"C10\",b.\"C11\",b.\"C12\",b.\"C13\",b.\"C14\",b.\"C15\",b.\"C16\",b.\"C17\",b.\"C18\",b.\"C19\",b.\"C20\",b.\"C21\",b.\"C22\",b.\"C23\",b.\"C24\",b.\"C25\",b.\"C26\",b.\"C27\",b.\"C28\",b.\"C29\",b.\"C30\",b.\"C31\",b.\"C32\",b.\"C33\",b.\"C34\",b.\"C35\",b.\"C36\",b.\"C37\",b.\"C38\",b.\"C39\",b.\"C40\",b.\"C41\",b.\"C42\",b.\"C43\",b.\"C44\",b.\"C45\",b.\"C46\",b.\"C47\",b.\"C48\",b.\"C49\",b.\"C50\",b.\"C51\",b.\"C52\",b.\"C53\",b.\"C54\",b.\"C55\",b.\"C56\",b.\"C57\",b.\"C58\",b.\"C59\",b.\"C60\",b.\"C61\",b.\"C62\",b.\"C63\",b.\"C64\",b.\"C65\",b.\"C66\",b.\"C67\",b.\"C68\",b.\"C69\",b.\"C70\",b.\"C71\",b.\"C72\",b.\"C73\",b.\"C74\",b.\"C75\",b.\"C76\",b.\"C77\",b.\"C78\",b.\"C79\",b.\"C80\",b.\"C81\",b.\"C82\",b.\"C83\",b.\"C84\",b.\"C85\",b.\"C86\",b.\"C87\",b.\"C88\",b.\"C89\",b.\"C90\",b.\"C91\",b.\"C92\",b.\"C93\",b.\"C94\",b.\"C95\",b.\"C96\",b.\"C97\",b.\"C98\",b.\"C99\",b.\"C100\" FROM new com.splicemachine.mlrunner.MLRunner('key_value', '53f5a36065ad', TRIM(CAST(NEWROW.Text as CHAR(41))), 'Text VARCHAR(5000)', 'transform', 'None') as b (\"C1\" DOUBLE,\"C2\" DOUBLE,\"C3\" DOUBLE,\"C4\" DOUBLE,\"C5\" DOUBLE,\"C6\" DOUBLE,\"C7\" DOUBLE,\"C8\" DOUBLE,\"C9\" DOUBLE,\"C10\" DOUBLE,\"C11\" DOUBLE,\"C12\" DOUBLE,\"C13\" DOUBLE,\"C14\" DOUBLE,\"C15\" DOUBLE,\"C16\" DOUBLE,\"C17\" DOUBLE,\"C18\" DOUBLE,\"C19\" DOUBLE,\"C20\" DOUBLE,\"C21\" DOUBLE,\"C22\" DOUBLE,\"C23\" DOUBLE,\"C24\" DOUBLE,\"C25\" DOUBLE,\"C26\" DOUBLE,\"C27\" DOUBLE,\"C28\" DOUBLE,\"C29\" DOUBLE,\"C30\" DOUBLE,\"C31\" DOUBLE,\"C32\" DOUBLE,\"C33\" DOUBLE,\"C34\" DOUBLE,\"C35\" DOUBLE,\"C36\" DOUBLE,\"C37\" DOUBLE,\"C38\" DOUBLE,\"C39\" DOUBLE,\"C40\" DOUBLE,\"C41\" DOUBLE,\"C42\" DOUBLE,\"C43\" DOUBLE,\"C44\" DOUBLE,\"C45\" DOUBLE,\"C46\" DOUBLE,\"C47\" DOUBLE,\"C48\" DOUBLE,\"C49\" DOUBLE,\"C50\" DOUBLE,\"C51\" DOUBLE,\"C52\" DOUBLE,\"C53\" DOUBLE,\"C54\" DOUBLE,\"C55\" DOUBLE,\"C56\" DOUBLE,\"C57\" DOUBLE,\"C58\" DOUBLE,\"C59\" DOUBLE,\"C60\" DOUBLE,\"C61\" DOUBLE,\"C62\" DOUBLE,\"C63\" DOUBLE,\"C64\" DOUBLE,\"C65\" DOUBLE,\"C66\" DOUBLE,\"C67\" DOUBLE,\"C68\" DOUBLE,\"C69\" DOUBLE,\"C70\" DOUBLE,\"C71\" DOUBLE,\"C72\" DOUBLE,\"C73\" DOUBLE,\"C74\" DOUBLE,\"C75\" DOUBLE,\"C76\" DOUBLE,\"C77\" DOUBLE,\"C78\" DOUBLE,\"C79\" DOUBLE,\"C80\" DOUBLE,\"C81\" DOUBLE,\"C82\" DOUBLE,\"C83\" DOUBLE,\"C84\" DOUBLE,\"C85\" DOUBLE,\"C86\" DOUBLE,\"C87\" DOUBLE,\"C88\" DOUBLE,\"C89\" DOUBLE,\"C90\" DOUBLE,\"C91\" DOUBLE,\"C92\" DOUBLE,\"C93\" DOUBLE,\"C94\" DOUBLE,\"C95\" DOUBLE,\"C96\" DOUBLE,\"C97\" DOUBLE,\"C98\" DOUBLE,\"C99\" DOUBLE,\"C100\" DOUBLE)\n\nDone.\nModel Deployed.\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"## Sweet! Let's try it out"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%sql\ninsert into word_vec_model values('It''s a great value!', 1);\ninsert into word_vec_model values('I hate this product', 2);\n\nselect * from word_vec_model_preds;","execution_count":109,"outputs":[{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"649b6b28-7ce6-4d07-8cab-c1870840c105","version_major":2}},"metadata":{}},{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"d7d4225f-68f0-4a2d-9899-ee3484628dd1","version_major":2}},"metadata":{}},{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"b002f839-64c8-4fd4-a71a-2b04a8fdc39d","version_major":2}},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"## Amazing! Up Next: GBM\n<blockquote>\n Now when we insert rows into our <code>WORD_VEC_MODEL</code> table, our machine learning vectorizer will run automatically and make \"predictions\". Now we need to deploy our GBM model in the same way.\n </i></br><footer>Splice Machine</footer></blockquote><br>\n\n"},{"metadata":{"trusted":true,"scrolled":true},"cell_type":"code","source":"# Get the run_id from the name. Note that multiple runs can have the same name, so this returns a list\nrun_id = mlflow.get_run_ids_by_name('GBM with word vectors')[0]\n# Get the model from that run\ngbm_vec_model = mlflow.load_model(run_id=run_id, name='vectorized_model')\ndeploy_df = hc.asSparkFrame(ext_reviews[predictors])\nschema = 'REPLACE_ME_DBSCHEMA'\nschema='splice'\nsplice._dropTableIfExists(f'{schema}.gbm_w2v_model')\nsplice._dropTableIfExists(f'{schema}.gbm_w2v_model_preds')\nmlflow.deploy_db(gbm_vec_model, deploy_df, schema, 'gbm_w2v_model', [('REVIEW_ID', 'INT')], classes=['negative', 'positive'], run_id=run_id, verbose=True)","execution_count":85,"outputs":[{"output_type":"stream","text":"Droping table splice.gbm_w2v_model\nDroping table splice.gbm_w2v_model_preds\nA model with this ID already exists in the table. We are NOT replacing it. We will use the currently existing model.\nTo replace, use a new run_id\nPrediction labels found. Using ['negative', 'positive'] as labels for predictions [0, 1] respectively\nDeploying model acbb4ff7f942 to table splice.gbm_w2v_model\nCreating data table ... \n CREATE TABLE splice.gbm_w2v_model (\n\tProductId VARCHAR(5000),\tUserId VARCHAR(5000),\tHelpfulnessNumerator SMALLINT,\tHelpfulnessDenominator SMALLINT,\tTime INTEGER,\tC1 DOUBLE,\tC2 DOUBLE,\tC3 DOUBLE,\tC4 DOUBLE,\tC5 DOUBLE,\tC6 DOUBLE,\tC7 DOUBLE,\tC8 DOUBLE,\tC9 DOUBLE,\tC10 DOUBLE,\tC11 DOUBLE,\tC12 DOUBLE,\tC13 DOUBLE,\tC14 DOUBLE,\tC15 DOUBLE,\tC16 DOUBLE,\tC17 DOUBLE,\tC18 DOUBLE,\tC19 DOUBLE,\tC20 DOUBLE,\tC21 DOUBLE,\tC22 DOUBLE,\tC23 DOUBLE,\tC24 DOUBLE,\tC25 DOUBLE,\tC26 DOUBLE,\tC27 DOUBLE,\tC28 DOUBLE,\tC29 DOUBLE,\tC30 DOUBLE,\tC31 DOUBLE,\tC32 DOUBLE,\tC33 DOUBLE,\tC34 DOUBLE,\tC35 DOUBLE,\tC36 DOUBLE,\tC37 DOUBLE,\tC38 DOUBLE,\tC39 DOUBLE,\tC40 DOUBLE,\tC41 DOUBLE,\tC42 DOUBLE,\tC43 DOUBLE,\tC44 DOUBLE,\tC45 DOUBLE,\tC46 DOUBLE,\tC47 DOUBLE,\tC48 DOUBLE,\tC49 DOUBLE,\tC50 DOUBLE,\tC51 DOUBLE,\tC52 DOUBLE,\tC53 DOUBLE,\tC54 DOUBLE,\tC55 DOUBLE,\tC56 DOUBLE,\tC57 DOUBLE,\tC58 DOUBLE,\tC59 DOUBLE,\tC60 DOUBLE,\tC61 DOUBLE,\tC62 DOUBLE,\tC63 DOUBLE,\tC64 DOUBLE,\tC65 DOUBLE,\tC66 DOUBLE,\tC67 DOUBLE,\tC68 DOUBLE,\tC69 DOUBLE,\tC70 DOUBLE,\tC71 DOUBLE,\tC72 DOUBLE,\tC73 DOUBLE,\tC74 DOUBLE,\tC75 DOUBLE,\tC76 DOUBLE,\tC77 DOUBLE,\tC78 DOUBLE,\tC79 DOUBLE,\tC80 DOUBLE,\tC81 DOUBLE,\tC82 DOUBLE,\tC83 DOUBLE,\tC84 DOUBLE,\tC85 DOUBLE,\tC86 DOUBLE,\tC87 DOUBLE,\tC88 DOUBLE,\tC89 DOUBLE,\tC90 DOUBLE,\tC91 DOUBLE,\tC92 DOUBLE,\tC93 DOUBLE,\tC94 DOUBLE,\tC95 DOUBLE,\tC96 DOUBLE,\tC97 DOUBLE,\tC98 DOUBLE,\tC99 DOUBLE,\tC100 DOUBLE,\tREVIEW_ID INT,\n\tPRIMARY KEY(REVIEW_ID)\n)\n\nDone.\nCreating prediction table ... \nCREATE TABLE splice.gbm_w2v_model_PREDS (\n \tCUR_USER VARCHAR(50) DEFAULT CURRENT_USER,\n \tEVAL_TIME TIMESTAMP DEFAULT CURRENT_TIMESTAMP,\n \tRUN_ID VARCHAR(50) DEFAULT 'acbb4ff7f942',\n \tREVIEW_ID INT,\n\tPREDICTION VARCHAR(5000),\n\t\"negative\" DOUBLE,\n\t\"positive\" DOUBLE,\n\tPRIMARY KEY(REVIEW_ID)\n)\n\nDone.\nCreating model prediction trigger ... \nCREATE TRIGGER runModel_splice_gbm_w2v_model_acbb4ff7f942\n \tAFTER INSERT\n \tON splice.gbm_w2v_model\n \tREFERENCING NEW AS NEWROW\n \tFOR EACH ROW\n \t\tINSERT INTO splice.gbm_w2v_model_PREDS(\tREVIEW_ID,PREDICTION) VALUES(\tNEWROW.REVIEW_ID,MLMANAGER.PREDICT_CLASSIFICATION('acbb4ff7f942',TRIM(CAST(NEWROW.ProductId as CHAR(41)))||','||TRIM(CAST(NEWROW.UserId as CHAR(41)))||','||TRIM(CAST(NEWROW.HelpfulnessNumerator as CHAR(41)))||','||TRIM(CAST(NEWROW.HelpfulnessDenominator as CHAR(41)))||','||TRIM(CAST(NEWROW.Time as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C1 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C2 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C3 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C4 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C5 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C6 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C7 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C8 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C9 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C10 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C11 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C12 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C13 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C14 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C15 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C16 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C17 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C18 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C19 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C20 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C21 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C22 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C23 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C24 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C25 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C26 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C27 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C28 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C29 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C30 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C31 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C32 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C33 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C34 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C35 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C36 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C37 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C38 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C39 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C40 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C41 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C42 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C43 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C44 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C45 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C46 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C47 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C48 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C49 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C50 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C51 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C52 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C53 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C54 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C55 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C56 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C57 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C58 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C59 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C60 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C61 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C62 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C63 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C64 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C65 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C66 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C67 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C68 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C69 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C70 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C71 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C72 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C73 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C74 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C75 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C76 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C77 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C78 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C79 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C80 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C81 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C82 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C83 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C84 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C85 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C86 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C87 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C88 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C89 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C90 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C91 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C92 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C93 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C94 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C95 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C96 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C97 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C98 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C99 as DECIMAL(38,10)) as CHAR(41)))||','||TRIM(CAST(CAST(NEWROW.C100 as DECIMAL(38,10)) as CHAR(41))),\n","name":"stdout"},{"output_type":"stream","text":"'ProductId VARCHAR(5000),UserId VARCHAR(5000),HelpfulnessNumerator SMALLINT,HelpfulnessDenominator SMALLINT,Time INTEGER,C1 DOUBLE,C2 DOUBLE,C3 DOUBLE,C4 DOUBLE,C5 DOUBLE,C6 DOUBLE,C7 DOUBLE,C8 DOUBLE,C9 DOUBLE,C10 DOUBLE,C11 DOUBLE,C12 DOUBLE,C13 DOUBLE,C14 DOUBLE,C15 DOUBLE,C16 DOUBLE,C17 DOUBLE,C18 DOUBLE,C19 DOUBLE,C20 DOUBLE,C21 DOUBLE,C22 DOUBLE,C23 DOUBLE,C24 DOUBLE,C25 DOUBLE,C26 DOUBLE,C27 DOUBLE,C28 DOUBLE,C29 DOUBLE,C30 DOUBLE,C31 DOUBLE,C32 DOUBLE,C33 DOUBLE,C34 DOUBLE,C35 DOUBLE,C36 DOUBLE,C37 DOUBLE,C38 DOUBLE,C39 DOUBLE,C40 DOUBLE,C41 DOUBLE,C42 DOUBLE,C43 DOUBLE,C44 DOUBLE,C45 DOUBLE,C46 DOUBLE,C47 DOUBLE,C48 DOUBLE,C49 DOUBLE,C50 DOUBLE,C51 DOUBLE,C52 DOUBLE,C53 DOUBLE,C54 DOUBLE,C55 DOUBLE,C56 DOUBLE,C57 DOUBLE,C58 DOUBLE,C59 DOUBLE,C60 DOUBLE,C61 DOUBLE,C62 DOUBLE,C63 DOUBLE,C64 DOUBLE,C65 DOUBLE,C66 DOUBLE,C67 DOUBLE,C68 DOUBLE,C69 DOUBLE,C70 DOUBLE,C71 DOUBLE,C72 DOUBLE,C73 DOUBLE,C74 DOUBLE,C75 DOUBLE,C76 DOUBLE,C77 DOUBLE,C78 DOUBLE,C79 DOUBLE,C80 DOUBLE,C81 DOUBLE,C82 DOUBLE,C83 DOUBLE,C84 DOUBLE,C85 DOUBLE,C86 DOUBLE,C87 DOUBLE,C88 DOUBLE,C89 DOUBLE,C90 DOUBLE,C91 DOUBLE,C92 DOUBLE,C93 DOUBLE,C94 DOUBLE,C95 DOUBLE,C96 DOUBLE,C97 DOUBLE,C98 DOUBLE,C99 DOUBLE,C100 DOUBLE'))\n\nDone.\nCreating parsing trigger ... \nCREATE TRIGGER PARSERESULT_splice_gbm_w2v_model_acbb4ff7f942\n \tAFTER INSERT\n \tON splice.gbm_w2v_model_PREDS\n \tREFERENCING NEW AS NEWROW\n \tFOR EACH ROW\n \t\tUPDATE splice.gbm_w2v_model_PREDS set \"negative\"=MLMANAGER.PARSEPROBS(NEWROW.prediction,0),\"positive\"=MLMANAGER.PARSEPROBS(NEWROW.prediction,1),PREDICTION=\n\t\tCASE\n\t\tWHEN MLMANAGER.GETPREDICTION(NEWROW.prediction)=0 then 'negative'\n\t\tWHEN MLMANAGER.GETPREDICTION(NEWROW.prediction)=1 then 'positive'\n\t\tEND WHERE REVIEW_ID=NEWROW.REVIEW_ID\n\nDone.\nModel Deployed.\n","name":"stdout"}]},{"metadata":{"trusted":false},"cell_type":"markdown","source":"## Almost Done!\n<blockquote>\n Now we have 2 models deployed in the database:\n <ul>\n <li> A model that takes a sentence and converts it into a vector</li>\n <li> A model that takes the vector + a few other features and makes a prediction about the review </li>\n </ul>\n Now, all we need to do is connect them with a pipeline. We'll connect the 2 tables together like the image above to create a full cycle ML Pipeline\n</i></br><footer>Splice Machine</footer></blockquote><br>"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%sql\n\ndrop trigger WORD2VEC_PIPELINE;\ndrop trigger WORD2VEC_PIPELINE2;\n\nCREATE TRIGGER WORD2VEC_PIPELINE\nAFTER INSERT\nON AMAZON_REVIEWS\nREFERENCING NEW AS N\nFOR EACH ROW\nINSERT INTO WORD_VEC_MODEL values(N.REVIEW, N.ID);\n\n\nCREATE TRIGGER WORD2VEC_PIPELINE2\nAFTER INSERT\nON WORD_VEC_MODEL_PREDS\nREFERENCING NEW_TABLE AS NEWROW\nFOR EACH STATEMENT\nINSERT INTO GBM_W2V_MODEL (PRODUCTID, USERID, HELPFULNESSNUMERATOR, HELPFULNESSDENOMINATOR, TIME, REVIEW_ID, C1,C2,C3,C4,C5,C6,C7,C8,C9,C10,C11,C12,C13,C14,C15,C16,C17,C18,C19,C20,C21,C22,C23,C24,C25,C26,C27,C28,C29,C30,C31,C32,C33,C34,C35,C36,C37,C38,C39,C40,C41,C42,C43,C44,C45,C46,C47,C48,C49,C50,C51,C52,C53,C54,C55,C56,C57,C58,C59,C60,C61,C62,C63,C64,C65,C66,C67,C68,C69,C70,C71,C72,C73,C74,C75,C76,C77,C78,C79,C80,C81,C82,C83,C84,C85,C86,C87,C88,C89,C90,C91,C92,C93,C94,C95,C96,C97,C98,C99,C100)\n SELECT a.PRODUCTID, a.USERID, a.HELPFULNESSNUMERATOR, a.HELPFULNESSDENOMINATOR, a.REVIEW_TIME, a.ID, N.C1,N.C2,N.C3,N.C4,N.C5,N.C6,N.C7,N.C8,N.C9,N.C10,N.C11,N.C12,N.C13,N.C14,N.C15,N.C16,N.C17,N.C18,N.C19,N.C20,N.C21,N.C22,N.C23,N.C24,N.C25,N.C26,N.C27,N.C28,N.C29,N.C30,N.C31,N.C32,N.C33,N.C34,N.C35,N.C36,N.C37,N.C38,N.C39,N.C40,N.C41,N.C42,N.C43,N.C44,N.C45,N.C46,N.C47,N.C48,N.C49,N.C50,N.C51,N.C52,N.C53,N.C54,N.C55,N.C56,N.C57,N.C58,N.C59,N.C60,N.C61,N.C62,N.C63,N.C64,N.C65,N.C66,N.C67,N.C68,N.C69,N.C70,N.C71,N.C72,N.C73,N.C74,N.C75,N.C76,N.C77,N.C78,N.C79,N.C80,N.C81,N.C82,N.C83,N.C84,N.C85,N.C86,N.C87,N.C88,N.C89,N.C90,N.C91,N.C92,N.C93,N.C94,N.C95,N.C96,N.C97,N.C98,N.C99,N.C100\n FROM NEWROW N, AMAZON_REVIEWS a --splice-properties useSpark=false\n WHERE a.ID=N.REVIEW_ID;\n ","execution_count":130,"outputs":[{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"10d4b9c4-5f38-4279-8f35-61377bfeba96","version_major":2}},"metadata":{}},{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"24b49926-76e9-4fcc-9bf9-d2ff238319ec","version_major":2}},"metadata":{}},{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"c59e8f24-462b-4f7f-a53e-18b7321e1c67","version_major":2}},"metadata":{}},{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"31ebf383-0d94-43a0-a2dd-4d885d2c8251","version_major":2}},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"reviews[reviews['PositiveReview']=='0']","execution_count":142,"outputs":[{"output_type":"display_data","data":{"text/html":"<table>\n<thead>\n<tr><th>ProductId </th><th>UserId </th><th>Summary </th><th style=\"text-align: right;\"> Score</th><th style=\"text-align: right;\"> HelpfulnessDenominator</th><th style=\"text-align: right;\"> Id</th><th>ProfileName </th><th style=\"text-align: right;\"> HelpfulnessNumerator</th><th style=\"text-align: right;\"> Time</th><th>Text </th><th style=\"text-align: right;\"> PositiveReview</th></tr>\n</thead>\n<tbody>\n<tr><td>B00141QYSQ </td><td>A1YS02UZZGRDCT</td><td>Do Not Buy </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 41471</td><td>Evan Eberhardt </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\">1.34836e+09</td><td>These are made in China (do not buy ANY pet food from China). Dogswell has been using propylene glycol to soften their treats (what are they thinkng?). Do not purchase or support this company in any way until they clean up their act. And for whatever reason Amazon doesn't allow returns of this item, so I had to toss mine out. Bad business all around on this one. </td><td style=\"text-align: right;\"> 0</td></tr>\n<tr><td>B0089SPEO2 </td><td>A3JOYNYL458QHP</td><td>Less lemon and less zing </td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 28582</td><td>coleridge </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.32391e+09</td><td>Everything is ok, except it just isn't as good as it is in the bags. Just considerably more bland -- less lemon and less zing. Boring. </td><td style=\"text-align: right;\"> 0</td></tr>\n<tr><td>B0041CIR62 </td><td>A16I6WJUEBJ1C3</td><td>okay but not as healthy as it appears</td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">138997</td><td>doctorsirena "doctorsirena" </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.34369e+09</td><td>I am always looking for healthier, whole grain versions of foods I enjoy. Unfortunately, these Peacock brand noodles are yet another food masquerading as healthy. The product title in big letters on the front says "Brown Rice Vermicelli", making the consumer think "this is made with brown rice, so it should be a healthy choice". But the first indication that it is not is when looking at the fiber content on the nutrition facts - only 0.6g per 2oz serving. Then onto the ingredients list to see why so low... contains brown rice, sago starch and water. The sago starch comes from palms and must not have much (if any) fiber.<br /><br />The Annie Chun's Maifun Brown Rice Noodles (sold on Amazon and in my local healthy grocer) has become one of my staples and is my frame of reference when comparing to the Peacock brand. The Annie Chun's product is made with 100% whole grain, with ingredients brown rice flour and water. Per 2oz serving, it has 4g fiber and pretty much the same calories and other nutrients as the Peacock brand.<br /><br />If you do try this Peacock brand noodles and have not used rice noodles before, you will need to seek guidance elsewhere on preparation. As others have pointed out, the Peacock package gives almost no directions on how to prepare the product, aside from a brief mention in the recipes (in the header text it does say that they are "easy-to-cook" but does not say how). It also contains a very strange recipe for rice noodles: Aglio Olio style - this is an Italian recipe for noodles with olive oil/garlic/sprinkled with grated cheese that I think would not be very tasty. The second recipe appears to be for a soup with veggie strips. Neither recipe gives amounts or much direction. In comparison, the Annie Chun's package gives clear, specific directions on rice noodle preparation and two recipes.<br /><br />I use rice noodles = maifun = rice sticks = sometimes called vermicelli for making the Vietnamese salad "bun tofu", to serve with stir-fried veggies or in lettuce rolls. They can also be used in spring rolls/egg rolls. When cooking with thin rice noodles, be careful not to oversoak/overcook/overmix or they tend to disintegrate. Asian rice noodle vermicelli (maifun) are not the same as Italian vermicelli and are not readily interchangeable. If making an Italian recipe, the best results would be expected from Italian pasta and not maifun.<br /><br />A few final notes... Both Peacock and Annie Chun's brown rice noodles are gluten free. The Peacock is made in Singapore and the Annie Chun's in Thailand. The Peacock noodles do taste fine (kind of bland), but so do the Annie Chun's. At this time, they are both approximately the same price. Peacock come in an plastic bag with some noodle crushage upon shipping; Annie Chun's are perfect upon removal from their cellophane bag in a box. Overall, I highly recommend the Annie Chun's Maifun as a healthier option over the Peacock brand. On a related note, the Annie Chun's soba and brown rice pad thai noodles are also excellent.<br /><br />Rating for this product: 2.5 stars rounded down to 2 stars.</td><td style=\"text-align: right;\"> 0</td></tr>\n<tr><td>B005K4Q4KG </td><td>A13L66J35SMYE5</td><td>Not good </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\">243065</td><td>Elizabeth Ramsoram </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.33065e+09</td><td>This does not taste like cocoa it is very weak no matter what cup size you use.I is just not good. </td><td style=\"text-align: right;\"> 0</td></tr>\n<tr><td>B00412W76S </td><td>A1ATV7O231DXIS</td><td>Not my cup of tea.. </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\"> 33918</td><td>JuneBug1783 </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.31233e+09</td><td>Supposedly increases milk production? Yeah, I didn't see a difference after drinking this for a good 3+ weeks. Aside from that, it doesn't smell or taste very good. Very herbal medicine-like. </td><td style=\"text-align: right;\"> 0</td></tr>\n<tr><td>B005BSR8GU </td><td>A3QLIJ6CHAA0BS</td><td>not seatles best... im pretty sure </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\">316099</td><td>D. E. Valazza </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.32771e+09</td><td>it seemed to have more caffeen than usual if that's even possiable.. aNd has a weird smell that everyone else could smell, but I could not. I thought it had less of the wonderful foldgers coffee... like before you use it when you just opened the tub of the stuff. THats a big this I missed with this coffee... however I do like the decafinated hazel nut pretty well... but this is not the best coffee i've ever had... douncan donuts is better :-/ </td><td style=\"text-align: right;\"> 0</td></tr>\n<tr><td>B001L4929Y </td><td>A27IP3T9ZKUMSS</td><td>Poor quality control </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">419889</td><td>W. Robert Kiser "Happily living in Tidewater"</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\">1.28347e+09</td><td>I've ordered these before and was quite pleased. This order was a disappointment. It would appear that the anchovy filets have broken down into a congealed mess of almost anchony paste. I will not order these again sight unseen. Although is product is about 20% more expensive in my local grocery, I will buy there where I can see exactly what I'll be taking home. My advice to others is to do the same. </td><td style=\"text-align: right;\"> 0</td></tr>\n<tr><td>B002HT1F88 </td><td>AFDVE0BR190N5 </td><td>TO MUCH VINEGAR </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\"> 93274</td><td>maryann rudder </td><td style=\"text-align: right;\"> 0</td><td style=\"text-align: right;\">1.3373e+09 </td><td>tHIS ORGANIC DRINK WAS NOT MY CUP OF TEA...THERE WAS A STRONG VINEGAR TASTE... TOO STRONG FOR ME...HOT OR COLD ..THE SMELL AND TASTE JUST WASN'T THERE...i DID NOT LIKE IT... </td><td style=\"text-align: right;\"> 0</td></tr>\n<tr><td>B008O3G2GG </td><td>A2IYX5W3A1N3VB</td><td>That's what she said </td><td style=\"text-align: right;\"> 1</td><td style=\"text-align: right;\"> 5</td><td style=\"text-align: right;\">436726</td><td>Orion </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\">1.32045e+09</td><td>This product is really dumb and pointless. My dog was simply not interested, and I was not interested in holding it in my hand while she licked it. When I ordered it, I thought I was getting something you could just give the dog, and she would entertain herself for a long time, by licking. It sounded like a good idea because my dog really likes to lick things. I give her something lick-able when I want her to go lay down and be quiet. This product is useless for that purpose. This is an interactive treat. You and your dog both have to interact with it. It gets old, pretty much immediately.<br /><br />The product itself is gross. It's sticky and unappealing. The smell is nauseating. My dog would take a sniff, and sometimes a trial lick, and then just walk away. I'd be left standing there holding a phallic shaped bottle of mysterious smelly goo.<br /><br />Here are some things my dog likes better than Lickety Stiks: dirty diapers, dead pigeons, week-old garbage, and other dog's butts.<br /><br />The only people who really enjoyed the product are the teenagers in my house, who think "Lickety Stik" is possibly the most hilarious product name they ever heard. The jokes lasted most of the afternoon. The packaging offers plenty of crude suggestions ("give a lick" tm they trademarked that??), as if my kids needed any encouragement. Even the shape of the bottle - a rigid cylinder with a ball on top - was fodder for hilarity. They thought it was just so funny.<br /><br />So this is a stupid product with a funny name and a suggestive shape, which solves no problems, has no appeal beyond comedy, and is completely worthless. </td><td style=\"text-align: right;\"> 0</td></tr>\n<tr><td>B001P76XAS </td><td>A14UH9I3P609X9</td><td>disappointed </td><td style=\"text-align: right;\"> 3</td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\"> 35521</td><td>Mike </td><td style=\"text-align: right;\"> 2</td><td style=\"text-align: right;\">1.31363e+09</td><td>I went to the store and noticed that Pop Secret changed the package size in the 6 Pack Homestyle from 3.5 to 3.2 ounces. Then they say you get more than Orville Redenbocker to try to justify the shrinking package. So, instead of getting 3.5 ounces as you did before, you're now getting 3.2 ounces and it takes less time to cook. I don't think this is right, I'm going to have to start shopping around even though I like this popcorn.<br /><a href="http://www.amazon.com/gp/product/B001P7531S">Pop Secret Homestyle Popcorn, Microwavable Popcorn, 3-Count, 10.5-Ounce Box (Pack of 6)</a> </td><td style=\"text-align: right;\"> 0</td></tr>\n</tbody>\n</table>"},"metadata":{}},{"output_type":"execute_result","execution_count":142,"data":{"text/plain":""},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%sql\nselect top 10 * from amazon_reviews where score =1","execution_count":174,"outputs":[{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"8d62e8df-e8a9-4df6-baea-65484a646ebd","version_major":2}},"metadata":{}}]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%sql\ninsert into amazon_reviews values ('B00141QYSQ', 'A1YS02UZZGRDCT', 'Do Not Buy', 2, 7, 9993329, 'Evan Eberhardt', 0, 1314489600, 'Nothing like what i expected');\ninsert into amazon_reviews values ('B0009XLVGA', 'A1NHQNQ3TVXTZF', 'An awesome choice', 5, 2, 999350, 'Evan Eberhardt', 0, 1314433500, 'You have to buy this! Its great!');\n\nselect * from GBM_W2V_MODEL_preds","execution_count":196,"outputs":[{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"2d6f1971-a4e5-4a5d-abba-e263fc94642b","version_major":2}},"metadata":{}},{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"949371b5-0a53-4c12-97e1-bdc7f4002693","version_major":2}},"metadata":{}},{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"931dcfff-9f42-4301-806b-2c86954db4f2","version_major":2}},"metadata":{}},{"output_type":"display_data","data":{"method":"display_data","application/vnd.jupyter.widget-view+json":{"version_minor":0,"model_id":"8434442a-0a49-4197-beb5-a03b76f8555e","version_major":2}},"metadata":{}}]},{"metadata":{"trusted":false},"cell_type":"markdown","source":"# Incredible!\n## Let's recap\n<blockquote>\n We:\n <ul>\n <li> Imported data from external sources in both SQL and Python</li>\n <li> Created a simple model with decent accuracy using H2O </li>\n <li> Imroved that model drastically using a word2vec pipeline with SKLearn </li>\n <li> Tracked, compared, and persisted all of our model and run information in mlflow </li>\n <li> Deployed the better model, along with the standalone word2vec pipeline to table directly in the database </li>\n <li> Chained those tables together with simple triggers </li>\n <li> Made predictions on new Amazon reviews </li>\n </ul>\n \n That's quite the accomplishment. Congratulations!\n</i></br><footer>Splice Machine</footer></blockquote><br>"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.7.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"toc":{"nav_menu":{},"number_sections":false,"sideBar":true,"skip_h1_title":false,"base_numbering":1,"title_cell":"Table of Contents","title_sidebar":"Contents","toc_cell":false,"toc_position":{"height":"calc(100% - 180px)","width":"212px","left":"10px","top":"150px"},"toc_section_display":true,"toc_window_display":true}},"nbformat":4,"nbformat_minor":4} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment