SparkML Fundaments 2.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "<p style=\"text-align:center\">\n    <a href=\"https://skills.network/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMBD0231ENSkillsNetwork26766988-2022-01-01\" target=\"_blank\">\n    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\"  />\n    </a>\n</p>\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# **Machine Learning with Apache Spark ML**\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Estimated time needed: **15** minutes\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "This lab goes introduces Machine Learning using Spark ML Lib (sparkml).\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "![](http://spark.apache.org/images/spark-logo.png)\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Objectives\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Spark ML Library is also commonly called MLlib and is used to perform machine learning operations using DataFrame-based APIs.\n\nAfter completing this lab you will be able to:\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "*   Import the Spark ML and Statistics Libraries\n*   Perform basic statistics operations using Spark\n*   Build a simple linear regression model using Spark ML\n*   Train the model and perform evaluation\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "***\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Setup\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "For this lab, we are going to be using Python and Spark (pyspark). These libraries should be installed in your lab environment or in SN Labs.\n"
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Collecting pyspark==3.1.2\n  Downloading pyspark-3.1.2.tar.gz (212.4 MB)\n\u001b[K     |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 212.4 MB 106 kB/s  eta 0:00:011    |\u258f                               | 1.1 MB 12.5 MB/s eta 0:00:17 |\u2588\u2588\u2588\u2588\u258a                           | 31.3 MB 12.5 MB/s eta 0:00:15\n\u001b[?25hCollecting py4j==0.10.9\n  Downloading py4j-0.10.9-py2.py3-none-any.whl (198 kB)\n\u001b[K     |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 198 kB 75.7 MB/s eta 0:00:01\n\u001b[?25hBuilding wheels for collected packages: pyspark\n  Building wheel for pyspark (setup.py) ... \u001b[?25ldone\n\u001b[?25h  Created wheel for pyspark: filename=pyspark-3.1.2-py2.py3-none-any.whl size=212880768 sha256=afe4be2e4862afde64faaaf10028c24b21ea6144eb8bcb808d19db8b1eb93272\n  Stored in directory: /home/spark/shared/.cache/pip/wheels/11/17/0b/53e7d10fe66ca7647d391cdba323fcf5b2f9dfcb7ebad87aa7\nSuccessfully built pyspark\nInstalling collected packages: py4j, pyspark\n  Attempting uninstall: py4j\n    Found existing installation: py4j 0.10.9.5\n    Uninstalling py4j-0.10.9.5:\n\u001b[31mERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: 'LICENSE.txt'\nConsider using the `--user` option or check the permissions.\n\u001b[0m\n\u001b[33mWARNING: Ignoring invalid distribution -y4j (/opt/ibm/conda/miniconda3.9/lib/python3.9/site-packages)\u001b[0m\n\u001b[33mWARNING: Ignoring invalid distribution -y4j (/opt/ibm/conda/miniconda3.9/lib/python3.9/site-packages)\u001b[0m\nCollecting findspark\n  Downloading findspark-2.0.1-py2.py3-none-any.whl (4.4 kB)\n\u001b[33mWARNING: Ignoring invalid distribution -y4j (/opt/ibm/conda/miniconda3.9/lib/python3.9/site-packages)\u001b[0m\nInstalling collected packages: findspark\n\u001b[33mWARNING: Ignoring invalid distribution -y4j (/opt/ibm/conda/miniconda3.9/lib/python3.9/site-packages)\u001b[0m\nSuccessfully installed findspark-2.0.1\n\u001b[33mWARNING: Ignoring invalid distribution -y4j (/opt/ibm/conda/miniconda3.9/lib/python3.9/site-packages)\u001b[0m\n\u001b[33mWARNING: Ignoring invalid distribution -y4j (/opt/ibm/conda/miniconda3.9/lib/python3.9/site-packages)\u001b[0m\n\u001b[33mWARNING: Ignoring invalid distribution -y4j (/opt/ibm/conda/miniconda3.9/lib/python3.9/site-packages)\u001b[0m\n"
                }
            ],
            "source": "# When you are executing on SN labs please uncomment the below lines and then run all cells.Next again Restart the kernel and run all cells.\n!pip3 install pyspark==3.1.2\n!pip install findspark\nimport findspark\nfindspark.init()"
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "metadata": {},
            "outputs": [],
            "source": "# Pandas is a popular data science package for Python. In this lab, we use Pandas to load a CSV file from disc to a pandas dataframe in memory.\nimport pandas as pd\nimport matplotlib.pyplot as plt\n# pyspark is the Spark API for Python. In this lab, we use pyspark to initialize the spark context. \nfrom pyspark import SparkContext, SparkConf\nfrom pyspark.sql import SparkSession"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Exercise 1 -  Spark session\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "In this exercise, you will create and initialize the Spark session needed to load the dataframes and operate on it\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 1: Creating the spark session and context\n"
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "metadata": {},
            "outputs": [
                {
                    "ename": "ValueError",
                    "evalue": "Cannot run multiple SparkContexts at once; existing SparkContext(app=python3.9, master=spark://jkg-deployment-807a44de-60ae-4899-9de2-8b31027e7e58-869959lmjkb:7077) created by getOrCreate at /usr/local/share/jupyter/kernels/python39/scripts/launch_ipykernel.py:84 ",
                    "output_type": "error",
                    "traceback": [
                        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
                        "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
                        "\u001b[0;32m/usr/local/share/jupyter/kernels/python39/scripts/launch_ipykernel.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Creating a spark context class\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0msc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mSparkContext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;31m# Creating a spark session\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0mspark\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mSparkSession\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m\\\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
                        "\u001b[0;32m/opt/ibm/spark/python/lib/pyspark.zip/pyspark/context.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, gateway, jsc, profiler_cls, udf_profiler_cls)\u001b[0m\n\u001b[1;32m    193\u001b[0m             )\n\u001b[1;32m    194\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 195\u001b[0;31m         \u001b[0mSparkContext\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_ensure_initialized\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgateway\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mgateway\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconf\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mconf\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    196\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    197\u001b[0m             self._do_init(\n",
                        "\u001b[0;32m/opt/ibm/spark/python/lib/pyspark.zip/pyspark/context.py\u001b[0m in \u001b[0;36m_ensure_initialized\u001b[0;34m(cls, instance, gateway, conf)\u001b[0m\n\u001b[1;32m    428\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    429\u001b[0m                     \u001b[0;31m# Raise error if there is already a running Spark context\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 430\u001b[0;31m                     raise ValueError(\n\u001b[0m\u001b[1;32m    431\u001b[0m                         \u001b[0;34m\"Cannot run multiple SparkContexts at once; \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    432\u001b[0m                         \u001b[0;34m\"existing SparkContext(app=%s, master=%s)\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
                        "\u001b[0;31mValueError\u001b[0m: Cannot run multiple SparkContexts at once; existing SparkContext(app=python3.9, master=spark://jkg-deployment-807a44de-60ae-4899-9de2-8b31027e7e58-869959lmjkb:7077) created by getOrCreate at /usr/local/share/jupyter/kernels/python39/scripts/launch_ipykernel.py:84 "
                    ]
                }
            ],
            "source": "# Creating a spark context class\nsc = SparkContext()\n\n# Creating a spark session\nspark = SparkSession \\\n    .builder \\\n    .appName(\"Python Spark DataFrames basic example\") \\\n    .config(\"spark.some.config.option\", \"some-value\") \\\n    .getOrCreate()"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 2: Initialize Spark session\n\nTo work with dataframes we just need to verify that the spark session instance has been created.\nFeel free to click on the \"Spark UI\" button to explore the Spark UI elements.\n"
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "\n            <div>\n                <p><b>SparkSession - in-memory</b></p>\n                \n        <div>\n            <p><b>SparkContext</b></p>\n\n            <p><a href=\"http://172.30.156.29:4040\">Spark UI</a></p>\n\n            <dl>\n              <dt>Version</dt>\n                <dd><code>v3.3.1</code></dd>\n              <dt>Master</dt>\n                <dd><code>spark://jkg-deployment-807a44de-60ae-4899-9de2-8b31027e7e58-869959lmjkb:7077</code></dd>\n              <dt>AppName</dt>\n                <dd><code>python3.9</code></dd>\n            </dl>\n        </div>\n        \n            </div>\n        ",
                        "text/plain": "<pyspark.sql.session.SparkSession at 0x7f191c5597f0>"
                    },
                    "execution_count": 4,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "spark"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 2: Importing Spark ML libraries\n\nIn this exercise we will import 4 SparkML functions.\n\n1.  (Feature library) VectorAssembler(): This function is used to create feature vectors from dataframes/raw data. These feature vectors are required to train a ML model or perform any statistical operations.\n2.  (Stat library) Correlation(): This function is from the statistics library within SparkML. This function is used to calculate correlation between feature vectors.\n3.  (Feature library) Normalized(): This function is used to normalize features. Normalizing features leads to better ML model convergence and training results.\n4.  (Regression Library) LinearRegression(): This function is used to create a Linear Regression model and train it.\n"
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "metadata": {},
            "outputs": [],
            "source": "from pyspark.ml.feature import VectorAssembler, Normalizer, StandardScaler\nfrom pyspark.ml.stat import Correlation\nfrom pyspark.ml.regression import LinearRegression"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Exercise 2 - Loading the data and Creating Feature Vectors\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "In this section, you will first read the CSV file into a pandas dataframe and then read it into a Spark dataframe\n\nPandas is a library used for data manipulation and analysis. Pandas offers data structures and operations for creating and manipulating Data Series and DataFrame objects. Data can be imported from various data sources, e.g., Numpy arrays, Python dictionaries and CSV files. Pandas allows you to manipulate, organize and display the data.\n\nIn this example we use a dataset that contains information about cars.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 1: Loading data into a Pandas DataFrame\n"
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "metadata": {},
            "outputs": [],
            "source": "# Read the file using `read_csv` function in pandas\ncars = pd.read_csv('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBM-BD0225EN-SkillsNetwork/labs/data/cars.csv')"
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>mpg</th>\n      <th>cylinders</th>\n      <th>displacement</th>\n      <th>horsepower</th>\n      <th>weight</th>\n      <th>acceleration</th>\n      <th>model</th>\n      <th>origin</th>\n      <th>car_name</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>18.0</td>\n      <td>8</td>\n      <td>307.0</td>\n      <td>130.0</td>\n      <td>3504.0</td>\n      <td>12.0</td>\n      <td>70</td>\n      <td>1</td>\n      <td>chevrolet chevelle malibu</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>15.0</td>\n      <td>8</td>\n      <td>350.0</td>\n      <td>165.0</td>\n      <td>3693.0</td>\n      <td>11.5</td>\n      <td>70</td>\n      <td>1</td>\n      <td>buick skylark 320</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>18.0</td>\n      <td>8</td>\n      <td>318.0</td>\n      <td>150.0</td>\n      <td>3436.0</td>\n      <td>11.0</td>\n      <td>70</td>\n      <td>1</td>\n      <td>plymouth satellite</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>16.0</td>\n      <td>8</td>\n      <td>304.0</td>\n      <td>150.0</td>\n      <td>3433.0</td>\n      <td>12.0</td>\n      <td>70</td>\n      <td>1</td>\n      <td>amc rebel sst</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>17.0</td>\n      <td>8</td>\n      <td>302.0</td>\n      <td>140.0</td>\n      <td>3449.0</td>\n      <td>10.5</td>\n      <td>70</td>\n      <td>1</td>\n      <td>ford torino</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "    mpg  cylinders  displacement horsepower  weight  acceleration  model  \\\n0  18.0          8         307.0      130.0  3504.0          12.0     70   \n1  15.0          8         350.0      165.0  3693.0          11.5     70   \n2  18.0          8         318.0      150.0  3436.0          11.0     70   \n3  16.0          8         304.0      150.0  3433.0          12.0     70   \n4  17.0          8         302.0      140.0  3449.0          10.5     70   \n\n   origin                   car_name  \n0       1  chevrolet chevelle malibu  \n1       1          buick skylark 320  \n2       1         plymouth satellite  \n3       1              amc rebel sst  \n4       1                ford torino  "
                    },
                    "execution_count": 7,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# Preview a few records\ncars.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "For this example, we pre process the data and only use 3 columns. This preprocessed dataset can be found in the `cars2.csv` file.\n"
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>mpg</th>\n      <th>hp</th>\n      <th>weight</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>18.0</td>\n      <td>130.0</td>\n      <td>3504.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>15.0</td>\n      <td>165.0</td>\n      <td>3693.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>18.0</td>\n      <td>150.0</td>\n      <td>3436.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>16.0</td>\n      <td>150.0</td>\n      <td>3433.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>17.0</td>\n      <td>140.0</td>\n      <td>3449.0</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "    mpg     hp  weight\n0  18.0  130.0  3504.0\n1  15.0  165.0  3693.0\n2  18.0  150.0  3436.0\n3  16.0  150.0  3433.0\n4  17.0  140.0  3449.0"
                    },
                    "execution_count": 9,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "cars2 = pd.read_csv('https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBM-BD0225EN-SkillsNetwork/labs/data/cars2.csv', header=None, names=[\"mpg\", \"hp\", \"weight\"])\ncars2.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 2: Loading data into a Spark DataFrame\n"
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {},
            "outputs": [],
            "source": "# We use the `createDataFrame` function to load the data into a spark dataframe\nsdf = spark.createDataFrame(cars2)"
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "root\n |-- mpg: double (nullable = true)\n |-- hp: double (nullable = true)\n |-- weight: double (nullable = true)\n\n"
                }
            ],
            "source": "# Let us look at the schema of the loaded spark dataframe\nsdf.printSchema()"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 3: Converting data frame columns into feature vectors\n\nIn this task we use the `VectorAssembler()` function to convert the dataframe columns into feature vectors.\nFor our example, we use the horsepower (\"hp) and weight of the car as input features and the miles-per-gallon (\"mpg\") as target labels.\n"
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {},
            "outputs": [],
            "source": "assembler = VectorAssembler(\n    inputCols=[\"hp\", \"weight\"],\n    outputCol=\"features\")\n\noutput = assembler.transform(sdf).select('features','mpg')"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "We now create a test-train split of 75%-25%\n"
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {},
            "outputs": [],
            "source": "train, test = output.randomSplit([0.75, 0.25])"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Exercise 3 - Basic stats and feature engineering\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "In this exercise, we determine the correlation between feature vectors and normalize the features.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 1: Correlation\n\nSpark ML has inbuilt Correlation function as part of the Stat library. We use the correlation function to determine the different types of correlation between the 2 features - \"hp\" and \"weight\".\n"
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Pearson correlation matrix:\nDenseMatrix([[1.        , 0.86712711],\n             [0.86712711, 1.        ]])\n"
                }
            ],
            "source": "r1 = Correlation.corr(train, \"features\").head()\nprint(\"Pearson correlation matrix:\\n\" + str(r1[0]))"
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Spearman correlation matrix:\nDenseMatrix([[1.        , 0.89651913],\n             [0.89651913, 1.        ]])\n"
                }
            ],
            "source": "r2 = Correlation.corr(train, \"features\", \"spearman\").head()\nprint(\"Spearman correlation matrix:\\n\" + str(r2[0]))"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "We can see that there is a 0.86 (or 86%) correlation between the features. That is logical as a car with higher horsepower likely has a bigger engine and thus weighs more. We can also visualize the feature vectors to see that they are indeed correlated.\n"
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAABBaElEQVR4nO29e5xdZX3v//5kMoQJtyQSLUwSEhGhBEoiKeBJbQGrIIiMogItgtaK9acVlBMNliNQQ0mlKPWcU1rUFhQLhIsxKBSVqyKBMzEJEIFyJ5lEiJAgkhgmk+/vj/XsYc2etfZa+7L23jPzfb9e85q1n3XZ37X23s/3eb63R2aG4ziO41RiXKsFcBzHcdofVxaO4zhOJq4sHMdxnExcWTiO4ziZuLJwHMdxMnFl4TiO42TiymIUI+mjkn5ex/m3SjqjkTKF614paVGjr+s49X7nq3iff5X0v3IeOyq+764sCkbSX0jqlfQ7SRtCB/wnrZarHEkXSLo63mZm7zGzq1olUxIj4Ycn6S5Jf13WdqSkda2SyQFJt0n6Qux1tyRLafuDStcys78xs680SC6T9JZGXKtIXFkUiKTPA5cB/wC8CZgB/AtwYg3XGp+nzRl9jKbPWVJHC9/+HuDPYq//FHg0oe1xM/t1MwUbCbiyKAhJewB/D3zazG4ys1fNrN/MbjazBeGYCZIuk7Q+/F0maULYd6SkdZK+KOnXwH+E0f8Nkq6W9Fvgo5L2kPTtMGvpk7Qo7Qcp6Z8lrZX0W0krJL0jtB8LfAk4OcyAVof2wRGypHGSzpP0rKQXJH0n3COSZobR0RmSnpP0G0l/l/GI9pT0E0mvSLpb0j4xOQ8I+16S9JikD4f2M4G/BL4Q5LxZ0sck3Rw79wlJS2Kv10qaU+m6sc/in4L8zwczQ1fZZ3FOuPcNkj6W9R2ohKS9JS0Lsjwh6ROxfUmf82FhhvrbIN/XYscfIekXkjZLWi3pyNi+uyRdLOkBSS9L+oGkKbH975O0Jpx7l6Q/DO2Neq5XSrpc0i2SXgWOSngWH5P0SPguPCXpk7F9FZ+9pDeE5/hbSQ8A+1Z47PcA8yWV+r13EA3m5pW13ZPzvhbFXn8hyLZe0l9r+GxhsqQfhXu8X9K+4bx7wv7V4Tt9cgX5W4uZ+V8Bf8CxwHZgfIVj/h5YDrwRmAr8AvhK2HdkOP8fgQlAF3AB0A/0ECn6LmAp8G/ALuE6DwCfDNf4KPDz2PudBrwBGA+cA/wa2DnsuwC4uky+u4C/Dtt/BTwBvBnYFbgJ+G7YNxMw4JtBpkOAbcAfptz3lcArRKO4CcA/l+QM97EW+FiQ823Ab4DZsXMXxa71ZmBzeB57Ac8CfbF9m8K+rOteBiwDpgC7ATcDF5d9Fn8PdALHAVuAySn3N/jcYm1HAutir+8mmmXuDMwBNgLvjH0W5Z/zfcBHwv5dgSPCdjfwYpBpHPCu8HpqTJY+4KDwDG4sfc7AW4FXwzmdwBfCZ7xTA5/rlcDLwPxw/M4Jz+t4ok5eRKP8LcDb8jx74FpgSZDjoHCvP0/5XCYAW4G54fXD4V7uLWs7Ped9LYr91n8NzAYmAt8l+j28JXbsS8Bh4VrfA66NyTV4bDv/tVyA0fpHNAL+dcYxTwLHxV4fAzwTto8EXov/uIg6kXtir99E1Cl3xdpOBe4M2x9N++GE/ZuAQ2LXrqQsbgf+v9i+/Yk6tPG8riymxfY/AJyS8r5Xlv1YdgUGgOnAycDPyo7/N+D82LmLyvavDT/mU4ArwnsfEH7oy8Ixqdcl6qReBfaN7Xs78HTss9hKTPEDLxA67IT7u4uoQ9sc+/sdQVmE+xwAdoudczFwZdLnHNruAS4E9ixr/yJBacfabgPOiMmyOLbvwPC96gD+F7Aktm8cUWd7ZCOea+zz+k6Vv52lwFlZzz7cQz9wQGzfP1D5O38XcBbRoKD0eSyOte0A9sl5XyVl8e+EgUV4/RaGK4tvxfYfBzwaez0ilMWosYW2IS8SmVrGm9n2lGP2JhqxlXg2tJXYaGa/LztnbWx7H6LR1gZJpbZxZccMIukc4K/DexiwO7Bn9q2kyjqeSGGViNt5txApgTQGZTSz30l6KbzHPsDhkjbHjh1PNFpL426iTuUtYXsz0Qj17eE1GdedSjQiXBF7jiLqjEq8WPY5Zt3fZ83sW4MXi0xDpQCCvYGXzOyV2PHPAvNir8s/w48Tja4flfQ0cKGZ/TDc14cknRA7thO4M+Vaz4b9e1L2mZrZDklriWYrUP9zTbuXIUh6D5HSfivR93ci8FDskLRnPzW8V/n9VeIeohntM0ApaurnRArwGWCtmT0bzEF5v4d7A72x10n3W81voy1xZVEc9wG/JzIl3JByzHqiH9ua8HpGaCuRVBI43raWaGaxZwWFBIAi/8QXgXcCa0LHsImoU0x7ryRZS8wgMg88D0zLODeJ6THZdiUa1a0nuqe7zexdKeclyXk3cAIwi2hkuZloZvd24P+EY1KvG+zVW4lMDH013Eu1rAemSNotpjBmEI3qSwy5TzN7HDg1yPoB4AZJbyC6r++a2SdIZ3psewbRaPw3QY6DSzsUacrpMTnqeq5p9xJHkY/uRiLTzw/MrF/SUl7/XlZiI9F3cDqRo7p0f5W4B/gbIsXws9B2L/Ct0FbyIeS5rxIbGPobmJ524EjGHdwFYWYvA18G/q+kHkkTJXVKeo+kr4bDrgHOkzRV0p7h+KvTrpnwHhuAHwOXStpdkRN6X0l/lnD4bkQ/rI3AeElfJppZlHgemBlz9JVzDfA5SbNC5/4PwHVZSqoCx0n6E0k7AV8B7jeztcAPgbdK+kh4Xp2S/rjkeA1yvrnsWncTOU67zGwdUSdwLJF/ZmU4JvW6ZraDyN/ydUlvhMEQymNqvLeKhPv8BXCxpJ0l/RHRzOF7aedIOk3S1CDr5tA8QPR9OUHSMZI6wvWOlBTvvE6TdKCkiUSzkxvMbIDI1n+8pHdK6iTyY20LskGdzzXn49iJyJewEdgeZhnvznNiuIebgAvC7+tA4IyM034BTCLy3/0sXGdTeP/TeF1ZVHNfS4CPSfrD8Iy/nEf+GEnf6bbDlUWBmNnXgM8D5xF9GdcCnyGyyQIsIpq+Pkg07f5laKuG04l+cL8i8kHcQOSQLOc24Fbgv4mm6r9n6HT5+vD/RUm/TDj/34mm4PcAT4fz/7ZKWeP8J5Hp4SXgUKIRK2Gk/W4iO/l6oul7yckP8G3gQEXRO0vDOf9N5BMo/fh/CzwF3Bs6lDzX/SKRc3e5ogiknxL5ZYriVCJfz3rg+0S28J9UOP5YYI2k3xEFBJxiZr8PiudEomi20ndsAUN/298lspv/msih/lkAM3uMqIP830QzjROAE8zstbC/Ec+1IuH8zxJ1uJuAvyAKNMjLZ4hMOr8O9/gfGe+3BVgR5Hs4tutnRAEi98TkynVfZnYr8A0i098TRFYFiBRvHi4Argrf6Q9nHdwqFBwsjuOMQiTdRRS48K2sY53GEGYfDwMT6ph5tx0+s3Acx6kTSe+XtJOkyUQzkJtHk6IAVxaO4ziN4JNEZsAniXxJn2qtOI3HzVCO4zhOJj6zcBzHcTIZtXkWe+65p82cObPVYjiO44woVqxY8Rszm1rePmqVxcyZM+nt7c0+0HEcxxlEUmIWvJuhHMdxnExcWTiO4ziZuLJwHMdxMnFl4TiO42TiysJxHMfJZNRGQzmO03qWruzjktseY/3mrew9qYsFx+xPz9zu7BOdtsOVheM4hbB0ZR/n3vQQW/sHAOjbvJVzb4rWNHKFMfJwM5TjOIVwyW2PDSqKElv7B7jktsdaJJFTD64sHMcphPWbt1bV7rQ3riwcxymEvSd1VdXutDeuLBzHKYQFx+xPV2fHkLauzg4WHFPkAoROUbiD23GcQig5sT0aanTgysJxnMLomdvtymGUUKgZStIzkh6StEpSb2ibIuknkh4P/yfHjj9X0hOSHpN0TKz90HCdJyR9Q5KKlNtxRgJLV/Yxf/EdzFr4I+YvvoOlK/taLZIzimmGz+IoM5tjZvPC64XA7Wa2H3B7eI2kA4FTgNnAscC/SCoZPC8HzgT2C3/HNkFux2lbSjkMfZu3Yryew+AKwymKVji4TwSuCttXAT2x9mvNbJuZPQ08ARwmaS9gdzO7z6I1YL8TO8dxxiSew+A0m6J9Fgb8WJIB/2ZmVwBvMrMNAGa2QdIbw7HdwPLYuetCW3/YLm8fhqQziWYgzJgxo5H34ThtxWjKYWhFSZCRWIak1TIXrSzmm9n6oBB+IunRCscm+SGsQvvwxkgZXQEwb968xGMcZzSw96Qu+hIUw0jLYWhFSZCRWIakHWQu1AxlZuvD/xeA7wOHAc8H0xLh/wvh8HXA9Njp04D1oX1aQrvjjFlakcNQhEO9Fea0kWjCaweZC1MWknaRtFtpG3g38DCwDDgjHHYG8IOwvQw4RdIESbOIHNkPBJPVK5KOCFFQp8fOcZwxSc/cbi7+wMF0T+pCQPekLi7+wMGFj8Yb7VBvhTltJJrw2kHmIs1QbwK+H6JcxwP/aWb/Jen/AUskfRx4DvgQgJmtkbQE+BWwHfi0mZVU6aeAK4Eu4Nbw5zhjmmbmMFQa2dYjQyvMaSPRhNcOMhc2szCzp8zskPA328wuCu0vmtk7zWy/8P+l2DkXmdm+Zra/md0aa+81s4PCvs+EqCjHcZpEUSPbVpjTRmIZknaQ2TO4HcfJpKiRbStKgozEMiTtILNG6yB93rx51tvb22oxHGdUUB6NA9HItkg/idMaJK2IJVEP4jMLx3EyaYeRrdNaXFk4jpMLLwo4tvH1LBzHcZxMfGbhOE7NtLoEhdM8XFk4jlMT7VCCIo12U2LtJk8tuBnKcZyaaIcSFEm0W/n2dpOnVlxZOM4IpdWLH7VDCYok2k2JtZs8teLKwnFGIO0wWk1LyGt12Yx2U2LtJk+tuLJwnBFEaTZx9nWrWj5aXXDM/sM6kHGhvZW0mxJrN3lqxZWF47SAWkxI8dlEGs0crfY++xI7ytp2hPZW0g51lNpZnlrxaCjHaTK1RhEl2b7LaeZo9Zr716a2L+o5uGlylNOsbPO8EU6jJfvdlYXjNJlay31nzRqaPVodSKkrl9beTIrONm/nsOGicDOU4zSZWh2elWYNRS9+lESHklY8Tm8fTVQT4dQOwQiNwJWF4zSZWh2eabbvy06ew70Lj276iPbUw6dX1T6aqEbhe+is4zg1UavDs9lLqWaxqOdgTjtixuBMokPitCNmtNRf0SyqUfijJXTWfRaO02TqcXi2W+XXRT0HjwnlUM6CY/ZPXN8jSeG3w5KojcCVheO0gHbr9J3qqEbhV6NY2pnClYWkDqAX6DOz90q6APgEsDEc8iUzuyUcey7wcWAA+KyZ3RbaDwWuBLqAW4CzfB1uZ6yTFbo5UorXjRQ5y8mr8D10Nj9nAY8Au8favm5m/xQ/SNKBwCnAbGBv4KeS3mpmA8DlwJnAciJlcSxwaxNkd5y2JCt0c6SEdo4UOetlNMwkC3VwS5oGHA98K8fhJwLXmtk2M3saeAI4TNJewO5mdl+YTXwH6ClKZscZCWRF2LRLBE5Wpnq7yOlkU/TM4jLgC8BuZe2fkXQ6kXnqHDPbBHQTzRxKrAtt/WG7vH0Yks4kmoEwY8aMBojvOK2jknkmreRHqT1PBE7R5p88s4bREik0FihsZiHpvcALZraibNflwL7AHGADcGnplITLWIX24Y1mV5jZPDObN3Xq1Jrkdpx2ICuRKy3vrdSeFdrZjESxPLOGRhTZa3Wp9mbR6vss0gw1H3ifpGeAa4GjJV1tZs+b2YCZ7QC+CRwWjl8HxLN5pgHrQ/u0hHbHGbVkdbRp4R2l9qxcjmaYf/LMGuotsjdasqOzaIf7LExZmNm5ZjbNzGYSOa7vMLPTgg+ixPuBh8P2MuAUSRMkzQL2Ax4wsw3AK5KOkCTgdOAHRcntOO1AveaZrAS+Zph/8swa6k00HCs+j3a4z1bkWXxV0hwiU9IzwCcBzGyNpCXAr4DtwKdDJBTAp3g9dPZWPBLKGeVkJXJNntjJpi39w/ZPntg5uF0pAqcZiWILjtmfBdevpn/H69OgznFqaH5Bo5Revf6bov0/7eDbaYqyMLO7gLvC9kcqHHcRcFFCey9wUEHiOU7bkZXIdf4Js1lww2r6B2IdcYc4/4TZg6+TOjCIRql9m7cihjr/CkkUK/etlL2uN3S2EUqvXhkaFf5bSeG0Qxa414ZynDakknmm1Kn0D9hgXabuSV1c8sFDBjuXJBv3ghtWs+D61YOdTjx6pIg6UyUZ4/QP2BDTSb3mlVp9HnFn8TlLVtclQyNMRFk+iXZYQMnLfThOm5JkRiofxQ6YDXYa8WOTOrDyjhsihdE9qYt7Fx7dcPnzmE4a4ZvpffYlrrl/LQMWKc+TDq2cAJf0DOuRoREmoqw1TtohC9yVheOMIPIunFRNR1WU3TuP6aRe88rSlX3cuKJvsMMfMOPGFX3M22dKakeaZ8XBamTIcw9ZPo08CqfVWeBuhnKcGqg35r3W8/OOYquxZRdl985jOqnXvFKLCSiPcqxGhqx7yBP22oh8k6JxZeE4VVJvzHs95+ftVJI6sM4O0TluqIc5q1OsRynmCYutN3S2FhNQ2jPskGqSIese8ii0Rvgkik7aczOU41RJrWtoN+L8pCgpgFe3bWfpyr7B89Ns3Eltae/ZiCifPKaTeswrtZix0iLN6nHwV7qHvCYmqN0n0YyCjK4sHKdK6nVo1nN+6Yd/4c1rhuRZbN7az4IbVg85Jq0Da0TCW7tUUK1lrYhmO4vzKrR6lGYzPitXFo5TJfU6Zes9v2duNxfevGZYe/+AcfZ1q7jktsca0vk1KxGsnoS2Wjv+ZjqLm7H4UTM+K1cWjlMl9f74jzpgKlcvfy6xPS9J2dslGmWCmJSSJT4pliVeL80ydbWSZsxkmpG058rCcaqk3h//nY9urKq9FhphgsgqVtgIRoKpqxEUrdCaMXtxZeE4NVDPj78RJgMpu9Ou1wTx8tbk2Uu8vd6aSO1Q82g00IzZiysLx2kyeU0GlTriPKP7cdKQCKlGy9kIE1I71DwaLRQ9e/E8C8dpMmm+iXh7Vi5Gd47OdMCMBdevrjnevhlrYrRDzSMnH64sHKfJ5PFZZHXEeZ3h/TuMC5YNj5zKQzPWxKg3Kc9pHm6GcpwaqMdW34gCe9U4wzen+B7yUO+aGHmeU7tHMzkRPrNwnCqpt9xHnpIdWcckddLNphE1kZyRgysLZ8TR6oXr67XVp5mQNm95bfCeZr4hWVmUzi1fU6gSkxuYFxGnETWRnJGDm6GcEUUzauBkUa2tvtwUs+W17YnHvfra6/eUNnP44eoNLOo5mLypDuWr5zWaemsiOSMHn1k4I4p2GK1WU046yRRTKfs6izz+h/hIP756XrMZCWW3nfwUriwkdUhaKemH4fUUST+R9Hj4Pzl27LmSnpD0mKRjYu2HSnoo7PuGpGpm4c4ooh1Gq9WEe+ZdaKda0kxLkyd2cu/Co3l68fHcu/DoljqOPSx2dNGMmcVZwCOx1wuB281sP+D28BpJBwKnALOBY4F/kVT6pl0OnAnsF/6ObYLcThvSDqPVasI9G63ESkri/BNm09kxdMxUtMmpWjwsdnRRqM9C0jTgeOAi4POh+UTgyLB9FXAX8MXQfq2ZbQOelvQEcJikZ4Ddzey+cM3vAD3ArUXK7rQnjaiBU2+JCsgf7pkWXjqpq5NdJoxn/eat7NHVyauvbR+yRnbHODGwY7hn4vg/2mvw/aG1azLnwcNiRw9FzywuA74A7Ii1vcnMNgCE/28M7d3A2thx60Jbd9gubx+GpDMl9Urq3bixcUXZnPah3tFqs8M5k0wxENV2WnDM/jy9+HhWnf9uLvngIUPuabcJyeO4eH5F77Mv8euXf48Bv3759/Q++1Ih9+A4UODMQtJ7gRfMbIWkI/OcktBmFdqHN5pdAVwBMG/evAbWxnTaiXZfJCZO6ZoXLFszxDm9aUv/kCiu8nuatfBHidcrmbXOW/rQkDLnA2aDrxf1HNzYm3Acip1ZzAfeF8xI1wJHS7oaeF7SXgDh/wvh+HXA9Nj504D1oX1aQrszRqknz6IVDvKeud3skjBTqBTFleWbueb+tYn709odp14KUxZmdq6ZTTOzmUSO6zvM7DRgGXBGOOwM4AdhexlwiqQJkmYRObIfCKaqVyQdEaKgTo+d44xCKimDZmRPZ8lQC9UqqaxIooGUsrNp7Y5TL63Is1gMvEvS48C7wmvMbA2wBPgV8F/Ap82sZC/4FPAt4AngSdy5PWrJUgb15lnkCefMo5CqVSbVRnH1zO3mpEO76QhR4h0SJx36uqlqXErweFq749RLU5SFmd1lZu8N2y+a2TvNbL/w/6XYcReZ2b5mtr+Z3Rpr7zWzg8K+z5j58Gm0kqUM6jUj5XGQZ8lQy+ym2pyDpSv7uHFF3+BMYcCMG1f0Db7HhPHJP920dsepFy/34bQVWcqgEYvlZDnIs2SoxUlebahr1nv8vn9H4nlp7aOZRoRCO9m4snDaiixl0Iw8iywZ0uo2xdvT3qNRZcxHygpzRXfk7VArbKzgc1anrcgy1zQjzyJLho6UajOl9kbkcmT5OEZCKY1m5LS0Q62wsYLPLJyGkjWSzNqfx1xTdJ5FlgxZkUiNyOXImkGNhAzuZuS0tEOtsLGCKwunYWSZBPKaDIosEZG3c6kkQ3eKCai0LnajlhuF9lYGWTSjIx8p5rjRgJuhnIaRZRJoB5NBIwoRZpmAGlXssGdud2oF2aUr+/j8klVDTDyfX7KqrVaha0bRx5FgjhstuLJwGkbWSLJRI816EuYa0blk+U3y5nLUk/T3pZsepLzO4A6L2tuFZnTkXtm2ebgZymkYWSaBvCaDSn6NeqNf6jXvlMv29ZPnDDs36z0aEcGzJSVENq29FTTLlOaVbZuDRmt+27x586y3t7fVYowpyjtBiEaSpZFe1v4815i/+I5Uf8G9C48u8O6yZctLI+5hZkqhQYBnFh+fW5YsPIdh7CFphZnNK293M5TTMLJMAo3Inm5l9EujfC6NuIe0tSIbuYZks8u5O+2Nm6GchpJlEqg3e7qV0S+NUlSNuIe/PHzGkBLl8fZG0axy7j57GRn4zMJpKlmO3TzJaJ1l1fI6x6kp0S+Niu4ZKRE8zZjF+exl5ODKwmkaS1f2seD61UM6hgXXr64qexpge1kYUPnromhUJ9+ICJ5mrGfRjNDXdgindvLhZiinaVywbA39ZR17/w7jgmVrhvg1ep99iWvuX8uA2bDS3F+66cFhyyRaaG+WaaQRJpN6I3gasZ5FlvmnEXW4svAM7JGDKwunacSXFU1rTyvNPW+fKfTM7W5KyGhWaGs72NM7pETFkFa3qpw84bvNCH31DOyRQy4zlKSz8rQ57UejV3wrmnYwSxQtQyM+k1MPn15Vezl577FSFnkjGCn+Gyf/zOIM4J/L2j6a0Oa0Ee1WvnnyxE42bRk+u5g8sXNwO8ssIUGSpaWRIaNFmkYa9Zks6jkYYIi57tTDpw+2Z9HIbPp6Zh6joQbWWKGispB0KvAXwCxJy2K7dgNeLFIwpz6WruzjnCWrh5kqigh9zMv5J8xmwQ2r6R94XabODnH+CbMHX2eZJZoRMlqkaaSR4aiLeg7OrRzKacQ9NkrxtYtpz6lMlhnqF8ClwKPhf+nvHODYYkVzaqX0I05zdjbbeVgyu3zuulXsOmE8k7o6B6OALvngIUM6iqMOmJp4jVL7op6Dmb/vlCH75u87peZOM4kiTSPt4tBtxD22g8nQaR4VZxZm9izwLPD2ai8saWfgHmBCeJ8bzOx8SRcAnwA2hkO/ZGa3hHPOBT4ODACfNbPbQvuhwJVAF3ALcJavw51O0o84TjOdh+Wjz01b+unq7EisqQRw56Mbh7XF25eu7OOXz708ZN8vn3uZpSv7GjY6zWMaqdX80i4O3UaYf9pF8TnNIZfPQtIHgH8E3ggo/JmZ7V7htG3A0Wb2O0mdwM8l3Rr2fd3M/qnsPQ4ETgFmA3sDP5X0VjMbAC4HzgSWEymLY4FbcRKp9GNttvOwWrNLEetf10Il00g95pdmhKPmpV7zT7soPqc55E3K+yrwPjPbw8x2N7PdMhQFFvG78LIz/FWaDZwIXGtm28zsaeAJ4DBJewG7m9l9YTbxHaAnp9xjkrQfa4dUMfmriMipakefWYlg7TCarcf8MppKansk09gir7J43sweqfbikjokrQJeAH5iZveHXZ+R9KCkf5c0ObR1A/H003WhrTtsl7cnvd+Zknol9W7cmGzOGAuk/Ygv/fAhmaPlRpddqDYLOMtn0Yys4izqVVg9c7tZcMz+7D2pi/Wbt3LJbY+1bUhzpQHEaFJ8TjZZ0VAfCJu9kq4DlhKZlwAws5sqnR9MSHMkTQK+L+kgIpPSV4hmGV8hcpj/FZFpa9glKrQnvd8VwBUQlSivJNtIp5LNvBZ7dFHmnSSzi4iU0fzFdwyTK8tn0Q5mnHrNL3nNWFl+kfOWPlRz6Gyj5PRIprFDls/ihNj2FuDdsdcGVFQWgweabZZ0F3Bs3Fch6ZvAD8PLdUA8o2gasD60T0toH7MU8SPOO1qu1rEbV1x9m7dGzq6wL0nupE443p5VDiQP9eYGpCmsow6YyvzFd2ReN49izvqMz1v60JAQ4gGzwdeNUhjN8g85I4OKZigz+1iFv7+qdK6kqWFGgaQu4M+BR4MPosT7gYfD9jLgFEkTJM0C9gMeMLMNwCuSjpAk4HTgB7Xc7GihiJDFPOadWk1VpSzg7kldw6aE1cqdVg4krxmnEea2JPPLSYd2c+OKvlzXzaOYsz7jZhQSbAf/kNM+5I2G+kZC88tAr5mlddx7AVdJ6iBSSkvM7IeSvitpDtEA8xngkwBmtkbSEuBXwHbg08GMBfApXg+dvZUxHglVxI84j3mn3pFmI+SuV4ZGjZbLZ27zF9+R+7p5zFhZz6oRhQSz8GgnJ05eB/fOwBzg8fD3R8AU4OOSLks6wcweNLO5ZvZHZnaQmf19aP+ImR0c2t8XZg6lcy4ys33NbH8zuzXW3huusa+ZfWas51gU4eTN46zMMhFlkUfutKodpfZ6FU5Ro+VqrpsniqgdHPke7eTEyVsb6i1EORPbASRdDvwYeBfwUEGyOSkU5eTN8nPUW+l0wTH7J5b7iMu90/hxbNs+vILsTuOjcc3EnTp49bXhCYcTd+oY1pZEUaPlaq6bJwChHRz5XrfJiZNXWXQDuxCZngjbe5vZgKRt6ac5RdCqH3EjTB8DZetZlL9OUhTx9i0JiqJSezlFdcLVXjfP8rOQ/hl3pyin7gbPPDzaySmRV1l8FVgVIpoE/CnwD5J2AX5akGxOBZr1I45HDqXNLPJ2UBfevIbyRe12WNSe917S1FJedVWUoi3iupU+43aYeThji1zKwsy+LekW4DAiZfElMyuFry4oSjintZSHbyYpimo6qKTy5OXtk7o6ExdJmtQVlTGv1xQGxSnaZo7C3UTkNJuspLwDzOxRSW8LTaW4vD+Q9Adm9stixXNaSVZBwqQch3pzGC5432wWXL96yPKrnePEBe+Lypifevj0xBLleRf9GU24ichpJlkzi88TFfC7NGGfAUc3XCIHqL/TbQRZUU7lS542Yn2DrBFznkV/2uHZOc5oQ6M1CnXevHnW29vbajFqorzThcjcU2/dnWo70X3PvSWX87p7Uhf3Ljya+YvvSHW63rvwaGYu/FHqNZ5ZfHy+m8igqGfnOGMFSSvMbF55e941uCdKOk/SFeH1fpLe22ghnYgiMrRryVzOG+VUyiVoVsZvpeJ2viCP4xRD3qS8/wBeA/5HeL0OWFSIRE4hnW4tnWjeKKdSjkNWIllWwl0espTeSChRUUQpeMcpmrzKYl8z+yrQD2BmW6nuN+5UQaVOt9aOppZONCmDN4lSjkNWefF6w14hW+m1Q+ZzJYoqBe84RZNXWbwWigEagKR9iZUqdxpLWpmFow6YOqyjOfu6Vcy58MdDOpskhVJLJ1peAiSNUmefVV48baZSTSJZltIrukRFvbMCN5M5I5W8SXnnA/8FTJf0PWA+8NGihBrrpEUEpYWybt7aPxh1BCRGJJWqohaRxFXKcciqHZWUSNY5Tmx5bTuzFv4ol9M9q6xGo/IPkoIBIPnZxt83i5FgJnOcJHJFQ0n6LlENqK3AU8D9ZvabgmWri5EcDZXGrIU/qmiyKY3Q0yKSSgonbyeaFFmUxGlHzGBRz8Gp0VMdEk9efNzgNUsy7NHVyauvbR9SK6qrs4OTDu3mzkc3JsrZjGintPeYMH5cYsJgKdorD1kRY47TatKiofLOLP4D+BOiwoFvJir9cY+Z/XMDZXQySBtVl6g0Ol2/eWvVSVx5kvLiOQ55akfFZZi/+I5hne/W/gG+t/y51AWSmpG5nGYqSnsW1cwKvEyHM1LJW+7jDkl3A38MHAX8DTAbcGXRRJI6mjh7V5hZ1OLgrdQJCviDPXZm3j5TBtsmT+xMLOkxeWJnVddPWyCpWUt5VmsSqubZepkOp1ZanWyad/Gj24kqzd4H/Az4YzN7oUjBnOGUvhgX3rxmWKccH53WOnIt/zLukVKnCRgSyVOSLc2imdaeNVOK00ybfppckyd28vv+HXXPCrxMh1MtjaiOUC95o6EeJMqzOIho4aODQnSU02R65naz8svv5rKT5yQuVJRnEaMkkkI6X31te+YXJB7J83KKYklrT4pcSou6aodFf84/YXZNz9Zx6qUdoujymqE+ByBpV+BjRD6MPwAmFCeaU4lKo9NaRq5JX8a447kSpVF/tQsLJZlkjjpgambUVtHT8SxTkSsHp9m0QxRdXjPUZ4B3AIcCzwL/TmSOckYJ9XzpSsqgFudtkmKbt8+U1I66WdNxNxU57UQ7rIeeNxqqC/gasKK0tGoWknYG7iGafYwHbjCz8yVNAa4DZgLPAB82s03hnHOBjwMDwGfN7LbQfihwZZDjFuCssb4Odz0kjcwr+Scq0Tnu9WVRG+W8rdRRV5qOe+fujFbaIYourxnqkhquvY1o3e7fSeoEfi7pVuADwO1mtljSQmAh8EVJBwKnEEVZ7Q38VNJbzWwAuJyoVPpyImVxLHBrDTKNedJG5uNqLd5Sdl68oy8ppc9dt6ph5qJ2mI47TrNphyi6vDOLqgkj/9+Fl53hz4ATgSND+1XAXcAXQ/u1ZrYNeFrSE8Bhkp4Bdjez+wAkfQfowZVFTaSNzGulf8ASR/VFmYvaYTruOK2g1abRwpQFgKQOYAXwFuD/mtn9kt5kZhsAzGyDpDeGw7uJZg4l1oW2/rBd3p70fmcSzUCYMWNGI2+l6RTlxC1iBJ50zTSl9Pklq7hg2Rpe3tpf030ddcDUxJXy0ooYOo7TGApVFsGENEfSJOD7kg6qcHiSIcQqtCe93xXAFRCV+6hO2vahSCduNbkNedmja3jSXZpS2mEM+kby3ldccaZx9fLnuPPRjbmUT6uTmxxnJJI3z6IuzGwzkbnpWOB5SXsBhP+l5L51QHwh5WnA+tA+LaF91FJkTHVaDkE9KEGd5zULZd1Xef5HpRFAnnLfRZQI9/UpnLFAYcpC0tQwoyAk8P058CiwDDgjHHYG8IOwvQw4RdIESbOA/YAHgsnqFUlHSBJweuycUUmRTty0pL1JCbODvGyOZZOXOs6+zVtzL3hSaaaTVZ+qnLjySerEG62IfX0KZ6xQpBlqL+Cq4LcYBywxsx9Kug9YIunjwHPAhwDMbI2kJcCvgO3Ap4MZC+BTvB46eyuj3LldtBM3zVG24PrV9O+o3npXkqvcfJb3SkkzkxK1KMj1m7emmvIaUQwwjofyOmOFIqOhHgTmJrS/CLwz5ZyLgIsS2nuJSo2MCVoRU90zt5veZ1/imvvXMmAW1qgwspK4RdQRz198B69u257YGUdXSqdSxkwtPpa9J3WlduIdUmJ13FoVsYfyOmOFpvgsnOqotb5TPSxd2ceNK/oGO9IBy6co4qXEKxUdrGY1vDhJPpbOcUqtZAtRZFRaZz1g1tCV9Np9GVfHaRS5Fj8aiYzGxY+KJG1RnjS6qxjxlxb2mXPhjxMVSlfnOKbsMiE1OikteintepO6OtllwviGLQJViWYsxuQ4zSRt8SNXFg6QvQpfnMkTO9m8pT/38RB10kcdMJXrHlg7xC8yDujo0JCihZ3jxK47j2fzlsq5GDMX/ij1/S47eU7uTrzeUFoPxXVGE/WulOeMcvL6Bjo7xPknzOaS2x6raibSt3krN67o47BZk1n+1KZBv8jOneN49bWyarc7bHC9jlpzTEo+i5KPojulE29ETkurM2sdpxm4z8IBkn0DJUrBSt2Turjkg4fQM7e74vFpbO0f4BdPvjTEL1KuKNLOSwptreS3KCmyko8ibbTfDusEOM5IwJWFA7zuVO9IiGM1Xu+YP3fdKuYvvgNgmBM+T65GrUbPJIf1+SfMprMjO5ujUufv0UyOkw83QzmD9Mzt5nPXrUrct2lL/xDT0NnXrWLyxE7OP2F26loTjWRSwiwiqRJnmmksrfP3woSOkw9XFmOUNKdsNXkNm7b0s+CG1cBQu321/oxS9FKpQ0+afcTjMM5b+tCQfJBTD5/Oop6DgfSorrTOvx3WCXCckYCbocYglUpUVOuL6B8wLrx5Te7jy41GXZ0dXPC+2dy78GieXnx86nmldbzPW/oQVy9/bojf4+rlz3He0sgpveCY/eksW5wjvkBTOa3IaXGckYjPLMYglZy69y48evCYamYYkG2G6hwnTj5sOnc+ujE1zDTLLHTN/WsTr33N/WsHZxfDNFKGW8OjmRwnG1cWY4iS6SnLrl/qPCvlMSSRVfRv153Hv96hp5BlFkoq1RFvv+S2x4bkbED6Ak2O4+THzVBjhLjpKY1x0mC11FqqpmZFEG3aklwOJE6WWSht+ddSe9r9NXoND8cZa/jMYhSQJ4P4gmVrMqOUBsw4+7pVXHjzmorF/ZKYv/gO9ujqTK0PBdARlFGWrJXMQhPGj2Nr/47E9tJ7JM0+kkKCHcfJjyuLEU6eDOSlK/sqduLl5JkBlNO3eSudHaJznFLLnA+YDSmD3rd5Kwuufz2aKg+/T1AU8fYsM5XjOLXhZqgRTlYG8tKVfZyzZHVTZOkfMHbdeXzqKF5imCLp32FcsCx/NFVWlde06ra1Vr11HCfClcUIp1IGcmnW0cxR9aYt/Vz64UMSy4CniVHNrOeoA6ZWbM/a7zhObbiyGKGUlgxNUwNpCwDF2WWnjrrX3y5HDC3iB687qRvBnY9urNietd9xnNpwn8UIJE9ZjfUh4S6Nrs4OLnp/1IFfsGxNVaP7SpSS/CDyE5RW0rvktsfYZaeOxMKBlQoClpNVy8lrPTlOMfjMYgSSNWOAygX7OqTBcNSeud2sOv/djRUwQY6+zVt5bfsOOsqzq0PJ87xk+Sx85TrHKYbClIWk6ZLulPSIpDWSzgrtF0jqk7Qq/B0XO+dcSU9IekzSMbH2QyU9FPZ9QxrbcZD1jJK7Oju49MOHDFuJLo0J48c1LOy0f4ex24TxQ3IoSiXP85JUjiSetJe133Gc2ijSDLUdOMfMfilpN2CFpJ+EfV83s3+KHyzpQOAUYDawN/BTSW81swHgcuBMYDlwC3AscGuBsrc11RT7g9fXyk5bAKjS2g07dlhDHeSbt/bXNZNJqjQbv6es/Y7j1EZhysLMNgAbwvYrkh4BKv1iTwSuNbNtwNOSngAOk/QMsLuZ3Qcg6TtAD2NYWSSVxKhESVGU6j6VU0nxpOVMlJN3Te5qJylpSXyVOn+v9eQ4jacpDm5JM4G5wP3AfOAzkk4HeolmH5uIFMny2GnrQlt/2C5vT3qfM4lmIMyYMaOxN9FGJI2eZ76hi3uffCn1nFIobVLHm5b1XAnBsFF7WnnwONW8TSOWPHUcpzEUriwk7QrcCJxtZr+VdDnwFaIB71eAS4G/Irk2qFVoH95odgVwBcC8efNGdcpu0ui5VL47iT26OlM73moVRYfEDjNe3badC29ew+euW8Xek7o46oCp3Liir2GLH1VKOHRl4TjNpVBlIamTSFF8z8xuAjCz52P7vwn8MLxcB0yPnT4NWB/apyW0j2mSZgmLeg5m3j5TEqu2SqR2vHlNSCVKyiUebtu3eSs3rujjpEO7ufPRjanXm9g5jvmL78jlT/AwWMdpH4qMhhLwbeARM/tarH2v2GHvBx4O28uAUyRNkDQL2A94IPg+XpF0RLjm6cAPipJ7JLB0ZR8Lrl89ZPGiBdevZunKvtSqrZtT6j2t37y16gWP0tjaP8A1969lwTH7c9nJcxIrxPbvsMRFl5LwMFjHaR9kBZWCkPQnwM+Ah4BS9bcvAacCc4hMSc8AnwwKAUl/R2SS2k5ktro1tM8DrgS6iBzbf2sZgs+bN896e3sbek9Fkqcaa4k5F/44MYluUldnaqRRmj+h5Pj+y2/eV9HnUSRpzvek5MOuzg5fyc5xCkTSCjObV95eZDTUz0n2N9xS4ZyLgIsS2nuBgxonXXtRrSM3Ldu6UhZ2pUWFzlv6UMsUBaSblTwM1nHaBy/3kUI1I/16aaQjN80fUKnjbVZV2jQqmZU8DNZx2gNXFgk0O2SzWkfu5ImdqWtOlExNSTKndbzNrEpbShAs4dnVjjMy8NpQCWStEdFoqnXknn/CbDo7srPbtvYPcOHNa5i/+A5mLfwR8xffkehMrqacR2eHmNTVOeg8P+2IGYm2xjT+x75TUpdMdRynffGZRQLNDtms5E9IomduN73PvsQ1969lwKxiUt2mLf2Ds5C0GdKph09Pzc/o6hzHzp0dbN7SP8y0VTLVVTMvWbP+lUILFzqOUwyuLBJIq71UVMhmtY7cpSv7uHFF36CCKJUCz9NpJ/lC5u0zJVVZ7NzZwcSdxg+G3l7f+xznLFmdqJzyyNCoUuiO4zQXVxYJVDPSb5QjPI8jt/ReSYqsmtF9Xyj9EVdSaZTPTCol75XS7Ud16rzjjFFcWSSQd6TfTEd4ngWPqiEuZyPNa1mKopqFjqqhmdFrjjMWKSwpr9U0IykvK9Eti2o6uDxF+mph8sRONm/tr6rAX610jBOXfqi69Svy4Ml7jtM40pLyPBqqDupxhJc6uLylL4pyrm/a0hxFMXliZyGKApofveY4YxFXFnVQT+2iajq4pSv7knPhA+2wbGB3xj2v/PK7Cxvle8FBxyke91nUQT2O8DSTUnkHV5qBVBr9N8uQ2CHx5qkTeWrjlsGQ3VMPn86inoOByma5Iml29JrjjEVcWdRBPY7wtKihUge3dGUfF968JjVTuxUMmLFu0++HreFdotp8kUbRqvd1nLGEO7hroNrIm7zO6c5x4pIPHQLAghtW0z/Q3p9N+cwCWheV5NFQjtMY0hzcriyqpJbIm1kLfzRicg86O8RhMyez/KlNuWtGnXbEjCEKw3GckYtHQ1XJ0pV9iTWVaom8Kdp2Xu/CRfFaTZd88BC+94m38+TFx+U+/5r719b1/o7jtD+uLBKoFNZaS+RNo1aiS+PiD7R2VN/MqrWO47QGVxYJVJo91BIuW77U6aSuzsQlR2slyzbf2SG6OtM/6ry5HmlUU7XWcZyRiSuLBCrNHhYcs/+w8uCdHcqMvOmZ2829C4/m6yfPYdv2Hexo8GC8YhkNg5MOnUZnDg1VSzLbqYdPr+p4x3FGHq4sEsicPZR39FV0/EmzlkZQaY2L/h3GnY9u5JIPHTLEP5FGSVlm5Ud0SO7cdpwxQmHKQtJ0SXdKekTSGklnhfYpkn4i6fHwf3LsnHMlPSHpMUnHxNoPlfRQ2PcNqVi7R5KPoRS3f8ltj9FfNi3o32G5R+ONru9UMgH1zO3mkg8eknrc+s1bB2c3Ty8+nnsXHp2qDEpKMe05XHbyHJ5ZfDxPXnycKwrHGSMUObPYDpxjZn8IHAF8WtKBwELgdjPbD7g9vCbsOwWYDRwL/IukUk91OXAmsF/4O7ZAuemZ281Jh3YPdsQdEicd2l2xQmveelCN1nJHvHnyYOTW565bleo/SJotVVKKMNzX4ivbOc7YpbAMbjPbAGwI269IegToBk4EjgyHXQXcBXwxtF9rZtuApyU9ARwm6RlgdzO7D0DSd4Ae4NaiZE9aXOjGFX3M22cKe3R1Ji7gs0dX5dLbS1f2cc6S1Q3Pt7jvqZd44JlNgwl8SZFJadnMeTLQ86yz4TjO6Kcp5T4kzQTmAvcDbwqKBDPbIOmN4bBuYHnstHWhrT9sl7cnvc+ZRDMQZsyYUbWclRYXKjl+0wxglQxjpVDcIkJMdxjsSMj07pDYYZaZzezKwHGcPBTu4Ja0K3AjcLaZ/bbSoQltVqF9eKPZFWY2z8zmTZ06tSo547kVaazfvHVwedFy0tqhOKd2JXaY8fTi4wf9LOXJhY7jONVQqLKQ1EmkKL5nZjeF5ucl7RX27wW8ENrXAfEYzGnA+tA+LaG9oeTp0Pee1FVTnkUlf0ZRyXp7T+pKTC783HWrmOmKw3GcKikyGkrAt4FHzOxrsV3LgDPC9hnAD2Ltp0iaIGkWkSP7gWCyekXSEeGap8fOaRhZDuqS3T/LKZxEmiLpkLj4Awc3PKktHrlVrgBLU7JaE/AcxxmbFDmzmA98BDha0qrwdxywGHiXpMeBd4XXmNkaYAnwK+C/gE+bWamn+xTwLeAJ4EkKcG5XmhnEo4BqiRBKUzClUt9HvHlyypn5SZInSwH6anKO4+TFq84G0qrJnnRoN3c+urHu0tdpJbST3rda0tb8zlMaXcDTi4+v+b0dxxldpFWd9cWPAj1zu+l99iWuuX/t4Cpwb5uxBzeu6BuyaNG5Nz00eHy11y+dU1Icn7tuFeOkxCipCePHsX3ABmV589SJPP7Cq8OO6xiXXmokaVGgcnw1Ocdx8uDKIpCUW/GLJ18aFnZVMt3kURbx2cSkiZ2Yweat/UNWyUsLp922fcfgdmmFuvn7TuHeJ18abBdw6mHTK4bFAoPhwOWr8/lqco7j5MWVRaCSM7icvNna8VF9fHnUWgx/W/sH+NWGV+jq7Bi8psFgsmCePApfTc5xnFpxZRHIowBK7D2pi/OWPjTEZFW+vGgRuRVJ63FXM9PxBDzHcWrFq84G0mz35UGtXZ0dzHxDF1cvf26Iyerq5c9x3tKHBo+rRvnUSzPfy3GcsYkri0BaeOtfHjFjWFjq8qc2JV4jvrxoNY7jjhzrTHR1djAppf6UO6kdxykaN0MF8hTVK3H2dasSrxF3Vh91wFSuXv5c5vt2T+ri1W3bE4sTligl7wGJ4b3upHYcp2hcWcTIa9PvSAl3jWdi3/noxszrTJ7Yyb0Lj2bWwh+lHtPV2TEs6c+d1I7jNBtXFjVw6uHTE2cN8eVF8/gRSsUH957UlZg8V5pReMlwx3FajfssamBRz8GcdsSMIYsjlS8vmsePkLUiXakciOM4TqvxmUWNLOo5uOKSolnZ0+Ur0oGblxzHaV9cWdRAnuS2cgWwR1cnUmR68hXpHMcZabiyqJLyzOxK9aJcATiOM1pwZVElSZnZ1WRR14KX6XAcp9W4sqiStCinorKoq5nJOI7jFIVHQ1VJLcuq1kOlmYzjOE6zcGVRJbUsq1oPzZ7JOI7jJOHKokpqWVa1Hpo9k3Ecx0nCfRY10Mwop6R8Da8H5ThOsylsZiHp3yW9IOnhWNsFkvokrQp/x8X2nSvpCUmPSTom1n6opIfCvm9Iyi7ROopo9kzGcRwniSJnFlcC/wf4Tln7183sn+INkg4ETgFmA3sDP5X0VjMbAC4HzgSWA7cAxwK3Fih32+H5Go7jtJrCZhZmdg/wUuaBEScC15rZNjN7GngCOEzSXsDuZnafmRmR4ukpRGDHcRwnlVY4uD8j6cFgppoc2rqBtbFj1oW27rBd3p6IpDMl9Urq3bgxu0S44ziOk49mK4vLgX2BOcAG4NLQnuSHsArtiZjZFWY2z8zmTZ06tU5RHcdxnBJNVRZm9ryZDZjZDuCbwGFh1zpgeuzQacD60D4tod1xHMdpIk1VFsEHUeL9QClSahlwiqQJkmYB+wEPmNkG4BVJR4QoqNOBHzRTZsdxHKfAaChJ1wBHAntKWgecDxwpaQ6RKekZ4JMAZrZG0hLgV8B24NMhEgrgU0SRVV1EUVC5IqFWrFjxG0nPNuh24uwJ/KaA6zYSl7FxjAQ5XcbGMRLkLFrGfZIaZQlrSTvpSOo1s3mtlqMSLmPjGAlyuoyNYyTI2SoZvdyH4ziOk4krC8dxHCcTVxbVc0WrBciBy9g4RoKcLmPjGAlytkRG91k4juM4mfjMwnEcx8nElYXjOI6TiSuLCkh6JpRHXyWpN7RNkfQTSY+H/5OzrlOgfPvHyr2vkvRbSWdXKgXfRNmSStSnPru0EvUtkPESSY+G+mXflzQptM+UtDX2TP+1GTJWkLPqcv8tkPG6mHzPSFoV2lvyLCVNl3SnpEckrZF0Vmhvm+9lBRlb/700M/9L+SNKHNyzrO2rwMKwvRD4x1bLGWTpAH5NlFBzAfA/WyzPnwJvAx7OenbAgcBqYAIwC3gS6GiRjO8Gxoftf4zJODN+XBs8y8TPuJ2eZdn+S4Evt/JZAnsBbwvbuwH/HZ5X23wvK8jY8u+lzyyq50TgqrB9Fe1TMv2dwJNmVkTWetVYcon6tGeXWKK+FTKa2Y/NbHt4uZyhtclaQsqzTKNtnmWJUKrnw8A1RctRCTPbYGa/DNuvAI8QVbFum+9lmozt8L10ZVEZA34saYWkM0PbmyyqWUX4/8aWSTeUUxj6Y0wqBd9q0p5dWon6VvNXDC0vM0vSSkl3S3pHq4SKUU25/1byDuB5M3s81tbSZylpJjAXuJ82/V6WyRinJd9LVxaVmW9mbwPeA3xa0p+2WqAkJO0EvA+4PjSllYJvV6oqRd8MJP0dUZ2y74WmDcAMM5sLfB74T0m7t0o+qi/330pOZehApqXPUtKuwI3A2Wb220qHJrQ15VmmydjK76UriwqY2frw/wXg+0RT0OcVqueG/y+0TsJB3gP80syeh4ql4FtN2rNLK1HfEiSdAbwX+EsLhuFgingxbK8gsl+/tVUyVviM2+1Zjgc+AFxXamvls5TUSdQJf8/MbgrNbfW9TJGx5d9LVxYpSNpF0m6lbSIH08NE5dTPCIedQXuUTB8yclN6KfhWk/bsEkvUt0A+JB0LfBF4n5ltibVPldQRtt8cZHyqFTIGGaoq999s+WL8OfComQ2ueNmqZxl8J98GHjGzr8V2tc33Mk3GtvheNsOLPhL/gDcTRUKsBtYAfxfa3wDcDjwe/k9psZwTgReBPWJt3wUeAh4k+sLv1QK5riGaIvcTjdA+XunZAX9HNCp6DHhPC2V8gshOvSr8/Ws49qTwPVgN/BI4ocXPMvUzbpdnGdqvBP6m7NiWPEvgT4jMSA/GPt/j2ul7WUHGln8vvdyH4ziOk4mboRzHcZxMXFk4juM4mbiycBzHcTJxZeE4juNk4srCcRzHycSVhTPmCZU72yUXxXHaElcWjlMHIUO57RkpcjrtiysLx4nokPTNsIbAjyV1SZojaXlsDYHJAJLukvQPku4GzpL0IUkPS1ot6Z5wTEdYg+D/hfM/GdqPlHRPuN6vJP2rpHFh36mK1k95WNI/hrYPS/pa2D5L0lNhe19JPw/bh4Yicisk3RYrXTFEzuY+Tme04aMNx4nYDzjVzD4haQlRZuwXgL81s7sl/T1wPnB2OH6Smf0ZgKSHgGPMrE9hURqiLOuXzeyPJU0A7pX047DvMKI1Cp4F/gv4gKRfEK1TcCiwiajacQ9wD7AgnPcO4EVJ3USZvj8LdYT+N3CimW2UdDJwEVFl0iFyOk49uLJwnIinzWxV2F5BVNF1kpndHdqu4vWqvhArjAfcC1wZlEyp8Nu7gT+S9MHweg8ihfQa8ICZlWYI1xB1/P3AXWa2MbR/D/hTM1sqaddQp2w68J9ECw29I7zX/sBBwE+iskJ0EJXdSJLTcWrGlYXjRGyLbQ8AkzKOf7W0YWZ/I+lw4HhglaQ5ROWt/9bMboufJOlIhpe5NpLLYZe4D/gYUX2inxHNGt4OnAPMANaY2duz5HScenCfheMk8zKwKbaYzEeAu5MOlLSvmd1vZl8GfkM0A7gN+FQwEyHpraF6McBhkmYFX8XJwM+JFrj5M0l7hiqip8be7x7gf4b/K4GjgG1m9jKRApkq6e3hfTolzW7cY3CcCJ9ZOE46ZwD/KmkiUdnnj6Ucd4mk/YhmB7cTVQB9kGh95F+GstMbeX25zvuAxcDBRArg+2a2Q9K5wJ3hOreYWalU9s+IFNA9ZjYgaS3wKICZvRZMXd+QtAfRb/oyokqkjtMwvOqs4zSRYIb6n2b23haL4jhV4WYox3EcJxOfWTiO4ziZ+MzCcRzHycSVheM4jpOJKwvHcRwnE1cWjuM4TiauLBzHcZxM/n97QEJLdBEZDQAAAABJRU5ErkJggg==\n",
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "plt.figure()\nplt.scatter(cars2[\"hp\"], cars2[\"weight\"])\nplt.xlabel(\"horsepower\")\nplt.ylabel(\"weight\")\nplt.title(\"Correlation between Horsepower and Weight\")\nplt.show()"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 2: Normalization\n\nIn order for better model training and convergence, it is a good practice to normalize feature vectors.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "normalizer = Normalizer(inputCol=\"features\", outputCol=\"features_normalized\", p=1.0)\ntrain_norm = normalizer.transform(train)\nprint(\"Normalized using L^1 norm\")\ntrain_norm.show(5, truncate=False)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 2: Standard Scaling\n\nThis is a standard practice to scale the features such that all columns in the features have zero mean and unit variance.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "standard_scaler = StandardScaler(inputCol=\"features\", outputCol=\"features_scaled\")\ntrain_model = standard_scaler.fit(train)\ntrain_scaled = train_model.transform(train)\ntrain_scaled.show(5, truncate=False)"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "test_scaled = train_model.transform(test)\ntest_scaled.show(5, truncate=False)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Exercise 4 - Building and Training a Linear Regression Model\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "In this exercise, we train a Linear Regression model `lrModel` on our training dataset. We train the model on the standard scaled version of features.\nWe also print the final RMSE and R-Squared metrics.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 1: Create and Train model\n\nWe can create the model using the `LinearRegression()` class and train using the `fit()` function.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Create a LR model\nlr = LinearRegression(featuresCol='features_scaled', labelCol='mpg', maxIter=100)\n\n# Fit the model\nlrModel = lr.fit(train_scaled)\n\n# Print the coefficients and intercept for linear regression\nprint(\"Coefficients: %s\" % str(lrModel.coefficients))\nprint(\"Intercept: %s\" % str(lrModel.intercept))\n\n# Summarize the model over the training set and print out some metrics\ntrainingSummary = lrModel.summary\n#trainingSummary.residuals.show()\nprint(\"RMSE: %f\" % trainingSummary.rootMeanSquaredError)\nprint(\"R-squared: %f\" % trainingSummary.r2)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "We see a RMSE (Root mean squared error) of 4.26. This means that our model predicts the `mpg` with an average error of 4.2 units.\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Task 2: Predict on new data\n\nOnce a model is trained, we can then `transform()` new unseen data (for eg. the test data) to generate predictions.\nIn the below cell, notice the \"prediction\" column that contains the predicted \"mpg\".\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "lrModel.transform(test_scaled).show(5)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Question 1 - Correlation\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Print the correlation matrix for the test dataset split we created above.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Code block for learners to answer"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Double-click **here** for the solution.\n\n<!-- The answer is below:\n\nr1 = Correlation.corr(test, \"features\").head()\nprint(\"Pearson correlation matrix:\\n\" + str(r1[0]))\n\n-->\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Question 2 - Feature Normalization\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Normalize the training features by using the L2 norm of the feature vector.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Code block for learners to answer"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Double-click **here** for the solution.\n\n<!-- The answer is below:\n\nnormalizer_l2 = Normalizer(inputCol=\"features\", outputCol=\"features_normalized\", p=2.0)\ntrain_norm_l2 = normalizer_l2.transform(train)\nrint(\"Normalized using L^1 norm\\n\"+str(train_norm_l2))\ntrain_norm_l2.show(5, truncate=False)\n\n-->\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Question 3 - Train Model\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Repeat the model training shown above for another 100 iterations and report the coefficients.\n"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# Code block for Question 3"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Authors\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "[Karthik Muthuraman](https://www.linkedin.com/in/karthik-muthuraman/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMBD0231ENSkillsNetwork26766988-2022-01-01)\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Other Contributors\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "[Jerome Nilmeier](https://github.com/nilmeier/)\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Change Log\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "| Date (YYYY-MM-DD) | Version | Changed By    | Change Description      |\n| ----------------- | ------- | ------------- | ----------------------- |\n| 2022-07-14        | 0.4     | Lakshmi Holla | Added code for pyspark  |\n| 2021-12-22        | 0.3     | Lakshmi Holla | Made changes in scaling |\n| 2021-08-05        | 0.2     | Azim          | Beta launch             |\n| 2021-07-01        | 0.1     | Karthik       | Initial Draft           |\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Copyright \u00a9 2021 IBM Corporation. All rights reserved.\n"
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.9 with Spark",
            "language": "python3",
            "name": "python39"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.9.13"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 4
}