correlation vs regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "570aface-9b0f-4b0b-a44a-7e6de23dfab2",
   "metadata": {},
   "source": [
    "# What is the relation between the slope of linear regression and the correlation coefficient"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3b4d80e-c850-429c-ac30-9dc3a0b0cf4b",
   "metadata": {},
   "source": [
    "Suppose we have $n$ data points $\\{x_i, y_i\\}_{i=1}^n$, the correlation coeeficient is given by\n",
    "\n",
    "$$\n",
    "r = \\frac{\\sum(x_i-\\bar{x})(y_i-\\bar{y})}{\\sqrt{\\sum(x_i-\\bar{x})^2\\sum(y_i-\\bar{y})^2}}\n",
    "$$\n",
    "\n",
    "and the least square solution to the regression equation $y = \\beta x + \\alpha + \\epsilon$ is given by\n",
    "$$\n",
    "\\beta = \\frac{\\sum(x_i-\\bar{x})(y_i-\\bar{y})}{\\sum(x_i-\\bar{x})^2}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0729956-4566-40dc-9669-b8a8ae9eb99b",
   "metadata": {},
   "source": [
    "doing some algebraic manipulation\n",
    "\n",
    "\\begin{align}\n",
    "  \\beta &= \\frac{\\sum(x_i-\\bar{x})(y_i-\\bar{y})}{\\sum(x_i-\\bar{x})^2} \\\\\n",
    "        &= \\frac{\\sum(x_i-\\bar{x})(y_i-\\bar{y})}{\\sqrt{\\sum(x_i-\\bar{x})^2\\sum(y_i-\\bar{y})^2}} \\frac{\\sqrt{\\sum(x_i-\\bar{x})^2\\sum(y_i-\\bar{y})^2}}{\\sum(x_i-\\bar{x})^2} \\\\\n",
    "        &= r \\cdot \\frac{\\sqrt{\\sum(y_i-\\bar{y})^2}}{\\sqrt{\\sum(x_i-\\bar{x})^2}} \\\\\n",
    "        &= r \\cdot \\frac{\\sqrt{\\sum(y_i-\\bar{y})^2} /n}{\\sqrt{\\sum(x_i-\\bar{x})^2}/n} \\\\\n",
    "        &= r \\cdot \\frac{std(y)}{std(x)}\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f45bf28a-b3ad-455c-9208-ced12a5eac57",
   "metadata": {},
   "source": [
    "in conclusion, $\\beta = r$ if and only if the standard deviation of $x$ and $y$ are the same, that is, if we do the normalization before the calculation."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}