Genomic Data Manipulation

.gitignore

.ipynb_checkpoints

M09: Performance evaluation.ipynb

Manipulating data elements.ipynb

W02: Biological sequences.ipynb

W03: Modules and IO.ipynb

W08: Multiple sample hypothesis tests.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multiple sample hypothesis tests\n",
    "\n",
    "---\n",
    "\n",
    "## Samples\n",
    "\n",
    "Recall that a **sample** is, for our purposes, an unordered list of one or more values, each represented one or more times.  Technically speaking, it's a series of **events**, each a single **set** drawn from an overall **sample space** that is the set of all possible outcomes of some probabilistic **experiment**.  The **probability** of an event is the fraction of times it will occur in a sample that grows arbitrarily large (at least by one definition), and the true probabilities of all events (e.g. as they would occur in an infinitely large sample) represent the larger underlying **population**.\n",
    "\n",
    "That's a lot of definitions very quickly!  Hopefully they're mostly a review, but a computational environment makes it very easily to visualize what this means.  We can define the sample space of all possible outcomes of a six-sided die roll:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "space_die = [1, 2, 3, 4, 5, 6]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's them possible to sample from this space once:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "random.choice( space_die )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or twice:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[3, 4]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def sample_with_replacement( space, number ):\n",
    "    \n",
    "    return [random.choice( space ) for i in range( number )]\n",
    "\n",
    "sample_with_replacement( space_die, 2 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or ten times:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2, 5, 6, 3, 3, 2, 4, 5, 6, 2]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_with_replacement( space_die, 10 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A histogram lets us see the (stochastic) frequency with which each event occurs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: pylab import has clobbered these variables: ['random']\n",
      "`%matplotlib` prevents importing * from pylab and numpy\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAEACAYAAAB8nvebAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADQRJREFUeJzt3V2MXHUZx/Hfr91SKBWiYAqhAnpBCN4UElpMMRwllAoR\nbkwUMU24IIZoIL4QCTHpemP0imj0hoAIWJTYiJCAkRqYGDBABcpriyBWXtI2NPISrJhiHy/mtCnT\nnTlndmbOmad8P8mms53/zjz8d/bbs2dmF0eEAAC5LGh7AADA8Ig3ACREvAEgIeINAAkRbwBIiHgD\nQEIzdRbZ3i7pbUn7JO2NiJWTHAoAMFiteKsb7SIi3pzkMACAeuqeNvEQawEAE1Y3yCFpk+3Ntq+c\n5EAAgGp1T5usjogdtj+ubsS3RsRDkxwMANBfrXhHxI7yzzds3yVppaQPxNs2vyQFAIYUEZ7Px1We\nNrG9xPbS8vLRktZIerbPEFP1tn79+tZnYKbDZ6aI/ccnMWVv64dcP/mv1Wn8/E3jTKOoc+S9TNJd\n5ZH1jKQNEXH/SPcKABhJZbwj4h+SVjQwCwCgpsP65X9FUbQ9wiGYqZ5pnGl6FW0PcIhp/PxN40yj\n8KjnXQ7ckB3jui1gWtnW/vPGeXnk860YD9uKST1hCQCYPsQbABIi3gCQEPEGgISINwAkRLwBICHi\nDQAJEW8ASIh4A0BCxBsAEiLeAJAQ8QaAhIg3ACREvAEgIeINAAkRbwBIiHgDQELEGwASIt4AkBDx\nBoCEiDcAJES8ASAh4g0ACRFvAEiIeANAQsQbABIi3gCQEPEGgISINwAkRLwBICHiDQAJEW8ASIh4\nA0BCxBsAEqodb9sLbD9h+55JDgQAqDbMkfc1kp6f1CAAgPpqxdv2ckkXSbppsuMAAOqoe+R9g6Rr\nJcUEZwEA1DRTtcD2xZJ2RcQW24Uk91s7Ozt74HJRFCqKYvQJAeAw0el01Ol0xnJbjhh8MG37h5K+\nJul9SUdJ+oik30XEup51UXVbQHa2lf8bUIuv1elgWxHR94B44McO80m0fZ6k70TEJXNcR7xx2CPe\nGKdR4s3rvAEgoaGOvAfeEEfe+BDgyBvjxJE3AHzIEG8ASIh4A0BCxBsAEiLeAJAQ8QaAhIg3ACRE\nvAEgIeINAAkRbwBIiHgDQELEGwASIt4AkBDxBoCEiDcAJES8ASAh4g0ACRFvAEiIeANAQsQbABIi\n3gCQEPEGgISINwAkRLwBICHiDQAJEW8ASIh4A0BCxBsAEiLeAJAQ8QaAhIg3ACREvAEgIeINAAkR\nbwBIiHgDQEIzVQtsL5b0Z0lHlOs3RsQPJj0YAKA/R0T1IntJROyxvVDSw5KujojHetZEndsCMrMt\nKfvj3OJrdTrYVkR4Ph9b67RJROwpLy5W9+ibzzwAtKhWvG0vsP2kpJ2SNkXE5smOBQAYpO6R976I\nOFPSckmrbJ8x2bEAAINUPmF5sIh4x/aDktZKer73+tnZ2QOXi6JQURQjjgcAh49Op6NOpzOW26p8\nwtL28ZL2RsTbto+S9EdJP4qI+3rW8YQlDns8YYlxGuUJyzpH3idKutX2AnVPs9zZG24AQLNqvVSw\n1g1x5I0PAY68MU4Tf6kgAGC6EG8ASIh4A0BCxBsAEiLeAJAQ8QaAhIg3ACREvAEgIeINAAkRbwBI\niHgDQELEGwASIt4AkBDxBoCEiDcAJES8ASAh4g0ACRFvAEiIeANAQsQbABIi3gCQEPEGgISINwAk\nRLwBICHiDQAJEW8ASIh4A0BCxBsAEiLeAJAQ8QaAhIg3ACREvAEgIeINAAkRbwBIiHgDQELEGwAS\nqoy37eW2H7D9nO1nbF/dxGAAgP4cEYMX2CdIOiEittheKulxSZdGxLaedVF1W0B2tiVlf5xbfK1O\nB9uKCM/nYyuPvCNiZ0RsKS+/K2mrpJPmc2cAgPEY6py37VMlrZD06CSGAQDUM1N3YXnKZKOka8oj\n8Km2bt1V2rDhprbHGNnZZ5+nRx75U9tjAJgyteJte0bdcN8eEXf3Wzc7O3vgclEUKopixPHmb9u2\nv2vfvrslXdDaDKPbrpdeyjw/ptPi8tx9bsuWnaKdO7e3PcZQOp2OOp3OWG6r8glLSbJ9m6TdEfHt\nAWum6gnLlSvXaPPm70pa0/YoI9iu444rtHv39rYHQelwecIy/3+DdDg88TrRJyxtr5Z0uaTP237S\n9hO2187nzgAA41F52iQiHpa0sIFZAAA18ROWAJAQ8QaAhIg3ACREvAEgIeINAAkRbwBIiHgDQELE\nGwASIt4AkBDxBoCEiDcAJES8ASAh4g0ACRFvAEiIeANAQsQbABIi3gCQEPEGgISINwAkRLwBICHi\nDQAJEW8ASIh4A0BCxBsAEiLeAJAQ8QaAhIg3ACREvAEgIeINAAkRbwBIiHgDQELEGwASIt4AkBDx\nBoCEiDcAJES8ASChynjbvtn2LttPNzEQAKBanSPvWyRdOOlBAAD1VcY7Ih6S9GYDswAAauKcNwAk\nNDPOG5udnT1wuSgKFUUxzpsHgIMslu22h2jNxOINAJP1X0nR9hAjmv8/PnVPm3ikewEAjFWdlwre\nIekvkk6z/YrtKyY/FgBgkMrTJhHx1SYGAQDUx6tNACAh4g0ACRFvAEiIeANAQsQbABIi3gCQEPEG\ngISINwAkRLwBICHiDQAJEW8ASIh4A0BCxBsAEiLeAJAQ8QaAhIg3ACREvAEgIeINAAkRbwBIiHgD\nQELEGwASIt4AkBDxBoCEiDcAJES8ASAh4g0ACRFvAEiIeANAQsQbABIi3gCQEPEGgISINwAkRLwB\nICHiDQAJEW8ASKhWvG2vtb3N9t9sf2/SQwEABquMt+0Fkn4m6UJJn5Z0me3TJz3YeDzV9gBz6LQ9\nwCE6nU7bIxxiGmeaXp22B5hDp+0B5tBpe4CxqnPkvVLSixHxz4jYK+k3ki6d7Fjj8nTbA8yh0/YA\nh5jGUE7jTNOr0/YAc+i0PcAcOm0PMFZ14n2SpFcPev+18u8AAC2ZaXuASTnyyEWamXlAS5Z8se1R\nPuC9917QkUc+XmttxB4dccSiCU8EICNHxOAF9jmSZiNibfn+dZIiIn7cs27wDQEADhERns/H1Yn3\nQkkvSDpf0g5Jj0m6LCK2zucOAQCjqzxtEhH/s/1NSfere478ZsINAO2qPPIGAEyfoX7C0vbNtnfZ\nnvM1eLbPs/2W7SfKt++PZ8yBMy23/YDt52w/Y/vqPut+avtF21tsr2h7pqb3yvZi24/afrKcaX2f\ndU3uU+VMbTymyvtdUN7fPX2ub2yf6szU4j5tt/1U+Tl8rM+aRveqaqaWOnWs7d/a3lp2YdUca4bb\np4io/SbpXEkrJD3d5/rzJN0zzG2O+ibpBEkrystL1T0/f3rPmi9Iure8vErSI1MwUxt7taT8c6Gk\nRyStbHOfas7U+D6V9/stSb+a677b2KcaM7W1Ty9L+uiA69t4TFXN1MbX3i8lXVFenpF0zKj7NNSR\nd0Q8JOnNimXzeuZ0viJiZ0RsKS+/K2mrDn0d+qWSbivXPCrpWNvLWp5Jan6v9pQXF6v7AOo9Z9bo\nPtWcSWp4n2wvl3SRpJv6LGl8n2rMJDW8Twfd56CONL5XNWbav6YRto+R9NmIuEWSIuL9iHinZ9nQ\n+zSJX0z1mfKw/17bZ0zg9vuyfaq63xk82nNV7w8ava6GftBowExSw3tVftv9pKSdkjZFxOaeJY3v\nU42ZpOYfUzdIulZz/0MitfN4qppJaudrLyRtsr3Z9pVzXN/GXlXNJDW7V5+UtNv2LeVpmhttH9Wz\nZuh9Gne8H5d0ckSsUPf3ofx+zLffl+2lkjZKuqY82m1dxUyN71VE7IuIMyUtl7Sq6X9c5zlTo/tk\n+2JJu8rvnKx2jmY/oOZMbX3trY6Is9T9ruAbts9t6H4HqZqp6b2akXSWpJ+Xc+2RdN2oNzrWeEfE\nu/u/DY6IP0haZPtj47yPudieUTeSt0fE3XMseV3SJw56f3n5d63N1NZelff3jqQHJa3tuarxfaqa\nqYV9Wi3pEtsvS/q1pM/Zvq1nTdP7VDlTW4+niNhR/vmGpLvU/V1IB2v8MVU1Uwt79ZqkVyPir+X7\nG9WN+cGG3qf5xLvv0cjB52hsr1T3pYj/msd9DOsXkp6PiJ/0uf4eSevKuc6R9FZE7Gpzpqb3yvbx\nto8tLx8l6QJJ23qWNbpPdWZqep8i4vqIODkiPiXpK5IeiIh1Pcsa3ac6M7XxtWd7SfndpWwfLWmN\npGd7ljX9mKqcqYXH1C5Jr9o+rfyr8yU937Ns6H0a6neb2L5DUiHpONuvSFov6YjufHGjpC/ZvkrS\nXkn/kfTlYW5/PmyvlnS5pGfKc6ch6XpJp+yfKyLus32R7Zck/VvSFW3PpOb36kRJt7r7K34XSLqz\n3Jevq6V9qjOTWnhMzaXlfaqcSe3s0zJJd7n7qzFmJG2IiPtb3qvKmdTOXl0taYPtReq+GuaKUfeJ\nH9IBgIT436ABQELEGwASIt4AkBDxBoCEiDcAJES8ASAh4g0ACRFvAEjo/4RhP+cQ3iDnAAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x104280358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%pylab inline\n",
    "\n",
    "number_of_rolls = 10\n",
    "number_of_bins = 6\n",
    "dummy = hist( sample_with_replacement( space_die, number_of_rolls ), number_of_bins )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADw1JREFUeJzt3GGMHPV9xvHnMUcQFHFSiGQiu+CWQisqIYeVsBGt7ChN\ng6lqv0GiBAmJFy1CQaAiVYkQkv22r6rQOHJOJTSOQuIEKUAKRKCDDYIXBoyvuGA3pmnBuHBqBG4E\nRhUkT1/cuLoedzd7Zvb27sf3I41uZue3O7+bnX127r836yQCAKx+a0bdAACgGwQ6ABRBoANAEQQ6\nABRBoANAEQQ6ABTRGui2z7K93/ZB24ds71yg7h7bR21P2d7YfasAgMWMtRUk+R/bn09y0vYZkp61\n/ViS507V2N4m6eIkl9jeJGmPpM3DaxsAMNdAQy5JTjazZ2nmTWDu1Ug7JO1tavdLGre9tqsmAQDt\nBgp022tsH5T0lqQnkjw/p2SdpGOzlo83twEAlsmgZ+i/SfI5SeslbbJ92XDbAgAsVesY+mxJfmX7\nKUnXSHpl1qrjkn571vL65rb/xzZfHAMApyGJ22oG+S+Xz9geb+bPlvRFSUfmlD0s6aamZrOkE0mm\nF2hr1U3j41fohRdeUJJFp507d7bWrOZpuX+/5T9ednb0OBr5c3W6z91qfY2uxH3e/Wuh3SBn6J+V\n9B3bazTzBrAvyaO2b5nZh5lolq+1/aqk9yTdPHAHAIBODPJvi4ckXTHP7d+as3xbh30BAJaIK0U7\ntHXr1lG3MFTVfz9p66gbGJr6zx0kyUsZn/nYG7NzarxrNRkf72lyckK9Xm/UrXyi2NZqPF4kL2nc\ncyVhn69MtpUuPhQFAKwOBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGg\nA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0AR\nBDoAFEGgA0ARBDoAFNEa6LbX237S9su2D9m+fZ6aLbZP2H6xme4eTrsAgIWMDVDzoaQ7k0zZPlfS\nAduPJzkyp+7pJNu7bxEAMIjWM/QkbyWZaubflXRY0rp5St1xbwCAJVjSGLrtDZI2Sto/z+qrbE/Z\nfsT2ZR30BgBYgkGGXCRJzXDLA5LuaM7UZzsg6cIkJ21vk/SgpEu7axMA0GagQLc9ppkw/26Sh+au\nnx3wSR6z/U3bn07y9kcfbdes+a3NBAA4pd/vq9/vL/l+TtJeZO+V9Mskdy6wfm2S6Wb+Skk/TLJh\nnrpI7dtbacbHe5qcnFCv1xt1K58otrUajxfJGuR1tRKxz1cm20rS+jll6xm67asl3SjpkO2Dmnm2\n75J0kaQkmZB0ne1bJX0g6X1J13+c5gEAS9ca6EmelXRGS81uSbu7agoAsHRcKQoARRDoAFAEgQ4A\nRRDoAFAEgQ4ARRDoAFAEgQ4ARRDoAFAEgQ4ARRDoAFAEgQ4ARRDoAFAEgQ4ARRDoAFAEgQ4ARRDo\nAFAEgQ4ARRDoAFAEgQ4ARRDoAFAEgQ4ARRDoAFAEgQ4ARRDoAFAEgQ4ARRDoAFAEgQ4ARRDoAFAE\ngQ4ARbQGuu31tp+0/bLtQ7ZvX6DuHttHbU/Z3th9qwCAxYwNUPOhpDuTTNk+V9IB248nOXKqwPY2\nSRcnucT2Jkl7JG0eTssAgPm0nqEneSvJVDP/rqTDktbNKdshaW9Ts1/SuO21HfcKAFjEksbQbW+Q\ntFHS/jmr1kk6Nmv5uD4a+gCAIRpkyEWS1Ay3PCDpjuZM/TTtmjW/tZkAAKf0+331+/0l389J2ovs\nMUn/JOmxJF+fZ/0eSU8l2dcsH5G0Jcn0nLpI7dtbacbHe5qcnFCv1xt1K58otrUajxfJGuR1tRKx\nz1cm20ritrpBh1y+LemV+cK88bCkm5oNb5Z0Ym6YAwCGq3XIxfbVkm6UdMj2Qc28fd8l6SJJSTKR\n5FHb19p+VdJ7km4eZtMAgI9qDfQkz0o6Y4C62zrpCABwWrhSFACKINABoAgCHQCKINABoAgCHQCK\nINABoAgCHQCKINABoAgCHQCKINABoAgCHQCKINABoAgCHQCKINABoAgCHQCKINABoAgCHQCKINAB\noAgCHQCKINABoAgCHQCKINABoAgCHQCKINABoAgCHQCKINABoAgCHQCKINABoIjWQLd9r+1p2y8t\nsH6L7RO2X2ymu7tvEwDQZmyAmvsk/b2kvYvUPJ1kezctAQBOR+sZepJnJL3TUuZu2gEAnK6uxtCv\nsj1l+xHbl3X0mACAJRhkyKXNAUkXJjlpe5ukByVdunD5rlnzW5sJAHBKv99Xv99f8v2cpL3IvkjS\nT5JcPkDtv0vqJXl7nnWR2re30oyP9zQ5OaFerzfqVj5RbGs1Hi+SNcjraiVin69MtpWkdWh70CEX\na4FxcttrZ81fqZk3iY+EOQBguFqHXGzfr5lxkfNtvy5pp6RPSUqSCUnX2b5V0geS3pd0/fDaBQAs\npDXQk3y5Zf1uSbs76wgAcFq4UhQAiiDQAaAIAh0AiiDQAaAIAh0AiiDQAaAIAh0AiiDQAaAIAh0A\niiDQAaAIAh0AiiDQAaAIAh0AiiDQAaAIAh0AiiDQAaAIAh0AiiDQAaAIAh0AiiDQAaAIAh0AiiDQ\nAaAIAh0AiiDQAaAIAh0AiiDQAaAIAh0AiiDQAaCI1kC3fa/tadsvLVJzj+2jtqdsb+y2RQDAIAY5\nQ79P0pcWWml7m6SLk1wi6RZJezrqDQCwBK2BnuQZSe8sUrJD0t6mdr+kcdtru2kPADCoLsbQ10k6\nNmv5eHMbAGAZjS3/JnfNmt/aTBiWCy7YoOnp10bdBoAl6Pf76vf7S76fk7QX2RdJ+kmSy+dZt0fS\nU0n2NctHJG1JMj1PbaT27a004+M9TU5OqNfrjbqVJbOt1bjPZ6zW3q1BXlcr0eo9XlbvPh+EbSVx\nW92gQy5upvk8LOmmZqObJZ2YL8wBAMPVOuRi+37NjIucb/t1STslfUpSkkwkedT2tbZflfSepJuH\n2TAAYH6tgZ7kywPU3NZNOwCA08WVogBQBIEOAEUQ6ABQBIEOAEUQ6ABQBIEOAEUQ6ABQBIEOAEUQ\n6ABQBIEOAEUQ6ABQBIEOAEUQ6ABQBIEOAEUQ6ABQBIEOAEUQ6ABQBIEOAEUQ6ABQBIEOAEUQ6ABQ\nBIEOAEUQ6ABQBIEOAEUQ6ABQBIEOAEUQ6ABQBIEOAEUMFOi2r7F9xPbPbX91nvVbbJ+w/WIz3d19\nqwCAxYy1FdheI+kbkr4g6T8lPW/7oSRH5pQ+nWT7EHoEAAxgkDP0KyUdTfJakg8k/UDSjnnq3Gln\nAIAlGSTQ10k6Nmv5jea2ua6yPWX7EduXddIdAGBgrUMuAzog6cIkJ21vk/SgpEs7emwAwAAGCfTj\nki6ctby+ue3/JHl31vxjtr9p+9NJ3v7ow+2aNb+1mQAAp/T7ffX7/SXfz0kWL7DPkPSvmvlQ9E1J\nz0m6IcnhWTVrk0w381dK+mGSDfM8VqTFt7cSjY/3NDk5oV6vN+pWlsy2VuM+n7Fae7faXlcr1eo9\nXlbvPh+EbSVp/Zyy9Qw9ya9t3ybpcc2Mud+b5LDtW2ZWZ0LSdbZvlfSBpPclXf/x2gcALNVAY+hJ\nfirp9+fc9q1Z87sl7e62NQDAUnClKAAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEE\nOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAU\nQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEDBbrta2wfsf1z219doOYe20dtT9ne2G2b\nAIA2rYFue42kb0j6kqQ/lHSD7T+YU7NN0sVJLpF0i6Q9Q+h1xev3+6NuYcj6o25gyPqjbmBo6h+b\nkAY7Q79S0tEkryX5QNIPJO2YU7ND0l5JSrJf0rjttZ12ugrUf9H0R93AkPVH3cDQ1D82IQ0W6Osk\nHZu1/EZz22I1x+epAQAM0dhyb/C88/58uTf5sb3//qs688wzR90GACzKSRYvsDdL2pXkmmb5a5KS\n5G9n1eyR9FSSfc3yEUlbkkzPeazFNwYAmFcSt9UMcob+vKTfs32RpDcl/YWkG+bUPCzpK5L2NW8A\nJ+aG+aANAQBOT2ugJ/m17dskPa6ZMfd7kxy2fcvM6kwkedT2tbZflfSepJuH2zYAYK7WIRcAwOqw\nLFeK2r7X9rTtl5Zje8vN9nrbT9p+2fYh27ePuqeu2D7L9n7bB5vfbeeoexoG22tsv2j74VH30jXb\n/2H7n5vn8LlR99M12+O2f2T7cPMa3DTqnrpi+9LmeXux+fnfi+XLspyh2/4jSe9K2pvk8qFvcJnZ\nvkDSBUmmbJ8r6YCkHUmOjLi1Ttg+J8lJ22dIelbS7UlKBYPtv5bUk3Reku2j7qdLtn8hqZfknVH3\nMgy2/1HSz5LcZ3tM0jlJfjXitjrXXOT5hqRNSY7NV7MsZ+hJnpFU8mCSpCRvJZlq5t+VdFiF/g8/\nyclm9izNfO5SapzO9npJ10r6h1H3MiRW0e9tsn2epD9Ocp8kJfmwYpg3/kTSvy0U5lLRJ3mUbG+Q\ntFHS/tF20p1mOOKgpLckPZHk+VH31LG/k/Q3KvZGNUskPWH7edt/OepmOvY7kn5p+75mWGLC9tmj\nbmpIrpf0/cUKCPQONcMtD0i6ozlTLyHJb5J8TtJ6SZtsXzbqnrpi+88kTTd/YbmZqrk6yRWa+Svk\nK80QaBVjkq6QtLv5HU9K+tpoW+qe7TMlbZf0o8XqCPSONGN3D0j6bpKHRt3PMDR/yj4l6ZpR99Kh\nqyVtb8aZvy/p87b3jrinTiV5s/n5X5J+rJnvZ6riDUnHkrzQLD+gmYCvZpukA81zuKDlDPSqZz+n\nfFvSK0m+PupGumT7M7bHm/mzJX1RUokPeyUpyV1JLkzyu5q5aO7JJDeNuq+u2D6n+ctRtn9L0p9K\n+pfRdtWd5gLGY7YvbW76gqRXRtjSsNygluEWaZm+y8X2/ZK2Sjrf9uuSdp76EKMC21dLulHSoWas\nOZLuSvLT0XbWic9K+k7zCfsaSfuSPDrinjC4tZJ+3Hztxpik7yV5fMQ9de12Sd9rhiV+oWIXNto+\nRzMfiP5Vay0XFgFADYyhA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFPG/nE21ND8G\n+kYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105bfcc18>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dummy = hist( sample_with_replacement( space_die, number_of_rolls ), number_of_bins )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADx5JREFUeJzt3FGMHfV9xfFzjBsLiFgpRDKRXeyWgioqIQdLmIhW3ihN\ng6mKXyJRhITEQ4tQEKhIVSIUyc5jX1qFppFrldA4CgkJUghpoCIS3KLkwVDbW1ywG9M0wdB41Six\nIzCqSDh92HG0XXZ3Zs3cnb2/fD/SyDN3fnvnd3e9Z//3d/eukwgAMPnWDd0AAKAfBDoAFEGgA0AR\nBDoAFEGgA0ARBDoAFNEa6LY32D5o+4jto7b3LFH3gO0Ttmdsb+u/VQDActa3FST5X9sfTnLW9gWS\nvmf7ySTPnauxvUvSFUmutL1D0j5J14+vbQDAQp1GLknONrsbNPdDYOG7kXZLOtDUHpQ0ZXtjX00C\nANp1CnTb62wfkXRK0neSPL+gZJOkk/OOX2tuAwCskq4r9LeTfFDSZkk7bF893rYAACvVOkOfL8nP\nbT8j6UZJL8079Zqk35x3vLm57f+xzR+OAYDzkMRtNV1+y+X9tqea/QslfVTS8QVlj0u6vam5XtLp\nJLNLNDWRW9N9y7anQ81qb/19zvfs2TP412GcW+XHV/mx/To8vq66rNA/IOmLttdp7gfAI0mesH3n\nXFZkf3N8k+2XJb0h6Y7OHQAAetHl1xaPSrp2kdv/fsHx3T32BQBYId4p2qvpoRsYq+np6aFbGKvK\nj6/yY5PqP76uvJL5zLu+mJ3VvF6fbOudv34/CbyiGRyAtce20seLogCAyUCgA0ARBDoAFEGgA0AR\nBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoA\nFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARrYFue7Ptp22/aPuo7XsW\nqdlp+7Ttw8326fG0CwBYyvoONb+QdF+SGdvvlXTI9lNJji+oezbJzf23CADoonWFnuRUkplm/3VJ\nxyRtWqTUPfcGAFiBFc3QbW+VtE3SwUVOf8j2jO1v2766h94AACvQZeQiSWrGLY9KurdZqc93SNLl\nSc7a3iXpMUlX9dcmAKBNp0C3vV5zYf6lJN9ceH5+wCd50vbnbb8vyU8X1u7du/dX+9PT05qenj6P\ntgGgrtFopNFotOKPc5L2IvuApJ8kuW+J8xuTzDb710n6WpKti9Sly/XWItuSJrF3a1I/5wDm2FaS\n1tcpW1fotm+QdJuko7aPaC7V7pe0RVKS7Jf0cdt3SXpL0puSbnk3zQMAVq7TCr23i7FCHwArdGDS\ndV2h805RACiCQAeAIgh0ACiCQAeAIgh0ACiCQAeAIgh0ACiCQAeAIgh0ACiCQAeAIgh0ACiCQAeA\nIgh0ACiCQAeAIgh0ACiCQAeAIgh0ACiCQAeAIgh0ACiCQAeAIgh0ACiCQAeAIgh0ACiCQAeAIgh0\nACiCQAeAIgh0ACiCQAeAIgh0ACiiNdBtb7b9tO0XbR+1fc8SdQ/YPmF7xva2/lsFACxnfYeaX0i6\nL8mM7fdKOmT7qSTHzxXY3iXpiiRX2t4haZ+k68fTMgBgMa0r9CSnksw0+69LOiZp04Ky3ZIONDUH\nJU3Z3thzrwCAZXRZof+K7a2Stkk6uODUJkkn5x2/1tw2u/A+9u7du5JLrgkXX3zx0C0AQKvOgd6M\nWx6VdG+zUj8vn/nMaN7R1mZb2zZs+OuhW3gXNsj20E2cl3XrLtLbb58duo0V27hxi06d+uHQbWCC\njUYjjUajFX+ck7QX2esl/ZOkJ5N8dpHz+yQ9k+SR5vi4pJ1JZhfURWq/3lozNbVdZ84c1iT2LlmT\n2bc0ub1bXb6vgK5sK0nryqzrry1+QdJLi4V543FJtzcXvl7S6YVhDgAYr9aRi+0bJN0m6ajtI5pb\nMt0vaYukJNmf5AnbN9l+WdIbku4YZ9MAgHdqDfQk35N0QYe6u3vpCABwXninKAAUQaADQBEEOgAU\nQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaAD\nQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBEEOgAUQaADQBGt\ngW77Qduztl9Y4vxO26dtH262T/ffJgCgzfoONQ9J+ltJB5apeTbJzf20BAA4H60r9CTflfSzljL3\n0w4A4Hz1NUP/kO0Z29+2fXVP9wkAWIEuI5c2hyRdnuSs7V2SHpN01dLle+ftTzcbAOCc0Wik0Wi0\n4o9zkvYie4ukbyW5pkPtf0nanuSni5yL1H69tWZqarvOnDmsSex9bho2iX1Lk9u71eX7CujKtpK0\njra7jlysJebktjfO279Ocz8k3hHmAIDxah252H5Yc3ORS22/ImmPpPdISpL9kj5u+y5Jb0l6U9It\n42sXALCUTiOX3i7GyGUAkzq2kCa3d0Yu6FffIxcAwBpHoANAEQQ6ABRBoANAEQQ6ABRBoANAEQQ6\nABRBoANAEQQ6ABRBoANAEQQ6ABRBoANAEQQ6ABRBoANAEQQ6ABRBoANAEQQ6ABRBoANAEQQ6ABRB\noANAEQQ6ABRBoANAEQQ6ABRBoANAEQQ6ABRBoANAEQQ6ABRBoANAEa2BbvtB27O2X1im5gHbJ2zP\n2N7Wb4sAgC66rNAfkvSxpU7a3iXpiiRXSrpT0r6eegMArEBroCf5rqSfLVOyW9KBpvagpCnbG/tp\nDwDQVR8z9E2STs47fq25DQCwitav/iX3ztufbjYAa8Fll23V7OyPhm5jxTZu3KJTp344dBu9GY1G\nGo1GK/44J2kvsrdI+laSaxY5t0/SM0keaY6PS9qZZHaR2kjt11trpqa268yZw5rE3iVrMvuWJrd3\nq8v31Vpk8zlfi2wridvquo5c3GyLeVzS7c1Fr5d0erEwBwCMV+vIxfbDmpuLXGr7FUl7JL1HUpLs\nT/KE7ZtsvyzpDUl3jLNhAMDiOo1cersYI5cBTOpTaGlye5/cp/+MXNamvkcuAIA1jkAHgCIIdAAo\ngkAHgCIIdAAogkAHgCIIdAAogkAHgCIIdAAogkAHgCIIdAAogkAHgCIIdAAogkAHgCIIdAAogkAH\ngCIIdAAogkAHgCIIdAAogkAHgCIIdAAogkAHgCIIdAAogkAHgCIIdAAogkAHgCIIdAAogkAHgCI6\nBbrtG20ft/19259c5PxO26dtH262T/ffKgBgOevbCmyvk/Q5SR+R9N+Snrf9zSTHF5Q+m+TmMfQI\nAOigywr9OkknkvwoyVuSvipp9yJ17rUzAMCKdAn0TZJOzjt+tbltoQ/ZnrH9bdtX99IdAKCz1pFL\nR4ckXZ7krO1dkh6TdFVP9w0A6KBLoL8m6fJ5x5ub234lyevz9p+0/Xnb70vy03fe3d55+9PNBgA4\nZzQaaTQarfjjnGT5AvsCSf+huRdFfyzpOUm3Jjk2r2Zjktlm/zpJX0uydZH7irT89daiqantOnPm\nsCax97mXNiaxb2lye7favq/WKpvP+VpkW0laX6dsXaEn+aXtuyU9pbmZ+4NJjtm+c+509kv6uO27\nJL0l6U1Jt7y79gEAK9W6Qu/1YqzQBzCpKy5pcnuf3NUiK/S1qesKnXeKAkARBDoAFEGgA0ARBDoA\nFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGg\nA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0ARBDoAFNEp0G3f\naPu47e/b/uQSNQ/YPmF7xva2ftsEALRpDXTb6yR9TtLHJP2epFtt/+6Cml2SrkhypaQ7Je0bQ68T\nYDR0A2M2GrqBMRsN3cDYjEajoVvAKuiyQr9O0okkP0rylqSvStq9oGa3pAOSlOSgpCnbG3vtdCKM\nhm5gzEZDNzBmo6EbGBsC/ddDl0DfJOnkvONXm9uWq3ltkRoAwBitX+0LXnLJn6z2Jd+1N998eegW\nAKCVkyxfYF8vaW+SG5vjT0lKkr+aV7NP0jNJHmmOj0vamWR2wX0tfzEAwKKSuK2mywr9eUm/Y3uL\npB9L+lNJty6oeVzSJyQ90vwAOL0wzLs2BAA4P62BnuSXtu+W9JTmZu4PJjlm+86509mf5AnbN9l+\nWdIbku4Yb9sAgIVaRy4AgMmwKu8Utf2g7VnbL6zG9Vab7c22n7b9ou2jtu8Zuqe+2N5g+6DtI81j\n2zN0T+Nge53tw7YfH7qXvtn+oe1/a76Gzw3dT99sT9n+uu1jzffgjqF76ovtq5qv2+Hm3zPL5cuq\nrNBt/76k1yUdSHLN2C+4ymxfJumyJDO23yvpkKTdSY4P3FovbF+U5KztCyR9T9I9SUoFg+2/kLRd\n0iVJbh66nz7Z/oGk7Ul+NnQv42D7HyX9S5KHbK+XdFGSnw/cVu+aN3m+KmlHkpOL1azKCj3JdyWV\n/M8kSUlOJZlp9l+XdEyFfg8/ydlmd4PmXncpNaezvVnSTZL+YehexsQq+nebbF8i6Q+SPCRJSX5R\nMcwbfyjpP5cKc6noF3lItrdK2ibp4LCd9KcZRxyRdErSd5I8P3RPPfsbSX+pYj+o5omk79h+3vaf\nDd1Mz35L0k9sP9SMJfbbvnDopsbkFklfWa6AQO9RM255VNK9zUq9hCRvJ/mgpM2Sdti+euie+mL7\njyXNNs+w3GzV3JDkWs09C/lEMwKtYr2kayX9XfMYz0r61LAt9c/2b0i6WdLXl6sj0HvSzO4elfSl\nJN8cup9xaJ7KPiPpxqF76dENkm5u5sxfkfRh2wcG7qlXSX7c/Ps/kr6hub/PVMWrkk4m+dfm+FHN\nBXw1uyQdar6GS1rNQK+6+jnnC5JeSvLZoRvpk+33255q9i+U9FFJJV7slaQk9ye5PMlva+5Nc08n\nuX3ovvpi+6LmmaNsXyzpjyT9+7Bd9ad5A+NJ21c1N31E0ksDtjQut6pl3CKt0t9ysf2wpGlJl9p+\nRdKecy9iVGD7Bkm3STrazJoj6f4k/zxsZ734gKQvNq+wr5P0SJInBu4J3W2U9I3mz26sl/TlJE8N\n3FPf7pH05WYs8QMVe2Oj7Ys094Lon7fW8sYiAKiBGToAFEGgA0ARBDoAFEGgA0ARBDoAFEGgA0AR\nBDoAFEGgA0AR/wftS5hT5NmylQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10426fba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dummy = hist( sample_with_replacement( space_die, number_of_rolls ), number_of_bins )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the closer we get to an \"infinite\" sample, the more our histogram resembles what we intuitively realize the underlying true, population distribution must be:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXEAAAEACAYAAABF+UbAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADkBJREFUeJzt3W2spGddx/HvrxyoLbhVEbrS1ZaS1AeSCiWhJEU9UrEV\nQusLH3hQBBN8IVhCSEOpL3ZfmWhikETfEMoKpkXSKlIMaCFlTKry1HZpoQsSCX0QehQpUEIglP59\ncWZxs+zuzJm559z7n34/yWRn5lz33L/TM+fX61xz5jqpKiRJPZ02dgBJ0uIscUlqzBKXpMYscUlq\nzBKXpMYscUlqbGaJJ7kuyVaSu47zsTcmeTTJj60mniTpZOaZiR8ELjv2ziT7gBcC9w4dSpI0n5kl\nXlW3AQ8d50NvAa4ePJEkaW4LrYknuQK4v6ruHjiPJGkHNnZ6QJIzgGvZXkr5/t2DJZIkzW3HJQ48\nAzgP+FSSAPuA25M8t6r++9jBSdycRZIWUFUzJ8jzLqdkeqGqPl1Ve6vq/Kp6OvAA8OzjFfhRQdb2\nsn///tEz+PmN/7lNn+mn2GX/HGP6fn+u83Pz/59Ts83zK4Y3AP8GXJDkviSvPmZI4XKKJI1i5nJK\nVb18xsfPHy6OJGknfMfmkjY3N8eOsFLr/Pmt8+e2bXPsACu1/l+/+WQnay8LnSCpVZ9DGtv2a/wd\nn+fZ0fqrdk8SasAXNiVJpyBLXJIas8QlqTFLXJIas8QlqTFLXJIas8QlqTFLXJIas8QlqTFLXJIa\ns8QlqTFLXJIas8QlqTFLXJIas8QlqTFLXJIas8QlqTFLXJIas8QlqTFLXJIas8QlqbGZJZ7kuiRb\nSe466r4/S3I4yaEkf5dkz2pjSpKOZ56Z+EHgsmPuuwV4ZlU9C/g88Oahg0mSZptZ4lV1G/DQMfd9\nuKoend78KLBvBdkkSTMMsSb++8AHB3gcSdIObSxzcJI/Br5bVTecbNyBAwe+f31zc5PNzc1lTitJ\na2cymTCZTHZ8XKpq9qDkXOD9VXXhUfe9CngN8IKq+s5Jjq15ziF1lgTo+DwPfn+empJQVZk1bt6Z\neKaXIw9+OXA18IsnK3BJ0mrNnIknuQHYBJ4MbAH7gWuBJwD/Ox320ar6wxMc70xca8+ZuIY270x8\nruWUJYNY4lp7lriGNm+J+45NSWrMEpekxixxSWrMEpekxixxSWrMEpekxixxSWrMEpekxixxSWrM\nEpekxixxSWrMEpekxixxSWrMEpekxixxSWrMEpekxixxSWrMEpekxixxSWrMEpekxixxSWrMEpek\nxixxSWpsZoknuS7JVpK7jrrvR5PckuRzSf45yVmrjSlJOp55ZuIHgcuOue8a4MNV9dPArcCbhw4m\nSZptZolX1W3AQ8fcfSXwzun1dwK/PnAuSdIcFl0Tf2pVbQFU1YPAU4eLJEma18ZAj1Mn++CBAwe+\nf31zc5PNzc2BTitJ62EymTCZTHZ8XKpO2r/bg5JzgfdX1YXT24eBzaraSrIX+EhV/ewJjq15ziF1\nloQZc5lTVPD789SUhKrKrHHzLqdkejniZuBV0+u/B7xvR+kkSYOYORNPcgOwCTwZ2AL2A/8A3Aj8\nJHAv8FtV9bUTHO9MXGvPmbiGNu9MfK7llCWDWOJae5a4hjb0cook6RRkiUtSY5a4JDVmiUtSY5a4\nJDVmiUtSY5a4JDVmiUtSY5a4JDVmiUtSY0NtRXtShw4d2o3TDOopT3kK55xzztgxJOmkdmXvlD17\nfn6l5xha1ffY2PgKX/3ql8eOoibcO0VDO6U2wOr35H6Y009/Gt/+9sNjB1ETlriG5gZYkvQYYIlL\nUmOWuCQ1ZolLUmOWuCQ1ZolLUmOWuCQ1ZolLUmOWuCQ1ZolLUmNLlXiSNyT5dJK7klyf5AlDBZMk\nzbZwiSd5GvBHwEVVdSHbOyK+dKhgkqTZlt2K9nHAE5M8CpwJfGn5SJKkeS08E6+qLwF/DtwH/Bfw\ntar68FDBJEmzLTwTT/IjwJXAucDXgZuSvLyqbvjB0QeOur45vUiSjphMJkwmkx0ft/B+4kl+A7is\nql4zvf27wMVV9bpjxrmfuNae+4lraLuxn/h9wPOS/FC2n8GXAoeXeDxJ0g4tsyb+ceAm4E7gU0CA\ntw2US5I0B/8823G5nKKdcTlFQ/PPs0nSY4AlLkmNWeKS1JglLkmNWeKS1JglLkmNWeKS1JglLkmN\nWeKS1JglLkmNWeKS1Niyf9lnbX3nO49M98Po5+yzz+XBB784dowd27v3PLa27h07xmPM6T7Pm3MD\nrON6GNhDv9xH9NzUqO8mUrC9iWfH7F1zQ9fn+bzcAEuSHgMscUlqzBKXpMYscUlqzBKXpMYscUlq\nzBKXpMYscUlqzBKXpMYscUlqbKkST3JWkhuTHE7ymSQXDxVMkjTbshtgvRX4QFX9ZpIN4MwBMkmS\n5rTwBlhJ9gB3VtUzZoxzA6xd13NjIDfAGkPX3ND1eT6v3dgA6+nAV5IcTHJHkrclOWOJx5Mk7dAy\nyykbwEXAa6vqk0n+ArgG2P+DQw8cdX1zetHq9N0jWnqsmkwmTCaTHR+3zHLK2cC/V9X509vPB95U\nVS85ZpzLKbuu64/IXXND3+xdc4PLKdsWXk6pqi3g/iQXTO+6FLhn0ceTJO3csr+dchVwfZLHA18A\nXr18JEnSvPzzbMflcso4uuaGvtm75gaXU7b5jk1JaswSl6TGLHFJaswSl6TGLHFJaswSl6TGLHFJ\naswSl6TGLHFJaswSl6TGLHFJaswSl6TGLHFJaswSl6TGLHFJaswSl6TGLHFJaswSl6TGLHFJaswS\nl6TGLHFJaswSl6TGLHFJamzpEk9yWpI7ktw8RCBJ0vyGmIm/HrhngMeRJO3QUiWeZB/wIuDtw8SR\nJO3EsjPxtwBXAzVAFknSDm0semCSFwNbVXUoySaQE48+cNT1zelFknTEZDJhMpns+LhULTaJTvIn\nwO8AjwBnAD8M/H1VvfKYcdVvov4wsId+uY8IPbN3zQ19s3fNDRAW7a8OklBVJ5kcT8cN8R8hyS8B\nb6yqK47zMUt813X9xuyaG/pm75obLPFt/p64JDU2yEz8pCdwJj6CrrOrrrmhb/auucGZ+DZn4pLU\nmCUuSY1Z4pLUmCUuSY1Z4pLUmCUuSY1Z4pLUmCUuSY1Z4pLUmCUuSY1Z4pLUmCUuSY1Z4pLUmCUu\nSY1Z4pLUmCUuSY1Z4pLUmCUuSY1Z4pLUmCUuSY1Z4pLUmCUuSY1Z4pLU2MIlnmRfkluTfCbJ3Umu\nGjKYJGm2VNViByZ7gb1VdSjJk4DbgSur6rPHjCtY7BzjeRjYQ7/cR4Se2bvmhr7Zu+YGCIv2VwdJ\nqKrMGrfwTLyqHqyqQ9Pr3wQOA+cs+niSpJ0bZE08yXnAs4CPDfF4kqT5bCz7ANOllJuA109n5Mdx\n4Kjrm9OLJOmIyWTCZDLZ8XELr4kDJNkA/hH4YFW99QRjXBPfdV3XObvmhr7Zu+YG18S3Lbuc8g7g\nnhMVuCRptZb5FcNLgFcAL0hyZ5I7klw+XDRJ0ixLLafMdQKXU0bQ9Ufkrrmhb/auucHllG2+Y1OS\nGrPEJakxS1ySGrPEJakxS1ySGrPEJakxS1ySGrPEJakxS1ySGrPEJakxS1ySGrPEJakxS1ySGrPE\nJakxS1ySGrPEJakxS1ySGrPEJakxS1ySGrPEJakxS1ySGrPEJamxpUo8yeVJPpvkP5K8aahQkqT5\nLFziSU4D/hK4DHgm8LIkPzNUsD4mYwdYscnYAVZoMnaAFZuMHUC7YJmZ+HOBz1fVvVX1XeBvgSuH\nidXJZOwAKzYZO8AKTcYOsGKTsQNoFyxT4ucA9x91+4HpfZKkXbKxGyfZs+clu3GaAT3CN74xdgZJ\nmi1VtdiByfOAA1V1+fT2NUBV1Z8eM26xE0jSY1xVZdaYZUr8ccDngEuBLwMfB15WVYcXekBJ0o4t\nvJxSVd9L8jrgFrbX1q+zwCVpdy08E5ckjW9l79hMcl2SrSR3reocY0myL8mtST6T5O4kV42daUhJ\nTk/ysSR3Tj+//WNnWoUkpyW5I8nNY2cZWpIvJvnU9Gv48bHzDC3JWUluTHJ4+n148diZhpDkgunX\n7I7pv1+f1S8rm4kneT7wTeBdVXXhSk4ykiR7gb1VdSjJk4DbgSur6rMjRxtMkjOr6lvT1z7+Fbiq\nqtaqDJK8AXgOsKeqrhg7z5CSfAF4TlU9NHaWVUjy18C/VNXBJBvAmVW1Vr9TNn1D5QPAxVV1/4nG\nrWwmXlW3AWv5BKqqB6vq0PT6N4HDrNnvyFfVt6ZXT2f7tZO1WndLsg94EfD2sbOsSFjTvZGS7AF+\noaoOAlTVI+tW4FO/AvznyQoc1vSLvJuSnAc8C/jYuEmGNV1quBN4EPhQVX1i7EwDewtwNWv2P6ej\nFPChJJ9I8pqxwwzs6cBXkhycLju8LckZY4dagd8G3j1rkCW+hOlSyk3A66cz8rVRVY9W1bOBfcDF\nSX5u7ExDSfJiYGv601Sml3VzSVVdxPZPG6+dLm+uiw3gIuCvpp/jt4Brxo00rCSPB64Abpw11hJf\n0HQd7ibgb6rqfWPnWZXpj6kfAS4fO8uALgGumK4bvxv45STvGjnToKrqy9N//wd4L9t7Ha2LB4D7\nq+qT09s3sV3q6+TXgNunX7+TWnWJr+ssB+AdwD1V9daxgwwtyY8nOWt6/QzghcDavGhbVddW1U9V\n1fnAS4Fbq+qVY+caSpIzpz8lkuSJwK8Cnx431XCqagu4P8kF07suBe4ZMdIqvIw5llJghXunJLkB\n2ASenOQ+YP+RFyK6S3IJ8Arg7um6cQHXVtU/jZtsMD8BvHP66vhpwHuq6gMjZ9L8zgbeO93yYgO4\nvqpuGTnT0K4Crp8uO3wBePXIeQaT5Ey2X9T8g7nG+2YfSerLNXFJaswSl6TGLHFJaswSl6TGLHFJ\naswSl6TGLHFJaswSl6TG/g9uA3WivLfUggAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105d046a0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "number_of_rolls = 50\n",
    "dummy = hist( sample_with_replacement( space_die, number_of_rolls ), number_of_bins )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEACAYAAABI5zaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD3hJREFUeJzt3H+s3XV9x/Hnq9QfoN6ustC7WRXdJqKZc7iJC2ruBAU1\noyRdmOicSOL+0A2jibFlf7T+M2XJYkw2/zBiVwzqgGnAxIzSlGOimaICgrR2ZAaozF4hGDpj1DLf\n++N+WW5qe3+cH/fb8+nzkZzc7/dzPt/zeZ97T1/9nM/3e06qCklSW9b1XYAkafwMd0lqkOEuSQ0y\n3CWpQYa7JDXIcJekBi0b7kmuSzKf5N5Fbf+Q5ECSe5L8W5KZRfdtT/JAd/+bJlW4JOnEVjJz3wVc\nfEzbHuDlVfVK4AFgO0CSlwGXA+cCbwY+mSTjK1eStBLLhntVfQ34yTFte6vqV93uN4DN3falwBeq\n6smqepCF4H/1+MqVJK3EONbcrwK+0m0/Dzi06L5HujZJ0hoaKdyT/B1wtKo+P6Z6JEljsH7YA5Nc\nCbwFeMOi5keA5y/a39y1He94v9RGkoZQVcuey1zpzD3dbWEnuQT4EHBpVf1iUb9bgbcleXqSFwG/\nC9y5RIHN3nbs2NF7DT4/n9+p+Pxafm5VK58TLztzT/I5YA44M8nDwA7gGuDpwO3dxTDfqKr3VtX+\nJDcC+4GjwHtrNdVIksZi2XCvqrcfp3nXEv0/Cnx0lKIkSaPxE6oTMjc313cJE+Xzm24tP7+Wn9tq\npK9VkySu2EjSKiWhxnhCVZI0RQx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMM\nd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCX\npAYZ7pLUIMNdU2F29mySTN1tdvbsvn91OkWlqvoZOKm+xtb0SQJM4+sl+DrXOCWhqrJcP2fuktSg\nZcM9yXVJ5pPcu6htY5I9SQ4muS3JhkX3bU/yQJIDSd40qcIlSSe2kpn7LuDiY9q2AXur6hxgH7Ad\nIMnLgMuBc4E3A5/MwvtpSdIaWjbcq+prwE+Oad4C7O62dwOXdduXAl+oqier6kHgAeDV4ylVkrRS\nw665n1VV8wBVdRg4q2t/HnBoUb9HujZJ0hoa1wlVLweQpJPI+iGPm0+yqarmk8wCP+7aHwGev6jf\n5q7tuHbu3Pn/23Nzc8zNzQ1ZjiS1aTAYMBgMVn3ciq5zT3I28OWq+v1u/1rg8aq6NsmHgY1Vta07\noXoDcD4LyzG3A793vAvavc5dq+F17tKClV7nvuzMPcnngDngzCQPAzuAjwE3JbkKeIiFK2Soqv1J\nbgT2A0eB95rgkrT2/ISqpoIzd2mBn1CVpFOY4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhL\nUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1\nyHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDVrf5+CPPfZYn8MPZd26dTz3uc/tuwxJWlKqqp+B\nk3rmM8/sZexRHD36P+zbdzuvf/3r+y7llJIE6Oe1OprQ178xtSkJVZXl+vU6c//5z6dv5j4zs5VH\nH3207zIkaUkjrbkn+UCS7yW5N8kNSZ6eZGOSPUkOJrktyYZxFStJWpmhwz3JbwN/C5xXVa9g4V3A\nFcA2YG9VnQPsA7aPo1BJ0sqNerXMacCzkqwHTgceAbYAu7v7dwOXjTiGJGmVhg73qvpv4B+Bh1kI\n9Seqai+wqarmuz6HgbPGUagkaeWGPqGa5DdYmKW/EHgCuCnJO/j1SxqWuFRg56Ltue4m6WQwO3s2\n8/MP9V3Gqm3a9EIOH36w7zLGZjAYMBgMVn3c0JdCJvlz4OKqek+3/07gNcAbgLmqmk8yC9xRVece\n5/iaxkvbZma28pnPvJ2tW7f2XcopxUsh156/85PTSi+FHGXN/WHgNUmemYVXwYXAfuBW4Mquz7uA\nW0YYQ5I0hKGXZarqziQ3A3cDR7ufnwKeA9yY5CrgIeDycRQqSVq5kT7EVFUfAT5yTPPjwEWjPK4m\nY1rXUKfbM7rlDWlt9foJVa2thWCf1rXIaQ3IX+DvXH3o9btlpvFFPzOzleTrPPHEfN+lDGn6fucL\npvfk3nTWDdNbuydUwZn7UBaCfRpfPM7EpFOF3+cuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4\nS1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrsk\nNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0aKdyTbEhyU5IDSe5Pcn6SjUn2JDmY5LYkG8ZV\nrCRpZUaduX8C+EpVnQv8AfB9YBuwt6rOAfYB20ccQ5K0SkOHe5IZ4HVVtQugqp6sqieALcDurttu\n4LKRq5QkrcooM/cXAY8l2ZXkriSfSnIGsKmq5gGq6jBw1jgKlSSt3PoRjz0PeF9VfTvJx1lYkqlj\n+h27v8jORdtz3U2S9JTBYMBgMFj1calaInuXOjDZBPxHVb24238tC+H+O8BcVc0nmQXu6Nbkjz2+\nlsz9k9TMzFaOHPki01g7hOmsG6a39mmtG6a39jBsrk2DJFRVlus39LJMt/RyKMlLuqYLgfuBW4Er\nu7Z3AbcMO4YkaTijLMsAXA3ckORpwA+AdwOnATcmuQp4CLh8xDEkSas0UrhX1XeBPz7OXReN8riS\npNH4CVVJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrsk\nNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KD\nDHdJapDhLkkNMtwlqUGGuyQ1aORwT7IuyV1Jbu32NybZk+RgktuSbBi9TEnSaoxj5v5+YP+i/W3A\n3qo6B9gHbB/DGJKkVRgp3JNsBt4CfHpR8xZgd7e9G7hslDEkSas36sz948CHgFrUtqmq5gGq6jBw\n1ohjSJJWaf2wByZ5KzBfVfckmVuia534rp2Ltue6myTpKYPBgMFgsOrjUrVE9i51YPL3wF8CTwKn\nA88BvgT8ETBXVfNJZoE7qurc4xxfS+b+SWpmZitHjnyRaawdwnTWDdNb+7TWDdNbexg216ZBEqoq\ny/Ubelmmqq6pqhdU1YuBtwH7quqdwJeBK7tu7wJuGXYMSdJwJnGd+8eANyY5CFzY7UuS1tDQa+6L\nVdVXga92248DF43jcSVJw/ETqpLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S\n1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkN\nMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWrQ0OGeZHOSfUnuT3Jfkqu79o1J9iQ5mOS2\nJBvGV64kaSVGmbk/CXywql4O/AnwviQvBbYBe6vqHGAfsH30MiVJqzF0uFfV4aq6p9v+KXAA2Axs\nAXZ33XYDl41apCRpdcay5p7kbOCVwDeATVU1Dwv/AQBnjWMMSdLKjRzuSZ4N3Ay8v5vB1zFdjt2X\nJE3Y+lEOTrKehWD/bFXd0jXPJ9lUVfNJZoEfn/gRdi7anutukqSnDAYDBoPBqo9L1fAT6yTXA49V\n1QcXtV0LPF5V1yb5MLCxqrYd59iaxkn9zMxWjhz5ItNYO4TprBumt/ZprRumt/YwSq6d7JJQVVmu\n39Az9yQXAO8A7ktyNwuvgmuAa4Ebk1wFPARcPuwYkqThDB3uVfV14LQT3H3RsI8rSRqdn1CVpAYZ\n7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEu\nSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLU\nIMNdkhpkuEtSgyYW7kkuSfL9JP+Z5MOTGkeS9OsmEu5J1gH/BFwMvBy4IslLJzHWyWvQdwETNui7\ngAkb9F3AhA36LkATNqmZ+6uBB6rqoao6CnwB2DKhsU5Sg74LmLBB3wVM2KDvAiZs0HcBmrBJhfvz\ngEOL9n/YtUmS1sD6PgefmfmzPocfyi9/+a2+S5CkZaWqxv+gyWuAnVV1Sbe/DaiqunZRn/EPLEmn\ngKrKcn0mFe6nAQeBC4EfAXcCV1TVgbEPJkn6NRNZlqmq/03yN8AeFtb1rzPYJWntTGTmLknq15p/\nQjXJdUnmk9y71mOvhSSbk+xLcn+S+5Jc3XdN45LkGUm+meTu7rnt6LumSUiyLsldSW7tu5ZxS/Jg\nku92f8M7+65n3JJsSHJTkgPdv8Hz+65pXJK8pPu73dX9fGKpfFnzmXuS1wI/Ba6vqles6eBrIMks\nMFtV9yR5NvAdYEtVfb/n0sYiyRlV9bPuvMrXgaurqqmQSPIB4FXATFVd2nc945TkB8Crquonfdcy\nCUn+BfhqVe1Ksh44o6qO9FzW2HUfFP0hcH5VHTpenzWfuVfV14AmX1gAVXW4qu7ptn8KHKCha/yr\n6mfd5jNYOGfT1Lpeks3AW4BP913LhIRGv1MqyQzwuqraBVBVT7YY7J2LgP86UbBDo3/kk0WSs4FX\nAt/st5Lx6ZYs7gYOA7dXVWsX/n8c+BCN/ae1SAG3J/lWkvf0XcyYvQh4LMmubuniU0lO77uoCfkL\n4PNLdTDcJ6RbkrkZeH83g29CVf2qqv4Q2Aycn+Rlfdc0LkneCsx377zS3VpzQVWdx8K7k/d1y6St\nWA+cB/xz9xx/Bmzrt6TxS/I04FLgpqX6Ge4T0K313Qx8tqpu6bueSeje7t4BXNJ3LWN0AXBpty79\neeBPk1zfc01jVVU/6n4+CnyJhe+BasUPgUNV9e1u/2YWwr41bwa+0/0NT6ivcG91VvSUzwD7q+oT\nfRcyTkl+M8mGbvt04I1AEyeKAarqmqp6QVW9GHgbsK+q/qrvusYlyRndO0qSPAt4E/C9fqsan6qa\nBw4leUnXdCGwv8eSJuUKllmSgR6+WybJ54A54MwkDwM7njoB0oIkFwDvAO7r1qYLuKaq/r3fysbi\nt4Dd3Zn6dcC/VtVXeq5JK7cJ+FL31R/rgRuqak/PNY3b1cAN3dLFD4B391zPWCU5g4WTqX+9bF8/\nxCRJ7XHNXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktSg/wNCc9jcpxtLuQAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105cac0b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "number_of_rolls = 500\n",
    "dummy = hist( sample_with_replacement( space_die, number_of_rolls ), number_of_bins )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEACAYAAABfxaZOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAF2FJREFUeJzt3VGM3Nd93fHvkWmJkkOxrBIu06USq1WWoYy6Fl2skqqu\nJ5BDmk5Lqn1wJBiQFLHNg9jKSIvApF/EfSgSFSgsB60EGHHEpSGboRSoohCCogh2GqSGTNqSQ9lL\nk0QM0tytuYrrgIYqwCDt04e5a45l7u6sMjuzs/d8gMH+5+69y9+flM7cvf//zJVtIiKiDtf1u4CI\niOidhH5EREUS+hERFUnoR0RUJKEfEVGRhH5EREU6Cn1Jn5T0enk8WtrWSDoi6bSklyStbuu/W9JZ\nSackbW5r3yTppKQzkp7o/ulERMRc5g19Se8DdgD/FPgA8C8l/SNgF3DU9gbgGLC79L8D+DiwEdgK\nPClJ5cc9BeywPQKMSNrS5fOJiIg5dDLT3wh8xfYPbf8I+Avg3wDbgPHSZxy4txxvA/bbvmL7HHAW\nGJW0Dlhl+0Tpt69tTERE9EAnof8N4ENlOecm4GPArcCQ7WkA2xeBtaX/MHChbfxUaRsGJtvaJ0tb\nRET0yIr5Otj+lqTHgZeBN4HXgB9dq2uXa4uIiC6bN/QBbD8NPA0g6T/TmslPSxqyPV2Wbt4o3ado\n/SYwY31pm639Z0jKC0hExDtgW3N9v9O7d36hfP0l4F8DXwQOAg+VLg8CL5Tjg8B9kq6XdBtwO3C8\nLAFdkjRaLuw+0DbmWoUv28djjz3W9xpybjm/nN/ye3Sio5k+8GeS/j5wGXjE9g/Kks8BSQ8D52nd\nsYPtCUkHgIm2/jPV7AT2AiuBQ7YPd/jnR0REF3S6vPMvrtH2feAjs/T/A+APrtH+NeAfL7DGiIjo\nkrwjtw8ajUa/S1g0y/ncIOc36Jb7+XVCna4D9ZIkL8W6IiKWMkm4GxdyIyJieUjoR0RUJKEfEVGR\nhH5EREUS+hERFUnoR0RUJKEfEVGRhH5EREU6/eydnvvyl7/c7xIWbMOGDdxyyy39LuMdWbfuvUxP\nn+93GQs2NPTLXLx4rt9lRAyMJfuO3NWrf73fZSzI5cvf58Mffj+HDh3odynvSOuDT5fefwvzU8ef\nLrjU5IU2uq2Td+Qu2Zn+pUuDNtN/kTff/Fy/i4gB0gr8wXvBmp5eydVtrwdLXrCWcOhHxFL1Qwbx\nxQpgenowX6y6KaHfRa+88hcDOwMaXDfk7zxiARL6XXT58g8Y1BkQDGpwDu6sc3D/zmOQdbpd4u9J\n+oakk5KeKVshrpF0RNJpSS9JWt3Wf7eks5JOSdrc1r6p/Iwzkp5YjBOKiIjZzRv6kv4B8B+ATbbf\nT+u3g/uBXcBR2xuAY8Du0v8OWlsnbgS2Ak/q6u/fTwE7bI8AI5K2dPl8IiJiDp2+OetdwHskrQBu\nBKaA7cB4+f44cG853gbst33F9jngLDAqaR2wyvaJ0m9f25iIiOiBeUPf9v8B/ivwHVphf8n2UWDI\n9nTpcxFYW4YMAxfafsRUaRsGJtvaJ0tbRET0yLwXciX9PVqz+l8GLgHPSvoEP3v1rMtX0/a0HTfK\nIyIiZjSbTZrN5oLGdHL3zkeAb9v+PoCk54F/BkxLGrI9XZZu3ij9p4Bb28avL22ztc9iT2dnEBFR\nqUaj8VObvY+Njc07ppM1/e8AvyZpZbkgew8wARwEHip9HgReKMcHgfvKHT63AbcDx8sS0CVJo+Xn\nPNA2JiIiemDemb7t45KeA14DLpevnwNWAQckPQycp3XHDrYnJB2g9cJwGXjEVz8cZSewF1gJHLJ9\nuLunExERc1myH7g2eG+4eZHWjUuDVveMwf3AtcGsGwa39kGtGwb5A/o60ckHruXz9CMiKpLQj4io\nSEI/IqIiCf2IiIok9CMiKpLQj4ioSEI/IqIiCf2IiIok9CMiKpLQj4ioSEI/IqIiCf2IiIok9CMi\nKpLQj4ioSEI/IqIiCf2IiIrMG/qSRiS9JunV8vWSpEclrZF0RNJpSS9JWt02Zreks5JOSdrc1r5J\n0klJZyQ9sVgnFRER1zZv6Ns+Y/tO25uADwL/D3ge2AUctb0BOAbsBpB0B62tEzcCW4Eny564AE8B\nO2yPACOStnT7hCIiYnYLXd75CPDXti8A24Hx0j4O3FuOtwH7bV+xfQ44C4xKWgessn2i9NvXNiYi\nInpgoaH/28AXy/GQ7WkA2xeBtaV9GLjQNmaqtA0Dk23tk6UtIiJ6ZEWnHSW9m9Ys/lOl6e27C3d5\nt+E9bceN8oiIiBnNZpNms7mgMR2HPq31+a/Z/l55Pi1pyPZ0Wbp5o7RPAbe2jVtf2mZrn8WeBZQW\nEVGfRqNBo9H4yfOxsbF5xyxkeed+4Ettzw8CD5XjB4EX2trvk3S9pNuA24HjZQnokqTRcmH3gbYx\nERHRAx3N9CXdROsi7u+2NT8OHJD0MHCe1h072J6QdACYAC4Dj9ieWfrZCewFVgKHbB/uxklERERn\ndDWPlw5J7volgkX3Iq1LHoNW9wwxmLUPat0wuLUPat0AYilmXrdIwrbm6pN35EZEVCShHxFRkYR+\nRERFEvoRERVJ6EdEVCShHxFRkYR+RERFEvoRERVJ6EdEVCShHxFRkYR+RERFEvoRERVJ6EdEVCSh\nHxFRkYR+RERFEvoRERXpKPQlrZb0rKRTkr4p6S5JayQdkXRa0kuSVrf13y3pbOm/ua19k6STks5I\nemIxTigiImbX6Uz/s7S2N9wI/BPgW8Au4KjtDcAxYDeApDtobZ24kdZm6k+WPXEBngJ22B4BRiRt\n6dqZRETEvOYNfUk3Ax+y/TSA7Su2LwHbgfHSbRy4txxvA/aXfueAs8CopHXAKtsnSr99bWMiIqIH\nOpnp3wZ8T9LTkl6V9LmyUfqQ7WkA2xeBtaX/MHChbfxUaRsGJtvaJ0tbRET0yIoO+2wCdtr+qqTP\n0Fraefvuwl3ebXhP23GjPCIiYkaz2aTZbC5oTCehPwlcsP3V8vzPaIX+tKQh29Nl6eaN8v0p4Na2\n8etL22zts9jTQWkREfVqNBo0Go2fPB8bG5t3zLzLO2UJ54KkkdJ0D/BN4CDwUGl7EHihHB8E7pN0\nvaTbgNuB42UJ6JKk0XJh94G2MRER0QOdzPQBHgWekfRu4NvA7wDvAg5Iehg4T+uOHWxPSDoATACX\ngUdszyz97AT2Aitp3Q10uFsnEhER89PVPF46JLnrlwgW3Yu0blwatLpniMGsfVDrhsGtfVDrBhBL\nMfO6RRK2NVefvCM3IqIiCf2IiIok9CMiKpLQj4ioSEI/IqIiCf2IiIok9CMiKpLQj4ioSEI/IqIi\nCf2IiIok9CMiKpLQj4ioSEI/IqIiCf2IiIok9CMiKtJR6Es6J+mvJL0m6XhpWyPpiKTTkl6StLqt\n/25JZyWdkrS5rX2TpJOSzkh6ovunExERc+l0pv9joGH7TtujpW0XcNT2BuAYsBtA0h20dtHaCGwF\nnizbIwI8BeywPQKMSNrSpfOIiIgOdBr6ukbf7cB4OR4H7i3H24D9tq/YPgecBUbL5umrbJ8o/fa1\njYmIiB7oNPQNvCzphKR/W9qGyqbplE3P15b2YeBC29ip0jYMTLa1T5a2iIjokU43Rr/b9ncl/QJw\nRNJpfnaTzOW78WRExDLRUejb/m75+jeS/gcwCkxLGrI9XZZu3ijdp4Bb24avL22ztc9iT9txozwi\nImJGs9mk2WwuaIzm2xle0k3AdbbflPQe4AgwBtwDfN/245I+BayxvatcyH0GuIvW8s3LwK/YtqRX\ngEeBE8CfA39k+/A1/kwP3i8OL9K6nDFodc8Qg1n7oNYNg1v7oNYNIObLvEEmCduaq08nM/0h4PlW\nELMCeMb2EUlfBQ5Iehg4T+uOHWxPSDoATACXgUd89W95J7AXWAkculbgR0TE4pl3pt8Pmen3w6DO\n3ga1bhjc2ge1bshMP+/IjYioSkI/IqIiCf2IiIok9CMiKpLQj4ioSEI/IqIiCf2IiIok9CMiKpLQ\nj4ioSEI/IqIiCf2IiIok9CMiKpLQj4ioSEI/IqIiCf2IiIok9CMiKtJx6Eu6TtKrkg6W52skHZF0\nWtJLkla39d0t6aykU5I2t7VvknRS0hlJT3T3VCIiYj4Lmel/ktYWiDN2AUdtbwCOAbsByh65Hwc2\nAluBJyXN7OTyFLDD9ggwImnL37H+iIhYgI5CX9J64GPAH7c1bwfGy/E4cG853gbst33F9jngLDAq\naR2wyvaJ0m9f25iIiOiBTmf6nwF+n5/eGHPI9jSA7YvA2tI+DFxo6zdV2oaBybb2ydIWERE9Mm/o\nS/otYNr212ntiDyb5bvbcETEMrGigz53A9skfQy4EVgl6QvARUlDtqfL0s0bpf8UcGvb+PWlbbb2\nWexpO26UR0REzGg2mzSbzQWNkd35BF3Sh4H/ZHubpP8C/F/bj0v6FLDG9q5yIfcZ4C5ayzcvA79i\n25JeAR4FTgB/DvyR7cPX+HM8eL84vEjrcsag1T1DDGbtg1o3DG7tg1o3gFhI5g0aSdiea0Wmo5n+\nbP4QOCDpYeA8rTt2sD0h6QCtO30uA4/46t/yTmAvsBI4dK3Aj4iIxbOgmX6vZKbfD4M6exvUumFw\nax/UuiEz/bwjNyKiKgn9iIiKJPQjIiqS0I+IqEhCPyKiIgn9iIiKJPQjIiqS0I+IqEhCPyKiIgn9\niIiKJPQjIiqS0I+IqEhCPyKiIgn9iIiKJPQjIiqS0I+IqEgnG6PfIOkrkl6T9Lqkx0r7GklHJJ2W\n9JKk1W1jdks6K+mUpM1t7ZsknZR0RtITi3NKERExm3lD3/YPgd+wfSfwAWCrpFFgF3DU9gbgGLAb\noOyR+3FgI7AVeFLSzE4uTwE7bI8AI5K2dPuEIiJidh0t79h+qxzeQGtfXQPbgfHSPg7cW463Aftt\nX7F9DjgLjEpaB6yyfaL029c2JiIieqCj0Jd0naTXgIvAyyW4h2xPA9i+CKwt3YeBC23Dp0rbMDDZ\n1j5Z2iIiokdWdNLJ9o+BOyXdDDwv6X387M7IXd5teE/bcaM8IiJiRrPZpNlsLmhMR6E/w/YPJDWB\njwLTkoZsT5elmzdKtyng1rZh60vbbO2z2LOQ0iIiqtNoNGg0Gj95PjY2Nu+YTu7e+fmZO3Mk3Qj8\nJnAKOAg8VLo9CLxQjg8C90m6XtJtwO3A8bIEdEnSaLmw+0DbmIiI6IFOZvq/CIxLuo7Wi8Sf2j4k\n6RXggKSHgfO07tjB9oSkA8AEcBl4xPbM0s9OYC+wEjhk+3BXzyYiIuakq3m8dEhy1y8RLLoXad24\nNGh1zxCDWfug1g2DW/ug1g0glmLmdYskbGuuPnlHbkRERRL6EREVSehHRFQkoR8RUZGEfkRERRL6\nEREVSehHRFQkoR8RUZGEfkRERRL6EREVSehHRFQkoR8RUZGEfkRERRL6EREVSehHRFQkoR8RUZFO\ntktcL+mYpG9Kel3So6V9jaQjkk5LemlmS8Xyvd2Szko6JWlzW/smSSclnZH0xOKcUkREzKaTmf4V\n4D/afh/w68BOSb8K7AKO2t4AHAN2A0i6g9bWiRuBrcCTZU9cgKeAHbZHgBFJW7p6NhERMad5Q9/2\nRdtfL8dv0toUfT2wHRgv3caBe8vxNmC/7Su2zwFngVFJ64BVtk+UfvvaxkRERA8saE1f0nuBDwCv\nAEO2p6H1wgCsLd2GgQttw6ZK2zAw2dY+WdoiIqJHVnTaUdLPAc8Bn7T9Zmvz8p/S5d2G97QdN8oj\nIiJmNJtNms3mgsZ0FPqSVtAK/C/YfqE0T0sasj1dlm7eKO1TwK1tw9eXttnaZ7Gnk9IiIqrVaDRo\nNBo/eT42NjbvmE6Xd/4EmLD92ba2g8BD5fhB4IW29vskXS/pNuB24HhZArokabRc2H2gbUxERPTA\nvDN9SXcDnwBel/QarWWcTwOPAwckPQycp3XHDrYnJB0AJoDLwCO2Z5Z+dgJ7gZXAIduHu3s6EREx\nF13N46Wjdb1g6dU1txdp3bg0aHXPEINZ+6DWDYNb+6DWDSCWYuZ1iyRsa64+eUduRERFEvoRERVJ\n6EdEVCShHxFRkYR+RERFEvoRERVJ6EdEVCShHxFRkYR+RERFEvoRERVJ6EdEVCShHxFRkYR+RERF\nEvoRERVJ6EdEVCShHxFRkXlDX9LnJU1LOtnWtkbSEUmnJb0kaXXb93ZLOivplKTNbe2bJJ2UdEbS\nE90/lYiImE8nM/2ngS1va9sFHLW9ATgG7AaQdAetbRM3AluBJ8t+uABPATtsjwAjkt7+MyMiYpHN\nG/q2/xL427c1bwfGy/E4cG853gbst33F9jngLDAqaR2wyvaJ0m9f25iIiOiRd7qmv9b2NIDti8Da\n0j4MXGjrN1XahoHJtvbJ0hYRET20oks/ZxF2Gt7Tdtwoj4iImNFsNmk2mwsa805Df1rSkO3psnTz\nRmmfAm5t67e+tM3WPoc977C0iIg6NBoNGo3GT56PjY3NO6bT5R2Vx4yDwEPl+EHghbb2+yRdL+k2\n4HbgeFkCuiRptFzYfaBtTERE9Mi8M31JX6S1tnKLpO8AjwF/CDwr6WHgPK07drA9IekAMAFcBh6x\nPbP0sxPYC6wEDtk+3N1TiYiI+ehqJi8dkrwolwkW1Yu0bl4atLpniMGsfVDrhsGtfVDrBhBLMfO6\nRRK2NVefvCM3IqIiCf2IiIok9CMiKpLQj4ioSEI/IqIiCf2IiIok9CMiKpLQj4ioSEI/IqIiCf2I\niIok9CMiKpLQj4ioSEI/IqIiCf2IiIok9CMiKtLz0Jf0UUnfknRG0qd6/edHRNSsp6Ev6TrgvwFb\ngPcB90v61V7WsDQ0+13AImr2u4BF1ux3AYus2e8CYpH1eqY/Cpy1fd72ZWA/sL3HNSwBzX4XsIia\n/S5gkTX7XcAia/a7gFhkvQ79YeBC2/PJ0hYRET0w78bo/XLzzf+q3yUsyJUrF3nrrX5XERExt55u\njC7p14A9tj9anu8CbPvxt/VbvjsXR0Qsovk2Ru916L8LOA3cA3wXOA7cb/tUz4qIiKhYT5d3bP9I\n0r8HjtC6nvD5BH5ERO/0dKYfERH9taTekSvp85KmJZ3sdy3dJmm9pGOSvinpdUmP9rumbpJ0g6Sv\nSHqtnN9j/a6p2yRdJ+lVSQf7XctikHRO0l+Vf8Pj/a6nmyStlvSspFPl/8G7+l1Tt0gaKf9mr5av\nl+bKlyU105f0z4E3gX2239/verpJ0jpgne2vS/o54GvAdtvf6nNpXSPpJttvlWs3/xt41PayCQ9J\nvwd8ELjZ9rZ+19Ntkr4NfND23/a7lm6TtBf4X7aflrQCuMn2D/pcVteVN8BOAnfZvnCtPktqpm/7\nL4Fl9x8cgO2Ltr9ejt8ETrHM3qNge+am1RtoXS9aOjOKvyNJ64GPAX/c71oWkVhimdANkm4GPmT7\naQDbV5Zj4BcfAf56tsCHZfgPPAgkvRf4APCV/lbSXWX54zXgIvCy7RP9rqmLPgP8PsvohewaDLws\n6YSkf9fvYrroNuB7kp4uSyCfk3Rjv4taJL8NfGmuDgn9HitLO88Bnywz/mXD9o9t3wmsB+6SdEe/\na+oGSb8FTJff1FQey9HdtjfR+o1mZ1luXQ5WAJuA/17O7y1gV39L6j5J7wa2Ac/O1S+h30NlLfE5\n4Au2X+h3PYul/Or8P4GP9ruWLrkb2FbWvL8E/IakfX2uqetsf7d8/RvgeVqflbUcTAIXbH+1PH+O\n1ovAcrMV+Fr595vVUgz95TyT+hNgwvZn+11It0n6eUmry/GNwG8Cy+Iite1P2/4l2/8QuA84ZvuB\nftfVTZJuKr+FIuk9wGbgG/2tqjtsTwMXJI2UpnuAiT6WtFjuZ56lHVhin70j6YtAA7hF0neAx2Yu\nvgw6SXcDnwBeL+veBj5t+3B/K+uaXwTGy90D1wF/avtQn2uKzg0Bz5ePQFkBPGP7SJ9r6qZHgWfK\nEsi3gd/pcz1dJekmWhdxf3fevkvpls2IiFhcS3F5JyIiFklCPyKiIgn9iIiKJPQjIiqS0I+IqEhC\nPyKiIgn9iIiKJPQjIiry/wHUeb17sFYdwAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105c4e630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "number_of_rolls = 50000\n",
    "dummy = hist( sample_with_replacement( space_die, number_of_rolls ), number_of_bins )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sample exercises\n",
    "\n",
    "**1.** What's the sample space of the sum of two dice?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# A reminder for one die:\n",
    "# space_die = [1, 2, 3, 4, 5, 6]\n",
    "space_two_dice = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**2.** Will choosing a value uniformly at random from this sample space yield the population distribution that our intuition tells us would really occur with two dice in the real world?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Type either \"True\" or \"False\", without quotes and with that exact capitalization, on the line below\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**3.** Consider the following slight modification of the \"shorthand\" for rolling one die that we used above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[8]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def sample_two_dice( space, number ):\n",
    "    \n",
    "    return [( random.choice( space ) + random.choice( space ) ) for i in range( number )]\n",
    "\n",
    "sample_two_dice( space_die, 1 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9, 8, 6, 8, 6, 10, 2, 7, 6, 11]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_two_dice( space_die, 10 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modify the text below so it shows us a histogram for 50 rolls of *two* dice rather than one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXEAAAEACAYAAABF+UbAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADQFJREFUeJzt3X+M5PVdx/HnCxYqFMFNTVntWa/8gcZGgjRKA21cSxTS\npmCINdRqU0z6VyukTQgU/7jzHxNMTG388QcpntCAGq72h0nVg+CSUBXawgGFozYhhW3aWyWHbQjG\nFHn7xw7lcnI3uzOz89337fORTJiZfGc+7+/c3ZPvfnd3JlWFJKmnU4YeQJI0OSMuSY0ZcUlqzIhL\nUmNGXJIaM+KS1NjYiCe5LclakseOuu+PkhxKcjDJZ5OcvbVjSpJey0aOxPcBlx9z3wHgrVV1IfBN\n4BOzHkySNN7YiFfVA8Dzx9x3b1W9PLr5b8CuLZhNkjTGLM6J/y7wDzN4HknSJk0V8SS/D/ygqu6a\n0TySpE1YmPSBST4EvBt415jtfHMWSZpAVWXcNhs9Es/osn4juQK4Abiyqv5nA4N4qWLPnj2Dz7Bd\nLr4W/V6L0b/mLb7sOc79O68jG7WRHzG8C/gX4Pwkzya5FvhT4CzgniQPJ/mLDa8oSZqZsadTquq3\nXuPufVswiyRpk/yNzTlaXl4eeoRtw9fiVb4WR1seeoB2splzLxMtkNRWryFp6yXhlfPTA6y+qfPE\nJ4Mk1Ay/sSlJ2oaMuCQ1ZsQlqTEjLkmNGXFJasyIS1JjRlySGjPiktSYEZekxoy4JDVmxCWpMSMu\nSY0ZcUlqzIhLUmNGXJIaM+KS1JgRl6TGjLgkNWbEJakxIy5JjRlxSWrMiEtSY0Zckhoz4pLUmBGX\npMaMuCQ1NjbiSW5LspbksaPuW0xyIMk3kvxTknO2dkxJ0mvZyJH4PuDyY+67Cbi3qn4GuA/4xKwH\nkySNNzbiVfUA8Pwxd18F3D66fjvw6zOeS5K0AZOeE39jVa0BVNVh4I2zG0mStFGz+sZmzeh5JEmb\nsDDh49aSnFtVa0mWgP840cZ79+794fXl5WWWl5cnXFaS5mdpaTdra88MPcYJpWr8QXSS3cDfV9XP\nj27fAhypqluS3AgsVtVNx3lsbWQNSdtbEob7ojsM0ZFtsM8Zu9W4FybJXcAy8AZgDdgDfB64G/gp\n4BngN6vqv47zeCMunQS2QdDmv+rw+zx9xKcew4hLJ4VtELT5rzr8Po+NuL+xKUmNGXFJasyIS1Jj\nRlySGjPiktSYEZekxoy4JDVmxCWpMSMuSY0ZcUlqzIhLUmNGXJIaM+KS1JgRl6TGjLg0oaWl3SSZ\n+2VpaffQu65txPcTlyY03HtN79j31p7/qsPvs+8nLkknMyMuSY0ZcUlqzIhLUmNGXJIaM+KS1JgR\nl6TGjLgkNWbEJakxIy5JjRlxSWrMiEtSY0ZckhqbKuJJPpbk60keS3JnktNnNZgkabyJI57kJ4Hf\nAy6qqguABeCaWQ0mSRpvYcrHnwq8PsnLwJnAd6YfSZK0URNHvKq+k+SPgWeBF4EDVXXva227d+/e\nSZeZ2AUXXMDVV18993UlaZ4m/mSfJD8GfBZ4H/A9YD9wd1Xddcx2Bb981D27R5et9D0WF/dz5Mjq\nFq+jncxP9pnr6jtgn1dGl1f8wYY+2WeaiP8GcHlVfXh0+3eAi6vqo8dsV/P/g19lcfESI64tZcTn\nuvpO3ect/Xi2Z4G3J/mRrO/pZcChKZ5PkrRJE0e8qh5i/RTKI8CjQIBbZzSXJGkD5vJp955O0cnI\n0ylzXX2n7rOfdi9JJzMjLkmNGXFJasyIS1JjRlySGjPiktSYEZekxoy4JDVmxCWpMSMuSY0ZcUlq\nzIhLUmNGXJIaM+KS1JgR10wsLe0mydwvS0u7h951aVDTftq9BMDa2jMM8b7La2tj325ZOql5JC5J\njRlxSWrMiEtSY0Zckhoz4pLUmBGXpMaMuCQ1ZsQlqTEjLkmNGXFJasyIS1JjRlySGpsq4knOSXJ3\nkkNJnkhy8awGkySNN+27GH4K+FJVvS/JAnDmDGaSJG3QxBFPcjbwzqr6EEBVvQR8f0ZzSZI2YJrT\nKW8BnkuyL8nDSW5NcsasBpMkjTfN6ZQF4CLgI1X11SR/AtwE7Pn/m+496vry6CJJetXK6LI5qZrs\n01iSnAv8a1WdN7r9DuDGqnrvMdvV/D/xZZXFxUs4cmR1zuvuXEkY4pN9IEz6d3jqlXfYPg+3v7CD\n93nsR1dNfDqlqtaA1STnj+66DHhy0ueTJG3etD+dch1wZ5LTgKeBa6cfSZK0UVNFvKoeBX5xRrNI\nkjbJ39iUpMaMuCQ1ZsQlqTEjLkmNGXFJasyIS1JjRlySGjPiktSYEZekxoy4JDVmxCWpMSMuSY0Z\ncUlqzIhLUmNGXJIaM+KS1JgRl6TGjLgkNWbEJakxIy5JjRlxSWrMiEtSY0Zckhoz4pLUmBGXpMaM\nuCQ1ZsQlqTEjLkmNTR3xJKckeTjJF2cxkCRp42ZxJH498OQMnkeStElTRTzJLuDdwKdnM44kaTOm\nPRL/JHADUDOYRZK0SQuTPjDJe4C1qjqYZBnI8bfee9T15dHl5LW0tJu1tWfmvu655/40hw9/a+7r\nSpqFldFlc1I12UF0kj8Efht4CTgD+FHg76rqg8dsV/M/UF9lcfESjhxZnfO665IwzBcnYdI/z6lX\ndp/nufIg+zzc/sIO3ucTHByvm/h0SlXdXFVvrqrzgGuA+44NuCRpa/lz4pLU2MTnxI9WVfcD98/i\nuSRJG+eRuCQ1ZsQlqTEjLkmNGXFJasyIS1JjRlySGjPiktSYEZekxoy4JDVmxCWpMSMuSY0ZcUlq\nzIhLUmNGXJIaM+KS1JgRl6TGjLgkNWbEJakxIy5JjRlxSWrMiEtSY0Zckhoz4pLUmBGXpMaMuCQ1\nZsQlqTEjLkmNGXFJamziiCfZleS+JE8keTzJdbMcTJI03sIUj30J+HhVHUxyFvC1JAeq6qkZzSZJ\nGmPiI/GqOlxVB0fXXwAOAW+a1WCSpPFmck48yW7gQuDBWTyfJGljpo746FTKfuD60RG5JGlOpjkn\nTpIF1gP+mar6wvG33HvU9eXRRZL0qpXRZXNSVRMvmeQO4Lmq+vgJtimYfI3JrLK4eAlHjqzOed11\nSZj/PgOEaf48p1rZfZ7nyoPs83D7Czt4nzNuq2l+xPBS4APAu5I8kuThJFdM+nySpM2b+HRKVX0Z\nOHWGs0iSNsnf2JSkxoy4JDVmxCWpMSMuSY0ZcUlqzIhLUmNGXJIaM+KS1JgRl6TGjLgkNWbEJakx\nIy5JjRlxSWrMiEtSY0Zckhoz4pLUmBGXpMaMuCQ1ZsQlqTEjLkmNGXFJasyIS1JjRlySGjPiktSY\nEZekxoy4JDVmxCWpMSMuSY1NFfEkVyR5Ksm/J7lxVkNJkjZm4ognOQX4M+By4K3A+5P87KwGOzmt\nDD3ANrIy9ADbyMrQA2wjK0MP0M40R+K/BHyzqp6pqh8AfwNcNZuxTlYrQw+wjawMPcA2sjL0ANvI\nytADtDNNxN8ErB51+9uj+yRJc7Iwj0XOPvu981jmh6r+m9NPP22ua0rSEFJVkz0weTuwt6quGN2+\nCaiquuWY7SZbQJJ2uKrKuG2mifipwDeAy4DvAg8B76+qQxM9oSRp0yY+nVJV/5vko8AB1s+t32bA\nJWm+Jj4SlyQNb8t+YzPJbUnWkjy2VWt0kGRXkvuSPJHk8STXDT3TUJK8LsmDSR4ZvRZ7hp5paElO\nSfJwki8OPcuQknwryaOjvxsPDT3PkJKck+TuJIdG3bj4hNtv1ZF4kncALwB3VNUFW7JIA0mWgKWq\nOpjkLOBrwFVV9dTAow0iyZlV9eLoeypfBq6rqh37jzbJx4C3AWdX1ZVDzzOUJE8Db6uq54eeZWhJ\n/gq4v6r2JVkAzqyq7x9v+y07Eq+qB4Ad/wdSVYer6uDo+gvAIXbwz9NX1Yujq69j/XsyO/Z8XpJd\nwLuBTw89yzYQfC8nkpwNvLOq9gFU1UsnCjj4os1Vkt3AhcCDw04ynNHpg0eAw8A9VfWVoWca0CeB\nG9jB/yM7SgH3JPlKkg8PPcyA3gI8l2Tf6DTbrUnOONEDjPicjE6l7AeuHx2R70hV9XJV/QKwC7g4\nyc8NPdMQkrwHWBt9lZbRZSe7tKouYv0rk4+MTsfuRAvARcCfj16PF4GbTvQAIz4Ho/Na+4HPVNUX\nhp5nOxh9ifjPwBVDzzKQS4ErR+eC/xr4lSR3DDzTYKrqu6P//ifwOdbfm2kn+jawWlVfHd3ez3rU\nj2urI+4Rxrq/BJ6sqk8NPciQkvx4knNG188AfhXYkd/graqbq+rNVXUecA1wX1V9cOi5hpDkzNFX\nqiR5PfBrwNeHnWoYVbUGrCY5f3TXZcCTJ3rMlr13SpK7gGXgDUmeBfa8crJ+J0lyKfAB4PHRueAC\nbq6qfxx2skH8BHD76G2MTwH+tqq+NPBMGt65wOdGb9GxANxZVQcGnmlI1wF3JjkNeBq49kQb+8s+\nktSY58QlqTEjLkmNGXFJasyIS1JjRlySGjPiktSYEZekxoy4JDX2f8nu3lpS63VCAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105f749e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "number_of_rolls = 50\n",
    "number_of_bins = 11\n",
    "dummy = hist( sample_with_replacement( space_die, number_of_rolls ), number_of_bins )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**4.** As a hint to the previous problem, modify the text below so it shows a histogram very close to the underlying population distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADZtJREFUeJzt3W+sZPVdx/H3BzZWWwKSRvam3ZaVB60Nad2QWKvEZGpV\niMZCfFAtTSw1MX0ghbSJgeKDvY9MaSKGKDyo/Aka0Fj8U2hKoWSdGDQtWlj5KzZp+Ge7t2pBIRpC\ny9cHd6DLZpd779yZOfd+5/1KJjlzZub8vnPuzmd/8zvzOydVhSRp9ztp6AIkSbNhoEtSEwa6JDVh\noEtSEwa6JDVhoEtSExsGepJ9SQ4leSTJQ0k+MVl/MMkzSe6f3M6ff7mSpBPJRr9DT7ICrFTV4SSn\nAF8HLgB+HXi+qq6ef5mSpI3s2egJVXUEODJZfiHJY8BbJw9njrVJkrZgS2PoSfYDB4CvTVZdkuRw\nkuuTnDbj2iRJW7DpQJ8Mt9wGXFZVLwDXAWdV1QHWe/AOvUjSgDYcQwdIsgf4InBnVV1znMfPBO6o\nqvcc5zFPFiNJU6iqLQ1rb7aHfiPw6NFhPjlY+opfAx5+naK8VXHw4MHBa9gpN/eF+8J98fq3aWx4\nUDTJucBHgIeSPAAUcCVwUZIDwMvAE8DHp6pAkjQTm/mVyz8AJx/noS/PvhxJ0rScKbpAo9Fo6BJ2\nDPfFD7gvfsB9sT2bOii6rQaSmncbktRNEmpOB0UlSTucgS5JTRjoktSEgS5JTRjoktSEgS5JTRjo\nktSEgS5JTRjoktSEgS5JTRjoktSEgS5JTRjoktSEgS5JTRjoktSEgS5JTRjoktSEgS5JTRjoktSE\ngS5JTRjoktSEgS5JTRjoktSEgS5JTRjoktSEgS5JTRjoktSEgS5JTRjoktSEga5dbWVlP0kGua2s\n7B/67UuvkaqabwNJzbsNLa8kwFD/voL/tjUvSaiqbOU19tAlqQkDXZKaMNAlqYkNAz3JviSHkjyS\n5KEkl07Wn57k7iSPJ7kryWnzL1eSdCIbHhRNsgKsVNXhJKcAXwcuAD4G/FdVfTbJ5cDpVXXFcV7v\nQVHNjQdF1dVcDopW1ZGqOjxZfgF4DNjHeqjfPHnazcCFWytXkjRLWxpDT7IfOAB8FdhbVWuwHvrA\nGbMuTpK0eXs2+8TJcMttwGVV9UKSY79rnvC75+rq6qvLo9GI0Wi0tSolqbnxeMx4PN7WNjY1sSjJ\nHuCLwJ1Vdc1k3WPAqKrWJuPsf1dV7zrOax1D19w4hq6u5jmx6Ebg0VfCfOJ24OLJ8keBL2ylYUnS\nbG3mVy7nAn8PPMR6V6iAK4H7gL8E3gY8CXyoqp47zuvtoWtu7KGrq2l66J7LRbuaga6uPJeLJC0x\nA12SmjDQJakJA12SmjDQJakJA12SmjDQJakJA12SmjDQJakJA12SmjDQJakJA12SmjDQJakJA12S\nmjDQJakJA12SmjDQJakJA12SmjDQJakJA12SmjDQJakJA12SmjDQJakJA12SmjDQJakJA12SmjDQ\nJakJA12SmjDQJakJA12SmjDQJakJA12SmjDQJakJA12SmjDQJamJDQM9yQ1J1pI8eNS6g0meSXL/\n5Hb+fMuUJG1kMz30m4DzjrP+6qo6Z3L78ozrkiRt0YaBXlX3As8e56HMvhxJ0rS2M4Z+SZLDSa5P\nctrMKpIkTWXaQL8OOKuqDgBHgKtnV5IkaRp7pnlRVf3HUXf/BLjj9Z6/urr66vJoNGI0Gk3TrHaw\nlZX9rK09OXQZC/YGksWPPO7deyZHjjyx8HY1X+PxmPF4vK1tpKo2flKyH7ijqt49ub9SVUcmy58E\nfqqqLjrBa2szbWh3Ww+2If7OQ7U7ZNvBz1R/SaiqLfUYNuyhJ7kVGAFvTvIUcBB4f5IDwMvAE8DH\nt1ytJGmmNtVD31YD9tCXgj30xbbrZ6q/aXrozhSVpCYMdElqwkCXpCYMdElqwkCXpCYMdElqwkCX\npCYMdElqwkCXpCYMdElqwkCXpCYMdElqwkCXpCYMdElqwkCXpCYMdElqwkCXpCYMdElqwkCXpCYM\ndElqwkCXpCYMdElqwkCXpCYMdElqwkCXpCYMdElqwkCXpCYMdElqwkCXpCYMdElqwkCXpCYMdElq\nwkCXtCkrK/tJMshtZWX/0G9/V0hVzbeBpObdhoaXBBji7zxUu0O2HYb4TA33N4ah3vOQklBV2cpr\n7KFLUhMGuiQ1sWGgJ7khyVqSB49ad3qSu5M8nuSuJKfNt0xJ0kY200O/CTjvmHVXAPdU1TuBQ8Cn\nZ12YJGlrNgz0qroXePaY1RcAN0+WbwYunHFdkqQtmnYM/YyqWgOoqiPAGbMrSZI0jVkdFF2u3xNJ\n0g60Z8rXrSXZW1VrSVaA77zek1dXV19dHo1GjEajKZuVpJ7G4zHj8Xhb29jUxKIk+4E7qurdk/tX\nAd+tqquSXA6cXlVXnOC1TixaAk4sWmy7Tizqb5qJRRsGepJbgRHwZmANOAj8LfB54G3Ak8CHquq5\nE7zeQF8CBvpi2zXQ+5tLoG+Xgb4cDPTFtmug9+fUf0laYga6JDVhoEtSEwa6JDVhoEtSEwa6JDVh\noEtSEwa6JDVhoEtSEwa6JDVhoEtSEwa6JDVhoEtSE9Ne4EI70MrKftbWnhy6DEkD8fS5jQx9elNP\nn7u4dj19bn+ePleSlpiBLklNGOiS1ISBLklNGOiS1ISBLklNGOiS1IQTi6Rd5w2T34RLr2WgS7vO\niww3iUs7mUMuktSEgS5JTRjoktSEgS5JTRjoktSEgS5JTRjoktSEv0OfA68cJM3aMJOp9u49kyNH\nnlh4u9PyikVzMNyVXZbz6j2+5+7tDtn2cFdK8opFkrTEDHRJasJAl6QmtnVQNMkTwH8DLwMvVdV7\nZ1GUJGnrtvsrl5eBUVU9O4tiJEnT2+6QS2awDUnSDGw3jAv4SpJ/SvLbsyhIkjSd7Q65nFtV307y\nY6wH+2NVde+xT1pdXX11eTQaMRqNttnsxpzcI2k3GY/HjMfjbW1jZhOLkhwEnq+qq49ZP8jEouEm\n94ATP5ah3SHbXrZ2h2x7SSYWJXljklMmy28Cfgl4eNrtSZK2ZztDLnuBv0lSk+3cUlV3z6YsSdJW\ntT2Xi0Muy9K277l/u0O2vSRDLpKkncVAl6QmDHRJasJAl6QmvGKRJJ3QMFdKmpaBLkkn9CLD/rJn\naxxykaQmDHRJasJAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJasJAl6QmDHRJasJA\nl6QmDHRJasJAl6QmFnI+9Esu+dQimpGkpbaQQL/22n2LaOYozy64PUkaXqrmezWOJLX4K348Dbyd\nYa80MkTbQ7U7ZNu+5/7tDtn2sO+5qrZ02SLH0CWpCQNdkpow0CWpCQNdkpow0CWpCQNdkpow0CWp\nCQNdkpow0CWpCQNdkpow0CWpiW0FepLzk/xrkn9LcvmsipIkbd3UgZ7kJOCPgfOAs4EPJ/mJWRXW\n03joAnaQ8dAF7CDjoQvYQcZDF7CrbaeH/l7gG1X1ZFW9BPwFcMFsyupqPHQBO8h46AJ2kPHQBewg\n46EL2NW2E+hvZf08ta94ZrJOkjSAhVzg4tRTf3URzbyq6v94/vmFNilJg5v6AhdJ3gesVtX5k/tX\nAFVVVx3zvKHODi9Ju9pWL3CxnUA/GXgc+ADwbeA+4MNV9dhUG5QkbcvUQy5V9f0klwB3sz4Wf4Nh\nLknDmfs1RSVJizG3maJJ9iU5lOSRJA8luXRebe0GSU5Kcn+S24euZUhJTkvy+SSPTf5t/PTQNQ0l\nySeTPJzkwSS3JPmhoWtapCQ3JFlL8uBR605PcneSx5PcleS0IWtclBPsi89OPieHk/xVklM32s48\np/5/D/hUVZ0N/AzwO0s+8egy4NGhi9gBrgG+VFXvAn4SWMphuiRvAT4BnFNV72F9+PM3hq1q4W5i\nfWLi0a4A7qmqdwKHgE8vvKphHG9f3A2cXVUHgG+wiX0xt0CvqiNVdXiy/ALrH9yl/J16kn3ALwPX\nD13LkCY9jJ+rqpsAqup7VfU/A5c1pJOBNyXZA7wR+NbA9SxUVd0LPHvM6guAmyfLNwMXLrSogRxv\nX1TVPVX18uTuV4F9G21nISfnSrIfOAB8bRHt7UB/CPwusOwHLH4c+M8kN02Gnz6X5EeGLmoIVfUt\n4A+Ap4B/B56rqnuGrWpHOKOq1mC9UwicMXA9O8VvAXdu9KS5B3qSU4DbgMsmPfWlkuRXgLXJt5VM\nbstqD3AOcG1VnQP8L+tfsZdOkh9lvTd6JvAW4JQkFw1b1Y607J0gkvwe8FJV3brRc+ca6JOvkrcB\nf1ZVX5hnWzvYucAHk3wT+HPg/Un+dOCahvIM8HRV/fPk/m2sB/wy+gXgm1X13ar6PvDXwM8OXNNO\nsJZkL0CSFeA7A9czqCQXsz5cu6n/7OfdQ78ReLSqrplzOztWVV1ZVW+vqrNYP+h1qKp+c+i6hjD5\nKv10kndMVn2A5T1Q/BTwviQ/nCSs74tlPEB87LfW24GLJ8sfBZapI/iafZHkfNaHaj9YVS9uZgPz\n/NniucBHgJ9P8sBkzPT8ebWnXeNS4JYkh1n/lcvvD1zPIKrqPta/oTwA/AvrH+TPDVrUgiW5FfhH\n4B1JnkryMeAzwC8meWUW+meGrHFRTrAv/gg4BfjKJD+v23A7TiySpB68BJ0kNWGgS1ITBrokNWGg\nS1ITBrokNWGgS1ITBrokNWGgS1IT/w8Xn6Kt/DwSDQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105e5f208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "number_of_rolls = 100\n",
    "dummy = hist( sample_two_dice( space_die, number_of_rolls ), number_of_bins )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Comparing two samples parametrically\n",
    "\n",
    "One of the most useful questions we can ask about two samples is *whether they were drawn from the same underlying population*. Type of quantitative test allows us to provide an answer to questions ranging from, \"Is pathway X involved in the response to simulus Y,\" to, \"Does treatment X improve outcomes for disease Y?\" The basic process for such a test is:\n",
    "\n",
    "1. Generate some measurements from a baseline condition, e.g. prior to stimulus or without treatment. This is typically referred to as the **control** condition.\n",
    "\n",
    "2. Perform the intervention, e.g. treat cells or patients.\n",
    "\n",
    "3. Generate some more measurements, typically referred to as the **case** condition.\n",
    "\n",
    "4. Test whether the *control* sample is statistically indistinguishable from the *case* sample or not.\n",
    "\n",
    "Formally, we're testing the null hypothesis that the population distribution of the first sample is the same as that of the second sample (as opposed to the alternative that they're different, of course).\n",
    "\n",
    "The most basic two-sample hypothesis tests are all *parametric* and make three assumptions:\n",
    "\n",
    "1. Each measurement is independent.  That is, the second measurement in either condition isn't influenced by the first, the third isn't influenced by the second, and so forth.  This assumption is violated in cases such as time courses, in which earlier measurements do affect later ones, or when the study has hidden population structure (e.g. test scores from two classes, one of which is systematically worse than the other).\n",
    "\n",
    "2. Measurement error is normally distributed around the true value being measured.  Another way of putting this is that outliers must become exponentially less likely the further away from the truth they become.  If the true value being measured is 5, and there's a 10% chance you'll measure a value <4.5 or >5.5 instead, then measuring <4.0 or >6.0 should only happen 5% of the time, <3.5 or >6.5 only 2.5% of the time, <3.0 or >7.0 only 1.25%, and so forth (approximately!)\n",
    "\n",
    "3. Error is **homoscedastic**, or not related to the magnitude of the measured value.  This can be easily seen in a contrast between data where the amount of error does not vary across measured values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.040936748157811675, 0.051932181994832405, 0.0753995315925623, 0.09761556132188332, 0.1488759425234074, 0.1668980197817861, 0.19884835738706663, 0.20765651913319727, 0.2541739425633667, 0.259582324592182, 0.27029793112182743, 0.3310330542078995, 0.33489808342704896, 0.36366973193599506, 0.38733752678052946, 0.3912758878156951, 0.4116528216820404, 0.41420874083839676, 0.47175873805568236, 0.5205243562552788, 0.5436630706457013, 0.5505429931196748, 0.550577430927736, 0.5645582918225521, 0.578838265765926, 0.6191973216674506, 0.6218882796934061, 0.637505703598559, 0.6610959427594361, 0.6785647326850366, 0.6786460240286062, 0.6813452394301233, 0.7012621658026598, 0.7054478072398629, 0.7343068195751148, 0.7452809350933348, 0.750602427384029, 0.7646034156363205, 0.7716461803988746, 0.8071512432290431, 0.8240062851324698, 0.8246716742389592, 0.8396380698862975, 0.8418281122011415, 0.8854736432389898, 0.8916521296828607, 0.9043935853482324, 0.9048583329840483, 0.9064241067825863, 0.9936271248620155]\n"
     ]
    }
   ],
   "source": [
    "true_values = sorted( [random.uniform( 0, 1 ) for i in range( 50 )] )\n",
    "print( true_values )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.15918804294142197, 0.18484552782670863, 0.21466917376365965, 0.14603722315054618, 0.2771358305580812, 0.27632866581219934, 0.31497094986022844, 0.3735435338021377, 0.28485739793392056, 0.35467553944318764, 0.3512842161277792, 0.5153579002290106, 0.5017994480202732, 0.5020086143953065, 0.3952705444571169, 0.48743570117625834, 0.5379839103298547, 0.43878315328287354, 0.59038781945921, 0.715551900374702, 0.6324896016361368, 0.7180440703355019, 0.6286960783217613, 0.5958314372963598, 0.7675822819218192, 0.7762654596151326, 0.7014142210841725, 0.7639870601751715, 0.6835314101476883, 0.7911839050113697, 0.8047093908637846, 0.7975381638886854, 0.8204903282950603, 0.7711076694981773, 0.9176523901010042, 0.894006113207086, 0.9155329813898643, 0.8728961493206623, 0.8356917761676668, 0.9573549401799798, 0.8265251703088526, 0.8499534211390284, 0.8994790038435025, 0.9095373671529134, 0.954522542433447, 0.9686895756500031, 0.9413951469587846, 0.9362352464468138, 0.958336677639756, 1.1901795686592185]\n"
     ]
    }
   ],
   "source": [
    "homoscedastic_measurements = [( d + random.uniform( 0, 0.2 ) ) for d in true_values]\n",
    "print( homoscedastic_measurements )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x105b914e0>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEPCAYAAABY9lNGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2UXHWd5/H3N6Q7aQkdjZaICN3yNFEgJNGOUTlDdQgS\nnV1xddbQrDpiI7ARZDhnZsgc1qVdnSO46+BDBhOgRdQlYRV3jaAC0dQ4WQ00EGiFRCLYzTMU6mQM\n2yENfPePezuprr5Vdevh1kP353VOndSt+vW93/Tput/6PZu7IyIikm9WowMQEZHmpAQhIiKRlCBE\nRCSSEoSIiERSghARkUhKECIiEinRBGFmg2b2rJkNlyjXY2bjZvbBJOMREZH4kq5B3ACcWayAmc0C\nrgRuTzgWEREpQ6IJwt23AX8sUexi4HvAc0nGIiIi5WloH4SZvRH4gLt/HbBGxiIiIpM1upP6y8Bl\nOcdKEiIiTWJ2g6//dmCTmRnwOuC9Zjbu7pvzC5qZFo0SEamAu1f05bseNQijQM3A3Y8JH28m6IdY\nE5Uccsq37OOKK65oeAyKv/FxzMT4Wzn26RB/NRKtQZjZTUAaeK2ZPQZcAbQD7u7X5hVXDUFEpIkk\nmiDc/Zwyyn4iyVhERKQ8je6knjHS6XSjQ6iK4m+sVo6/lWOH1o+/GlZtG1W9mJm3SqwiIs3CzPAm\n7qQWEZEWpAQhIiKRlCBERCSSEoSIiERSghARkUhKECIiEkkJQkREIilBiIhIJCUIERGJpAQhIiKR\nlCBERCSSEoSIiERSghARkUhKECIiEkkJQkREIilBiIhIJCUIERGJpAQhIiKRlCBERCSSEoSIiERS\nghARkUiJJggzGzSzZ81suMD755jZA+Fjm5mdnGQ8IiISX9I1iBuAM4u8/yjw5+5+CvB54LqE4xER\nkZhmJ3lyd99mZl1F3t+ec7gdODLJeEREJL5m6oM4D/hxo4MQEQHIZrMMDQ2RzWYbHUrDJFqDiMvM\neoFzgVOLlRsYGDjwPJ1Ok06nE41LRGamjRtvpr9/De3t3ezfP8Lg4DX09a1udFixZDIZMplMTc5l\n7l6TExW8QNDE9EN3X1Tg/UXALcAqd3+kyHk86VhFRLLZLF1dCxkb2wosAobp6OhldHQXqVSq0eGV\nzcxwd6vkZ+vRxGThY+obZkcTJIePFksOIiL1MjIyQnt7N0FyAFhEW1sXIyMjjQuqQRJtYjKzm4A0\n8Fozewy4AmgH3N2vBT4DLACuMTMDxt19WZIxiYgU090dNCvBMBM1iPHxUbq7uxsaVyMk3sRUK2pi\nEpF6meiDaGvrYnx8tKX6IPJV08SkBCEiEiGbzTIyMkJ3d3dL9j1MUIIQEZFIzd5JLSIiLUgJQkRE\nIilBiIhIJCUIERGJpAQhIiKRlCBERCSSEoSIiERSghARkUhKECIiEkkJQkREIilBiIhIJCUIERGJ\npAQhIiKRlCBERCSSEoSIiERSghARkUhKECIiEkkJQkREIilBiIhIJCUIERGJpAQhIiKREk0QZjZo\nZs+a2XCRMl81s91mdr+ZLU4yHhERiS/pGsQNwJmF3jSz9wLHuvvxwAXA+oTjERGRmBJNEO6+Dfhj\nkSJnAd8Ky94FzDezw5OMSURE4ml0H8SRwOM5x0+Gr4mISIPNbnQA5RgYGDjwPJ1Ok06nGxaLiEgz\nymQyZDKZmpzL3L0mJyp4AbMu4IfuvijivfXAVne/OTzeBZzm7s9GlPWkYxURmW7MDHe3Sn62Hk1M\nFj6ibAY+BmBmy4F/jUoOIiK5stksQ0NDZLPZRocyrSU9zPUm4BfACWb2mJmda2YXmNn5AO7+I+B3\nZvZbYAOwJsl4RKT1bdx4M11dCznjjAvp6lrIxo03NzqkaSvxJqZaUROTSOvKZrOMjIzQ3d1NKpWq\n6jxdXQsZG9sKLAKG6ejo5d57t7F3796qzz8dNXsTk4jMYLX8xj8yMsLs2V0EyQFgEe7zWbLkXapR\nJEA1CBFJTKFv/KOjuyr6pr9hw3VceOElwPbwfBngfTnH1Z1/OlINQkSa0sjICO3t3eR+429r62Jk\nZKTsc2WzWS69dC0wAPQCpwCrmDv3mJqcX6ZSghCRxHR3d7N//wgwsRzbMOPjo3R3d5d9roPJ5u+A\nXcD1tLe/jn37Hq3J+WUqJQgRSUwqlWJw8Bo6Onrp7FxKR0cvg4PXVNT8MznZpIAX2L//eYLb2DuB\n44DlXH31lWpeqpHYfRBm9ip3/38Jx1Ps+uqDEGmAWoxAqtUopo0bb6a/fw1tbV3s27eb/ftfAX4J\nHAHcCZzH7bf/H97znvdUfI3pJtE+CDN7l5k9RFCnw8xOMbNrKrmYiLSWqBFIlUxSS6VS9PT0VP3N\nvq9vNaOju9iyZQPf/vYg8EaC/ocUcE54LLVSsgZhZncBfwlsdvcl4Wu/dveT6hBfbhyqQYjUUdQI\npLa2U5k9u4329qC5Z3DwGvr6VtfkWuXWMLLZLG960/Hs3//zA/G1t/85TzyxW01MORIfxeTuj+e9\n9HIlFxOR1jF1BNIRjI+/zNjYVvbsuZexsa3096+permLiVpKb+8ny5rHkEql+OY3N9DR0cuhh55C\nR0cv3/zmBiWHGopTg/ge8I/AOuAdwCXA29397OTDmxSHahAidTS1BrEe+BKw+0CZzs6lbNmygZ6e\nnoqvUW0toFb9G9NV0jWIC4FPEezT8CSwODwWkWksdwRSMNfgUuApajmkdMeOHezfnyJ3HsP+/a9j\nx44dZcVZi/4NmarkfhDu/jzwn+oQi4g0mb6+1SxevIglS94F3AXsBNLAAjo6/ljxkNXJJpLOovDf\np6s8n9RKyQRhZjcAU9p23P0TiUQkIk1l7969zJ17DC++uIjgJr6CQw89le9/f2PVw0mXLFlCW9ss\nxsfTQDcwQlvbLJYsWVJ13FK9OE1MtwK3hY+fAp3A3iSDEpHmMXU29NO88srzLFmypOp9GVKpFDfe\neD1z5zqHHvoCc+c6N954vZqLmkTZi/WZ2Sxgm7u/K5mQCl5XndQiDZI7QW18fJTBwWAqVH//mpoM\neVVHc3Kq6aSuJEH8GXCbux9XyQUrpQQh0li5N3Ggpqu0SnKqSRBx+iD+RNAHYeG/zwCXVXIxEWld\nqVTqwM1/aGiI9vZuxsamrqIaJ0GoxtAaSvZBuPth7t6Z8+8J7n5LPYITkeZUzSqt2jK0dRRsYjKz\npcV+0N3vSySiAtTEJNJcovolSvVB1HoDISktqSamLxV5z4EVlVxQRKaHvr7VrFy5gpGREebNm8fe\nvXvJZrNFb/QTy3dU2jQl9VUwQbh7bz0DEZHWk0ql2LLlZ7FHM01umgpqENrgp3nFGsVkZicBbwXm\nTrzm7t9KMK6oGNTEJJKAajqMK2kyuvjiS1i37jrgTcATXHTRJ/na175S7X9DCkh6P4grgK+Fj17g\ni8D7K7mYiDSXajuMy91zOpvNMjj4HeBHwP8EfsTg4HeqXhFWkhFnJvVfAqcDz7j7uQQ7hc+PewEz\nW2Vmu8zsYTObMjzWzDrNbLOZ3W9mvzKzj8c9t4hULpvN0t+/pqrlu8sdzXQwoaSBHiBdNKFIY8VJ\nEGPu/grwkpl1As8BR8U5eTjreh1wJnAi0GdmC/OKfQp40N0XE9RQvmRmJedniEh58pfFKPfbf5Ry\n95yuZnis1F+cBHGPmb0auA64F7iPYBPYOJYBu9191N3HgU3AWXllHDgsfH4Y8Ht3fynm+UUkhomm\npNNP7+eoo45jw4branazzt0GdHR0V9GhruUmFGmsspbaMLNuoNPdh0sUnSj/IeBMdz8/PP4IsMzd\nP51TZh6wGVgIzANWu/uPI86lTmqRChzsSL4MuIqgAeBh1q//Cp2dnWXPZahVTJpJXR9JL7WxmeCb\n/w/cfaSSi5RwJrDD3VeY2bHAnWa2yN2nrBg7MDBw4Hk6nSadTicQjsj0MjIygvvhBMnh4GijSy45\njccff5jR0V11v1nnLtshtZXJZMhkMjU5V5wtR08DVgN/AQwRJItb3X1fyZObLQcG3H1VeLwWcHe/\nKqfMrcAX3P3/hsc/BS5z93vyzqUahAjlf/veuXMnb33rEoJK+v0HXp83bzE/+9l1FW8XKq0h0WGu\n7v7P7r4GOAbYAHyYoKM6jiHgODPrMrN24GyC5qRco8BKADM7HDgBeDTm+UWmjTh7K1QyLDXY8Odw\n4GEK9TdUu6+DTFPuXvIBdBAkhluA3wFfi/Nz4c+uAn5DsNP52vC1C4Dzw+dHALcT/OUOA30FzuMi\n09VNN23yjo4FPn/+Uu/oWOA33bRpSpnnnnvOOzoWODzg4A4PeEfHAn/ooYf87rvv9ueeey7y3Ad/\n7jMOr3Y42aHD16+/Nva1pXWF985Y9+v8R5wb/P8CRoD1BMNQZ1V6sWoeShAyXRW68eff8O+++26f\nP39pWCZ4zJ37Zp8z59Ulb+4TSWDevJN8zpzOA8kh7rWldVWTIOLMNxgMv9W/XGVlRUQixF3Abuo6\nRhn27XsG2B7uFz1Mf38vK1eumNI3kbuwXm7fhRbPk2Li9EHcruQgkpy48xHy5xDMmXMWHR3HEXei\nWyqVoqenp0jSKXxtmZniTJQTkQSVM3ksd1Lajh3bgSep5uauiWtSTNl7UjeKhrnKdFfJ5LFKNu2p\n1bWlNVQzzFU7yom0ON3cpZikEsTW8Olc4O3AA4ARNHje4+7vrOSClVKCkOlCN3Spp0Qmyrl7rwe7\nyj0NLHX3t7v724AlBA2fIlJAoYln1e6/IFJPcZbaeNDdTyz1WtJUg5BWMdEvkL8FZyW7r4lUK9Gl\nNoBhM7vezNLh4zoODpsQkRzFNuGpxf4LudfR0hiStDgJ4lzgQeCS8PFQ+JqI5CmWBGo150DNVFIv\nsYa5mlkHcLS7/yb5kArGoCYmaXqlmpGqHZaqZiopV6JNTGb2foI1gn8SHi8O94gQkTylJp6Vs/ta\nlHKaqdQMJdWK00l9L7ACyLj7kvC1X7n7yXWILzcO1SCkZSQ1lDVuDaJQR7nMPInMg8g5+XZ3X25m\nO3ISxLC7Lyr6gzWmBCESKNVMpWYoyZXolqPAg2Z2DnCImR0PfBr4RSUXE5HqFVqZdYJWaJVaiVOD\neBVwOfCe8KXbgc97jC1Ha0k1CJF4VIOQXInVIMzsEOC/ufvfECQJEWlyEx3l/f29k5qhlBykXLH7\nIOoUT7E4VIMQKYPWfBJIvpP668CRwHeBFyZed/fvV3LBSilBiIiUL+lO6rnA7wmGuk5woK4JQqTV\n6Ru9tBptGCRSB7nzEl588VEuv/xvueCCTypRSOKSbmK6gaDGMIm7f6KSC1ZKCUJa1eRRRTuB/wy8\nlo6OP2gCmyQu6dVcbwVuCx8/BTqBvZVcTGQmOrg8xhHAGiAD7J600qtIMyrZB+Hut+Qem9lGYFvc\nC5jZKuDLBMlo0N2viiiTBq4G2oBsuFGRyLRwcBXXO4FuotZRUlOTNKM4NYh8xwOvj1PQzGYB64Az\ngROBPjNbmFdmPvBPwL9z95OA/1hBTCJNa2Jewty5nwJ2Ue1y3yL1Emc11z+Z2b9NPIAfApfFPP8y\nYLe7j7r7OLAJOCuvzDnALe7+JIC7Px8/fJHW0Ne3mscee5jPfe7ySSu9Xn31lYyMjKiZSZpSoqOY\nzOxDwJnufn54/BFgmbt/OqfMRNPSicA84Kvu/u2Ic6mTWhqmlkNUJ8513333c+mla7XiqiQq0XkQ\nZvZu4H53fyG8wS8FvuLuo5VcsEAMSwnmWRwK/NLMfunuv80vODAwcOB5Op0mnU7XKASRwmq9dPZE\ngjnttFWMjW0NF9Ubpr+/l5UrV6g/QqqSyWTIZDI1OVecYa7DwCkEPWvfBK4HPuzup5U8udlyYMDd\nV4XHawHP7ag2s8uAue7+2fD4euDHEZ3jqkFI3SW18N3Q0BBnnHEhe/bce+C1zs6lbNmygZ6enuoD\nFwklPcz1pfDOfBawzt3/CTgs5vmHgOPMrMvM2oGzgfzd6H4AnGpmh4Qrx76DYLC4SMOVs4NbOWq1\nP7VIkuIkiD+Z2d8DHwFuC0cmtcU5ubu/DFwE3AE8CGxy951mdoGZnR+W2UWwhPgwsB241t0fKv+/\nIlJ7Sd3IS21NKtIM4jQxvYFgpNGQu/+LmR0NpN39W/UIMCcONTFJQ5Tawa0aWp9JkpboUhvNQglC\nGkk3cmlVSa/FtBz4GvAWoB04BNjr7vMruWCllCBERMqXdCf1OqAP2A10AOcB11RyMZFGymazDA0N\naVKaSEyxltoI5yQc4u4vu/sNwKpkwxKprY0bb6arayFnnHEhXV0L2bjx5kaHJNL04jQx/RxYSTD/\n4RngaeDj7n5K8uFNikNNTFKRpOYyiLSCpJuYPhqWu4hgy9GjgA9VcjGRRkhqLoPIdBdnue9RM+sA\njpiY7SxSL7UYPTR5LkNQg9CkNJHS4qzm+u+B+4GfhMeLzSx/NrRIzdWq30CT0kQqE6cP4l6ChfQy\n7r4kfO1X7n5yHeLLjUN9EDPI5H6DI4A7mTv3Uzz22MOTbuzl1DA0l0FmoqT7IMbdfU/ea7pTS6IO\n9hvsBBYCX2Lfvv1s2HDdgTLl1jBSqRQ9PT1KDiIxxalBDBLsRb2WoHP600Cbu1+YfHiT4lANYgbJ\nZrMcffQJ7NtnBHs4Tx59BJQcmaQag0jyNYiLCTbzeRHYCPwb8NeVXEwkrlQqxeWX/y3wWqJGH5Ua\nmaR5DyLV01pM0rSKzV+AwjWIYu+pJiEzTSI7ypUaqeTu76/kgiJxTYw+6u/vnbSS6sRNvtB7Q0ND\ntLd3hzu1QW7tQglCJL6CNQgzywKPEzQr3QVMykDu/s+JRzc5HtUgZqhifQlR72nmtMhBiazmamaH\nAGcQLNS3CLgN2OjuD1YaaDWUIKQcSe7hINJKEt8PwszmECSK/w581t3XVXKxaihBSLk0ikkkwQQR\nJoa/IEgO3QT7SX/D3Z+s5GLVUIIQESlfUk1M3wJOAn5EsJf0rysPsXpKEFILqlXITJNUgniFYPVW\nmDxz2gB3985KLlgpJQip1kS/RHt7sHif+iVkJtCe1CIlaGSTzFRJz6QWaXnaE0KkfEoQMiNM3hMC\ntCeESGmJJwgzW2Vmu8zsYTO7rEi5HjMbN7MPJh2TzDzaE0KkfIn2QZjZLOBh4HTgKWAIONvdd0WU\nuxMYIxhG+/2Ic6kPQqqmUUwy0ySyFlONLAN2u/sogJltAs4CduWVuxj4HtCTcDwyQxRKBKlUSolB\nJKakm5iOJFjPacIT4WsHmNkbgQ+4+9fJW+9JpBJa6lukNpKuQcTxZSC3b6JgkhgYGDjwPJ1Ok06n\nEwtKWlM2m6W/fw1jY1vD1VyH6e/vZeXKFao5yIyQyWTIZDI1OVfSfRDLgQF3XxUeryWYZHdVTplH\nJ54CryOYnHe+u2/OO5f6IBqsFdrvh4aGOOOMC9mz594Dr3V2LmXLlg309KgFU2aeZp4HMQQcZ2Zd\nZtYOnE2wntMB7n5M+HgzQT/EmvzkII3XKs02Gs4qUjuJJgh3fxm4CLgDeJBgTaedZnaBmZ0f9SNJ\nxiOVyW222bPnXsbGttLfv4ZsNtvo0KbQcFaR2tFSG1JSpc02jWySaoXmMJF6aOZhrjINTG62CTp+\nSzXb5C+Md/XVV7J06eK63bA1nFWkeqpBSCzl7NA2dWG8LwIDHHbYQl56Sbu7idSTVnOVuojbbDO5\nSSoLLAS0iqpII6iJSeoibrPN5CapF4GjiFpFVQlCpLlpNVepudyRRPPmfYJgOS4NOxVpNWpiksRM\nNEndd9/9XHrp2lj9FyJSW+qDkETVYsiohp2KNIYShCRG+ziLtDYlCEmE9nEWaX3NvBaTtDDt4ywy\nsylBSEHVLHyXzWYZGhpqyvWaRCQeJQgpqNKF71pl5VcRKU59EDNQ/oiiUiOMyhmBpH4LkeaiPgiJ\nLf/b/cUX/3XJb/upVIqenp5YN3j1W4hMH6pBzCBTv91ngPcB26nVt33VIESai2oQEsvUb/eHUmid\npEppwx6R6UM1iBmkHjWI3Gtp5rRI46kGIbFM/Xb/IS666JOJfNsvp99CRJqTahAzULmjmESkdWmp\nDRERiaQmJhERqTklCBERiaQE0UK0vpGI1FPiCcLMVpnZLjN72Mwui3j/HDN7IHxsM7OTk46pFWl9\nIxGpt0Q7qc1sFsGGxKcDTwFDwNnuviunzHJgp7vvMbNVwIC7L48414ztpNbsZBGpVDN3Ui8Ddrv7\nqLuPA5uAs3ILuPt2d98THm4Hjkw4ppaj9Y1EpBGSThBHAo/nHD9B8QRwHvDjRCNqAfl9DdXsyyAi\nUqnZjQ5ggpn1AucCpxYqMzAwcOB5Op0mnU4nHle9FdoDenDwGvr7e2lr62J8fFTrG4lIpEwmQyaT\nqcm5ku6DWE7Qp7AqPF4LuLtflVduEXALsMrdHylwrmnfB1Gqr0EznkWkXNX0QSRdgxgCjjOzLuBp\n4GygL7eAmR1NkBw+Wig5zBQTfQ1jY1P7GlKp1IGHiEg9JJog3P1lM7sIuIOgv2PQ3Xea2QXB234t\n8BlgAXCNmRkw7u7LkoyrWc2bN499+x4l6GsIahDqaxCRRtFaTE1iou8B5jM29gwdHccCTx3ogxAR\nqYQW62txUfs0zJlzFjt2bOctb3lLo8MTkRbWzPMgJIap8xzSzJlzLHv37i35s1p+Q0SSogTRBCqd\n56DlN0QkSWpiahITfRC58xyK9T1o+Q0RiaOZh7lKTH19q1m5ckXseQ6lhsSKiFRLCaKJlDPPYXKz\nlIbEikjtqQ+iRaVSKQYHr6Gjo5fOzqV0dPRq+Q0RqSn1QbQ4Lb8hIsVoHoSIiETSPAgREak5JQgR\nEYmkBCEiIpGUIEREJJIShIiIRFKCEBGRSEoQIiISSQlCREQiKUGIiEgkJQgREYmkBCEiIpGUIERE\nJJIShIiIREo8QZjZKjPbZWYPm9llBcp81cx2m9n9ZrY46ZhERKS0RBOEmc0C1gFnAicCfWa2MK/M\ne4Fj3f144AJgfZIxNUomk2l0CFVR/I3VyvG3cuzQ+vFXI+kaxDJgt7uPuvs4sAk4K6/MWcC3ANz9\nLmC+mR2ecFx11+p/ZIq/sVo5/laOHVo//moknSCOBB7POX4ifK1YmScjyoiISJ2pk1pERCIluuWo\nmS0HBtx9VXi8FnB3vyqnzHpgq7vfHB7vAk5z92fzzqX9RkVEKlDplqOzax1IniHgODPrAp4Gzgb6\n8spsBj4F3BwmlH/NTw5Q+X9QREQqk2iCcPeXzewi4A6C5qxBd99pZhcEb/u17v4jM3ufmf0WeAE4\nN8mYREQknkSbmEREpHU1bSe1mb3GzO4ws9+Y2e1mNj+izJvM7Gdm9qCZ/crMPt2IWPNiatmJgaVi\nN7NzzOyB8LHNzE5uRJyFxPndh+V6zGzczD5Yz/hKifm3kzazHWb2azPbWu8Yi4nx99NpZpvDv/tf\nmdnHGxBmJDMbNLNnzWy4SJmm/NxC6fgr/uy6e1M+gKuAvwufXwZcGVHmDcDi8Pk84DfAwgbGPAv4\nLdAFtAH358cDvBe4LXz+DmB7o3/XZcS+HJgfPl/VLLHHjT+n3E+BW4EPNjruMn//84EHgSPD49c1\nOu4y4/974AsTsQO/B2Y3OvYwnlOBxcBwgfeb8nNbRvwVfXabtgZBMIHuxvD5jcAH8gu4+zPufn/4\nfC+wk8bOoWjliYElY3f37e6+JzzcTnPNV4nzuwe4GPge8Fw9g4shTvznALe4+5MA7v58nWMsJk78\nDhwWPj8M+L27v1THGAty923AH4sUadbPLVA6/ko/u82cIF7v4Wgmd38GeH2xwmbWTZBB70o8ssJa\neWJgnNhznQf8ONGIylMyfjN7I/ABd/860Gyj4uL8/k8AFpjZVjMbMrOP1i260uLEvw54q5k9BTwA\nXFKn2GqhWT+3lYj92U16mGtRZnYnkJuFjeBbxn+JKF6wN93M5hF8K7wkrElIgsysl2C02amNjqVM\nXyZorpzQbEmilNnAUmAFcCjwSzP7pbv/trFhxXYmsMPdV5jZscCdZrZIn9n6Kfez29AE4e5nFHov\n7HA53N2fNbM3UKBJwMxmEySHb7v7DxIKNa4ngaNzjt8UvpZf5qgSZRohTuyY2SLgWmCVuxerktdb\nnPjfDmwyMyNoA3+vmY27++Y6xVhMnPifAJ53933APjP7OXAKQdt/o8WJ/1zgCwDu/oiZ/Q5YCNxT\nlwir06yf29gq+ew2cxPTZuDj4fO/Agrd/L8BPOTuX6lHUCUcmBhoZu0EEwPzbz6bgY/BgZnmkRMD\nG6Bk7GZ2NHAL8FF3f6QBMRZTMn53PyZ8vJngS8WaJkkOEO9v5wfAqWZ2iJm9iqCzdGed4ywkTvyj\nwEqAsP3+BODRukZZnFG4Vtmsn9tcBeOv+LPb6N73Ir3yC4AtBCOT7gBeHb5+BHBr+PzdwMsEIyZ2\nAPcRZMdGxr0qjHk3sDZ87QLg/Jwy6wi+9T0ALG307zpu7MB1BCNP7gt/33c3OuZyf/c5Zb9BE41i\nKuNv528IRjINAxc3OuYy/36OAG4PYx8G+hodc07sNwFPAS8CjxHUdlricxsn/ko/u5ooJyIikZq5\niUlERBpICUJERCIpQYiISCQlCBERiaQEISIikZQgREQkkhKETFtmtiBcGvs+M3vazJ7IOW7YKgJm\ndrqZ/e9GXV8kroYutSGSJHf/A7AEwMz+K7DX3f8xv5yZmdd/QpAmIEnTUw1CZooDSxCY2bHhJlPf\nMbNfA0eZ2R9z3l9tZteFz19vZreY2d1mtt3Mlk05cbCy6vE5x/9iZovM7B1m9gszuzd87diIn/2c\n5Wx0ZWY7w1VnMbOPmdldYY1nXc1+EyIxKUHITPVnwJfc/SSCRdfyv9FPHH8VuMrdlwGrgcGIc20K\n38PMjgRe4+7DwEPAqe7+NuDzwD/EiMvD85wI/Afgne6+FGgzs7PL+P+JVE1NTDJTPeLuO2KUWwmc\nEK4AC8FGMXPc/cWcMt8lWMzt8wSJ4rvh668Bvp1TcyinWWklweqz94TXnkuwxo5I3ShByEz1Qs7z\nV5hcm54AjTDyAAABBklEQVSbV7bH3V8udCJ3f8zM9prZWwgSxF+Fb/0D8BN3Xx8miahNWl7Ku3ZH\n+K8B33D3K0r/V0SSoSYmmakO9EmEHdR/CPsmZhE07UzYQrBNafBDZqcUON/NBHsut7v7rvC1Tg7u\nGXBugZ8bAd4WnnsZB/cc2AJ82MxeG763wMyOijyDSEKUIGSmym/uWUuwrPw2Jm8teRHwbjN7IOzQ\nPq/A+b4H9BEkiglfBP6Hmd0Tcb0J3wXeYGbD4bkfAXD3XwOfBbaY2QMEy2QX3XZXpNa03LeIiERS\nDUJERCIpQYiISCQlCBERiaQEISIikZQgREQkkhKEiIhEUoIQEZFIShAiIhLp/wMEGSG6ETixmwAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105e527b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "scatter( true_values, homoscedastic_measurements )\n",
    "xlabel( \"True value\" )\n",
    "ylabel( \"Measured value\" )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "versus measurement data where the amount of error does vary proportionally to the underlying, true value itself:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.09623829648444468, 0.058775464102589055, 0.14145072131329206, 0.2904673294012996, 0.19934668377156464, 0.316236302292913, 0.43430318037069393, 0.4168172761001913, 0.3451729537322144, 0.5684365688870423, 0.6359851147783534, 0.7610690406779221, 0.7676811842179918, 1.0561337768240802, 0.5991351320488052, 0.6602746022700587, 0.8465389237756398, 0.9415900576415738, 1.2834564709402057, 1.0105026546652311, 1.1680929815278585, 0.8106203626633012, 0.670900185237046, 1.4861458299456554, 0.974365828990696, 1.62045610309969, 1.847120398375771, 0.9563481886359584, 1.4629314883336166, 1.7035921920443347, 1.6173133544014267, 1.7124929890540779, 0.7534003481786462, 0.735911818312324, 1.6580003622924513, 1.4016279690421625, 1.9491269462987937, 1.428524359463613, 1.1502880956051094, 2.3451207819680486, 1.1727714552038373, 2.212693969403356, 1.709436254316034, 1.3654227187118526, 1.4829514670768136, 1.044883828785077, 1.3915710615393249, 2.518694622453876, 1.898361268662552, 1.399754948609797]\n"
     ]
    }
   ],
   "source": [
    "# Note the one, small, critical difference in the following line:\n",
    "heteroscedastic_measurements = [( d + random.uniform( 0, 2 * d ) ) for d in true_values]\n",
    "print( heteroscedastic_measurements )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x105bf0048>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuUXGWZ7/HvD0KgFZMxTBs5XBIHQYIKJJgmCGvoIGBw\nOTKDLiHMCGLLwEAw6pkjOLom4cjM6Jw543BZHC4nMjAOId5F8YJoWg4q6SYQiNARcEy4KNCKRIGg\nQZ7zx96dVDpd3VW7atfeVf37rFUrddlV++nqVD39vs97UURgZmZWr12KDsDMzNqTE4iZmWXiBGJm\nZpk4gZiZWSZOIGZmlokTiJmZZVJoApG0u6Q1ku6RtF7SsirHXSbpIUnrJB3e6jjNzGxnU4o8eUT8\nTtLCiHhe0q7ADyR9MyIGRo6RdBJwQEQcKOlI4CpgQVExm5lZovAurIh4Pr26O0lCGz2z8WTghvTY\nNcB0STNbF6GZmY2l8AQiaRdJ9wBPAN+JiMFRh+wDPFpx+/H0PjMzK1DhCSQiXoqIucC+wJGSDik6\nJjMzm1ihNZBKEfEbSauBRcADFQ89DuxXcXvf9L6dSPLCXmZmdYoIZXle0aOw/ljS9PR6F3ACsGHU\nYTcDZ6THLACeiYgnq71mRLTlZdmyZYXH4PiLj8Pxt+elneNvRNEtkL2B6yXtQpLMVkXENySdA0RE\nXJPefpukh4HngLOKDNjMzBJFD+NdD8wb4/6rR91e0rKgzMysJoUX0S3R29tbdAgNcfzFcvzFavf4\ns1KjfWBlIik66ecxM8ubJKIdi+hmZta+nEDMzCwTJxAzM8vECcTMzDJxAjEzs0ycQMzMLBMnEDMz\ny8QJxMzMMnECMTOzTJxAzMwsEycQMzPLxAnEzMwycQIxM7NMnEDMzCwTJxAzM8vECcTMzDJxAjEz\ns0ycQMzMLBMnEDMzy8QJxMzMMnECMTOzTApNIJL2lfQ9SfdLWi/pA2Mcc6ykZyTdnV4+XkSsZta+\nhoeHGRwcZHh4uOhQOkrRLZAXgQ9HxOuBo4DzJR08xnG3R8S89HJJa0M0s3a2cuUqZs06mBNOOJdZ\nsw5m5cpVRYfUMRQRRcewjaSvAJdHxHcr7jsW+NuI+LManh9l+nnMrFjDw8PMmnUwW7asBg4F7qOr\nayGbNm2gu7u76PBKQRIRoSzPLboFso2k2cDhwJoxHj5K0jpJt0g6pKWBmVnb2rhxI1OnziZJHgCH\nsttus9i4cWNxQXWQKUUHACBpT+ALwNKIeHbUw2uB/SPieUknAV8BDqr2WsuXL992vbe3l97e3qbH\na2btYfbs2fz+9xuB+xhpgWzduonZs2cXGleR+vv76e/vb8prFd6FJWkK8HXgmxFxaQ3H/ww4IiKe\nHuMxd2GZ2Q5WrlxFX9957LbbLLZu3cSKFVeyePGpRYdVGo10YZUhgdwA/DIiPlzl8ZkR8WR6vQf4\nXETMrnKsE4iZ7WR4eJiNGzcye/Zs1z5GadsEIulo4HZgPRDp5e+AWUBExDWSzgf+BtgKbAE+FBFj\n1UmcQMzM6tS2CaTZnEDMzOrTEaOwzMysvTiBmJlZJk4gZlY4LzXSnpxAzKxQXmqkfbmIbmaF8VIj\nxXMR3czakpcaaW9OIGZWmB2XGgEvNdJenEDMrDDd3d2sWHElXV0LmTZtHl1dC1mx4kp3X7UJ10DM\nrHBeaqQ4nomecgIxM6uPi+hmZtZyTiBmVkqeXFh+TiBmVjqeXNgeXAMxs1Lx5MLWcg3EzDqGJxe2\nDycQMysVTy5sH04gZlYqnlzYPlwDMbNS8uTC1vBEwpQTiJlZfVxENzOzlnMCMTOzTJxAzMwsEycQ\nMzPLpNAEImlfSd+TdL+k9ZI+UOW4yyQ9JGmdpMNbHaeZme1sSsHnfxH4cESsk7QnsFbSrRGxYeQA\nSScBB0TEgZKOBK4CFhQUr5mZpQptgUTEExGxLr3+LDAE7DPqsJOBG9Jj1gDTJc1saaBmZraT0tRA\nJM0GDgfWjHpoH+DRituPs3OSMTOzFiu6CwuAtPvqC8DStCWS2fLly7dd7+3tpbe3t6HYzMw6SX9/\nP/39/U15rcJnokuaAnwd+GZEXDrG41cBqyNiVXp7A3BsRDw5xrGeiW5WMC9B0l7afSb6Z4AHxkoe\nqZuBMwAkLQCeGSt5mFnxvBHU5FJoC0TS0cDtwHog0svfAbOAiIhr0uOuABYBzwFnRcTdVV7PLRCz\ngngjqPbUSAuk0BpIRPwA2LWG45a0IBwza8DIRlBbtuy8EVTRCcTdavkoQxeWmXWAsm4E5W61/BRe\nRG8md2GZFWvlylX09Z3HbrvNYuvWTaxYcSWLF59aWDzuVptY23ZhmVk+iuqyWbz4VI4//rjSdBeV\nuVutE7gLy6zDFN1l093dzfz580vxBV3WbrVO4S4ssw7iLpudla1brWxa0oUl6WUR8XyWk5hZa7jL\nZmdl61brJBN2YUl6s6QHgA3p7cMkXZl7ZGZWN3fZjK1M3WqdpJYayKeBtwK/AoiIe4E/zTMoM8um\nu7ubFSuupKtrIdOmzaOrayErVlzpL07LxYQ1EElrIuJISfdExNz0vnsj4rCWRFgH10DMEp44Z7XK\nuwbyqKQ3AyFpN2Apyb4dZlZS3d3dThyWu1q6sM4FzifZg+Nxkj07zs8zKDMzKz8P4zWzbYaGhhgY\nGKCnp4c5c+YUHc427pLLT65dWJKuI1kldwcR8b4sJzSzcrrggg9yxRXXAPsBj7Jkydlcfnm1XRZa\nZ2Qex9SpyQgzz+Moj1qK6O+suLkH8BfAzyPiA3kGloVbIGbZDA0NccghRwB3MjIBERbwwANrC22J\neGJk/nJtgUTEF0edbCVwR5aTmVk5DQwMkLQ8tk9AhH0ZGBiYMIHk2b3kiZHllmUtrAOBVzU7EDMr\nTk9PD/AolRMQ4bH0/uryXnfLEyPLrZaZ6L+V9JuRf4GvARfmH5qZtcqcOXNYsuRsYAFwELCAJUvO\nHrf1MTw8TF/feWzZsprNm9eyZctq+vrOY3h4uGlxeWJkuXkUllmHaaRL6Qc/+AG33norJ554Ikcf\nffS4xw4ODnLCCeeyefPabfdNmzaP2267mvnz52eKvRqPwspPIzWQqglE0rzxnlhtX/IiOYHYZNfI\niKV6n+sCd2fIK4GsHud5ERHHZTlhnpxAbDJr5As963O9VHr7y2UUVkQszB6SmbVaIyOWsj7XS6VP\nbjXtByLpDcAhJPNAAIiIG/IKyszqt+OIpaQVUeuIpUae63W3Jq9aRmEtAy5PLwuBfwbekXNcZlan\nRkYsebSTZVHLTPT1wGHAPRFxmKSZwGcj4oSmBCCtAN4OPBkRh47x+LHAV4H/Su/6UkRcUuW1XAOx\nSa+REUse7TT55L2c+5aIeEnSi5KmAU+RTFltlutIWjfjdYndHhFu9ZjVoJEuJXdHWT1qmYl+l6Q/\nAq4F1gJ3Az9qVgARcQfw6wkOy5Qdzdrd8PAwg4ODTZ2c1+n8nrXOhAkkIs6LiGci4irgBODMiDgr\n/9B2cJSkdZJukXRIi89tVoi8lwnpRH7PWquWGsjNwE3AVyPiuVyCkGYBX6tSA9kTeCkinpd0EnBp\nRBxU5XVi2bJl22739vbS29ubR8hmuWrFJL1Oq3d4YmNt+vv76e/v33b74osvzlwDISLGvQDHAlcC\nm4AvAO8C9pjoefVcgFnAfTUe+zNgRpXHwqwTDAwMxPTp8wJi22XatLkxMDDQlNe/8caboqtrRkyf\nPi+6umbEjTfe1JTXLVLe71mnSr83M31319KF9f2IOA/4E+Bq4N0khfRmElXqHOmor5HrPSStpqeb\nfH6zUslzFdpWLIJYea5W1SO8cm/r1bScu6Qu4J0k+6PPB65vVgCSbgR+CBwk6RFJZ0k6R9Jfp4e8\nS9KPJd0D/BvgdRKs4+U5L2Nk1nnl3h8js86bqdX1CM9lab1aaiCfA3qAbwGrgO9HxEstiK1ungdi\nnSaPOkWetYKRePfcc0+OOOKYQuoRY71neb2PnVBDamQeSC01h7cCu2btI2vlBddAzGoyUgOZNm1u\n02oglXWV3XefFl1dbyxFPSKPek8n1ZBooAbi/UDMJqlm/gW9c6umH3gblXusN9oCyRJvHq2tThvt\n1UgLJMuWtmbWAbq7u5k/f35OdZVe9thjJrvvfmxT6hFZ6yl51HtaVUNqB26BmFnDqv1VvnbtHTz7\n7LMNtXKK2Oek1a9ZpFzWwmrHHQnNrBgjI6D6+hbusLnUeHuq16qRfU6qxdXIF30er9muatmRcA/g\nTcC9JHM1DgXuioijWhJhHdwCMStWWUeNeRRWdblsaVvx4l8ClkXE+vT2G4DlEfGuLCfMkxOIFaVT\nvkzKylvn5ifvBHJ/RLx+ovvKwAnEijDy5TZ1ajIT2l9u+XCSzkfeCWQl8Bzw2fSuvwT2jIjFWU6Y\nJycQa7VOK6ja5JP3MN6zgPuBpenlgfQ+s0nPQzptMptwR8KIeEHSVcA3IuInLYjJrG3suIBf0gLx\nAn42WUzYApH0DmAdyVpYSDo83SPEbNLzAn42mdVSA1kLHAf0R8Tc9L71EfHGFsRXF9dArCgu8Fq7\nymUiYYWtEbFZ2uH1/S1tVqG7u7tpK9k6Cfm9aBe1FNHvl3Q6sKukAyVdTrJ/h5k1iffy3s7vRfuo\npQvrZcDHgBPTu74NXBIRL+QcW93chWXtaDIMBa61RTEZ3ouyyW0Yr6Rdgf8ZER+LiPnp5eNlTB5m\n7arThwLX06Lo9Pei04ybQCLiD8AxLYrFbFLq5L28691/vZPfi05USw3kHkk3S3qPpFNGLrlHZjZJ\ndPJQ4HpbFJ38XnSiWmog141xd0TE+/IJKTvXQKyddeLIo6w1jU58L8oq17Ww2okTiFn5eCXdcst7\nMcXrGGPeh1sg1g78l2w5FPF78O++Nnkvpvh14Jb08l1gGvBslpOZtZLnE5RHM/dfr4V/961RdxeW\npF2AOyLizU0JQFoBvB14MiIOrXLMZcBJJMvKvzci1lU5zi0QA8o/n8B/Heen7L/7ssm7BTLagcCr\nspysiuuAt1Z7UNJJwAERcSBwDnBVE89tHarM8wn813G+yvy77zS1rMb7W0m/GbkAXwMubFYAEXEH\n8OtxDjkZuCE9dg0wXdLMZp3fOlNZ5xPUOy/C6lfW330nmjCBRMQrImJaxeWgiPhiK4JL7QM8WnH7\n8fQ+s6rKOp+g0b+Oh4eHGRwcdMIZR1l/951owtV4JR0NrIuI5yT9FTAPuDQiNuUeXQbLly/fdr23\nt5fe3t7CYrFiLV58Kscff1ypag2NbEDVyN7rk63mUsbffVn09/fT39/fnBeLiHEvJP/TBRwG3AOc\nD3x/oufVcwFmAfdVeewq4NSK2xuAmVWODbNGPPXUUzEwMBBPPfVUbue48caboqtrRkybNje6umbE\njTfeVFNcXV0zAu4NiIB7o6trRk1xjpxv+vR5NZ/PJo/0ezPbd/eEB8Dd6b9/D/RV3tesCzAbWF/l\nsbcBt6TXFwB3jvM6zXtXbdJp5RdtvYlqYGAgpk+flyaP5DJt2twYGBiY8DxZE49NDo0kkFo2lPqt\npI8CfwX8aTqMd7eMDZ6dSLoR6AX2kvQIsAyYmv5Q10TENyS9TdLDJMN4z2rWuc1GVBa3t2xJupb6\n+hZy/PHH5dL9Ue8GVFm7vkZqLsnPBJU1F3frWKNqSSCnAqeTtD6ekLQ/8L+aFUBEnF7DMUuadT6z\nsZT9i3akMNzXt3CHJUEmiq2RmovZRLwWlhntM/ksSzHca1HZePJeC2sBcDkwh6RraVfg2YiYnuWE\neXICsUZ08hftZBuFZbXLO4HcBZwGfB54E3AGcFBEfDTLCfPkBGKNGhoaYmBggJ6eHubMmVN0OGa5\ny30pk4h4GNg1Iv4QEdcBi7KczKzMVq5cxRFHHMPSpZdxxBHHeIkRswnU0gK5HTge+L/AE8AvSBY0\nPCz/8OrjFohlVaYaiLubrJXyboG8Jz1uCckw2v2Ad2Y5mVlZlWUBPi+0aO2kplFYkrqA/SPiJ/mH\nlJ1bIJZVGVogZYjBJp9cWyCS/gxYB3wrvX24pJuznMwsD0NDQ1x//fUMDQ1lfo0yLMBXllaQWa1q\nqYGsBY4D+iNibnrf+oh4Ywviq4tbIJPPBRd8kCuuuIakZ/VRliw5m8svvzTz6xVZfyhrC8Q1mc6W\ndw1ka0RsHnWfv6WtcENDQ2nyuBP4CXAnV1xxbcMtkVZuvTr63EW3gkYbqcm85S197Lffa7n66msL\ni8XKp5alTO6XdDqwq6QDgQ8AP8w3LLOJDQwMkLQ8tnf5wL4MDAy07RyOMi1Dvn19sAuBTwGv4dxz\nlwJwzjlnFxaXlUctLZALgNcDvwNWAr8BPphnUGa16OnpIdlrbPvOc/BYen/7KrIVVGnjxo1MmbIP\nSfJYTVIKvZOlSz/iDa0MqG1Hwucj4mMRMT8i3pRef6EVwZmNZ86cOSxZcjbJKv8HAQtYsuTstm19\nlE2yEOMmRrfypk6d7cK+AeMU0ScaaRUR78gloga4iD45ddryI2UqWl999bVpt9WdlKmwb83TSBF9\nvBrIUST9AyuBNSS7EpqVzpw5czIljjJ9UY9oZNvaPIzUOpYuPZapU2fz4ouPFF7Yt/IYrwWyK3AC\nsJjkT49bgJURcX/rwquPWyDtq9Vf5mX7oobyDuMdia1sydaaI5dhvOnCid+KiDNJOpkfBvoleXMn\na6pWL99Rufvg5s1r2bJlNX195xVeGC7zRMKyFPatXMYtokvaXdIpwGeB84HLgC+3IjCbHIr4Mi/r\nF/WOuweCdw+0squaQCTdAPwImAdcnI7C+kREPN6y6KzjFfFlXtYv6jJOJDQbz3g1kJdIVt+FHWee\nC4iImJZzbHVzDaT9FNHvv3LlKs488/1s3foSsDdTp/6Sf//3qwuvgYxwvcFaKdcdCduJE0h7Gfmi\nvPvudXzoQxe1ZCvZHRPW3sB32GOP83nkkQf9ZW2TUl7DeM1yM3oU1Kc//UnmzTs897+6R7rMtmwZ\n6TI7nalT/4WNGzc6gZjVyS0Qa7kih6uWeaisWRFy3xM9T5IWSdog6UFJF47x+LGSnpF0d3r5eBFx\nWvMUOQrKhWqz5im0BSJpF+BB4C3Az4FB4LSI2FBxzLHAf69l6RS3QNpDGVoBLlSbJdq5BtIDPBQR\nmwAk3QScDGwYdZyXUekgI62Avr6FOxTOW/lF3t3d7cRh1qCiE8g+JOttjXiMJKmMdpSkdcDjwP+I\niAdaEZzlp0z7XphZNkUnkFqsBfaPiOclnQR8hWTt7jEtX7582/Xe3l56e3vzjs8ycivArPX6+/vp\n7+9vymsVXQNZACyPiEXp7YtIJil+apzn/Aw4IiKeHuMx10AK4HqCWftq51FYg8BrJc2SNBU4Ddhh\nHxJJMyuu95AkvZ2ShxWjciHE/fc/iEsu+cfCFyU0s9YofB6IpEXApSTJbEVEfFLSOSQtkWsknQ/8\nDbAV2AJ8KCLWVHktt0BaaMfRVEMkv6a96Op6uhTLo5vZxLyUScoJpLUGBwc54YRz2bz5W8DBJPtm\ne3KeWTtp5y4sa2PbV7X9DjCbsi2Pbmb5cgKxzEbmc+yxx/kkU3fKtTy6meXLCcQasnjxqTzyyIN8\n4hMfq2t5kOHhYQYHB11wN2tjroFY09Q6nLeM+5GbTVYuoqecQPLRzHkeZVgHy8y2cxHdxtVId1Hl\nPI9Zsw5m5cpVDcVS1v3Izax+TiAdrpEEMDw8TF/feWzZsprNm9eyZctq+vrOa6huUdb9yM2sfk4g\nHazRBJBHa8H7cZh1jnZYTNEy2nn71u0JoJYv7B1bC0m9ohmtBa/Ea9YZnEA6WKMJIM99O7wSr1n7\n8yisDjcyZLYyAdQ7ZNar7Zp1Lg/jTTmBjK3eBOCEYTZ5OIGknEAa50l+ZpOLE0jKCaQxnuRnNvl4\nIqE1hSf5mVk9nEA6SKMLFHqSn5nVwwmkQzRjyRFP8jOzergG0gGaXbvwKCyzyaORGognEraRal/s\njc44H82T/MysFu7CahPjdVG5dmFmRXALpA1ULoqYtDLu433vO5a99nolc+fOzXXJETOzapxA2sDO\nXVRDvPDC7znllI/w0kuPbpvs5wUKzayVXERvAzsWyfcGXgf048l+Ztaotp5IKGmRpA2SHpR0YZVj\nLpP0kKR1kg5vdYxFqxxe+/KXHwPshSf7mVnRCk0gknYBrgDeCrweWCzp4FHHnAQcEBEHAucAV7U8\n0BJYvPhUNm3awJe+dDldXU/jgrmZFa3oFkgP8FBEbIqIrcBNwMmjjjkZuAEgItYA0yXNbG2Y5dDd\n3c2JJ57oyX5mVgpFF9H3AR6tuP0YSVIZ75jH0/uezDe08nLB3MzKoOgE0nTLly/fdr23t5fe3t7C\nYsmTJ/uZWRb9/f309/c35bUKHYUlaQGwPCIWpbcvAiIiPlVxzFXA6ohYld7eABwbETu1QDp1FJaZ\nWV7aeRTWIPBaSbMkTQVOA24edczNwBmwLeE8M1byMDOz1iq0Cysi/iBpCXArSTJbERFDks5JHo5r\nIuIbkt4m6WHgOeCsImM2M7OEJxKamU1i7dyFZWZmbcoJxMzMMnECMTOzTJxAzMwsEycQMzPLxAnE\nzMwycQIxM7NMnEDMzCwTJxAzM8vECcTMzDJxAjEzs0ycQMzMLBMnEDMzy8QJxMzMMnECMTOzTJxA\nzMwsEycQMzPLxAnEzMwycQIxM7NMnEDMzCwTJxAzM8vECcTMzDKZUtSJJb0SWAXMAjYC746IzWMc\ntxHYDLwEbI2InhaGaWZmVRTZArkIuC0iXgd8D/holeNeAnojYm4nJ4/+/v6iQ2iI4y+W4y9Wu8ef\nVZEJ5GTg+vT69cCfVzlOTIKutnb/D+j4i+X4i9Xu8WdV5BfzqyLiSYCIeAJ4VZXjAviOpEFJZ7cs\nOjMzG1euNRBJ3wFmVt5FkhA+PsbhUeVljo6IX0jqJkkkQxFxR5NDNTOzOimi2vd2zieWhkhqG09K\nejWwOiLmTPCcZcBvI+JfqzxezA9jZtbGIkJZnlfYKCzgZuC9wKeAM4Gvjj5A0suAXSLiWUkvB04E\nLq72glnfBDMzq1+RLZAZwOeA/YBNJMN4n5G0N3BtRLxd0muAL5N0b00B/jMiPllIwGZmtoPCEoiZ\nmbW3th0eK+mVkm6V9BNJ35Y0fYxj9pX0PUn3S1ov6QNFxDoqpkWSNkh6UNKFVY65TNJDktZJOrzV\nMY5novglnS7p3vRyh6Q3FhFnNbW8/+lx8yVtlXRKK+ObSI3/f3ol3SPpx5JWtzrGamr4vzNN0s3p\n//v1kt5bQJhVSVoh6UlJ941zTCk/uxPFnvlzGxFteSGpnXwkvX4h8Mkxjnk1cHh6fU/gJ8DBBca8\nC/Awyez73YB1o+MBTgJuSa8fCdxZ9HtdZ/wLgOnp9UXtFn/Fcd8Fvg6cUnTcdb7/04H7gX3S239c\ndNx1xP5R4J9G4gZ+BUwpOvaK+I4BDgfuq/J4mT+7E8We6XPbti0QapiIGBFPRMS69PqzwBCwT8si\n3FkP8FBEbIqIrcBNJD9HpZOBGwAiYg0wXdJMymHC+CPizti+JM2dFPt+j1bL+w9wAfAF4KlWBleD\nWuI/HfhiRDwOEBG/bHGM1dQSewCvSK+/AvhVRLzYwhjHFcn0gV+Pc0hpP7sTxZ71c9vOCaTWiYgA\nSJpNkoHX5B5ZdfsAj1bcfoydf1Gjj3l8jGOKUkv8ld4PfDPXiOozYfyS/hvw5xHxf0jmLZVJLe//\nQcAMSavTybfvaVl046sl9iuAQyT9HLgXWNqi2JqlzJ/detT8uS1yGO+EmjQREUl7kvxFuTRtiVjO\nJC0EziJpOreTfyPpEh1RtiQykSnAPOA44OXAjyT9KCIeLjasmrwVuCcijpN0AMnE4UP9mW2dej+3\npU4gEXFCtcfSgtDM2D4RcczuBklTSJLHf0TETnNNWuxxYP+K2/um940+Zr8JjilKLfEj6VDgGmBR\nRIzX5G+1WuJ/E3CTJJH0w58kaWtE3NyiGMdTS/yPAb+MiBeAFyTdDhxGUn8oUi2xnwX8E0BE/FTS\nz4CDgbtaEmHjyvzZnVCWz207d2GNTESEKhMRU58BHoiIS1sR1AQGgddKmiVpKnAayc9R6WbgDABJ\nC4BnRrrqSmDC+CXtD3wReE9E/LSAGMczYfwR8Sfp5TUkf3icV5LkAbX9//kqcIykXdOJuEeS1P6K\nVkvsm4DjAdLawUHAf7U0yomJ6q3SMn92YZzYM39uix4d0MCoghnAbSQjq24F/ii9f2/g6+n1o4E/\nkIz4uAe4myS7Fhn3ojTmh4CL0vvOAf664pgrSP5ivBeYV/R7XU/8wLUko2fuTt/zgaJjrvf9rzj2\nM5RoFFYd/3/+lmQk1n3ABUXHXMf/nb2Bb6dx3wcsLjrmUfHfCPwc+B3wCEmLqS0+uxPFnvVz64mE\nZmaWSTt3YZmZWYGcQMzMLBMnEDMzy8QJxMzMMnECMTOzTJxAzMwsEycQm7QkzUiXPb9b0i8kPVZx\nu7BVGiS9RdKXizq/Wa1KvZSJWZ4i4mlgLoCkvweejYh/HX2cJEXrJ0x5gpaVnlsgZoltSzxIOiDd\nhOyzkn4M7Cfp1xWPnyrp2vT6qyR9UdKApDsl9ez0wsmquAdW3P5/kg6VdKSkH0pam953wBjP/YQq\nNkKTNJSuGIykMyStSVtMVzTtnTCrkROI2dheB/zviHgDyYJ4o1sEI7cvAz4VET3AqcCKMV7rpvQx\nJO0DvDIi7gMeAI6JiCOAS4B/qCGuSF/n9cBfAEdFxDxgN0mn1fHzmTXMXVhmY/tpRNxTw3HHAwel\nq/dCsonQ7hHxu4pjPk+y0N4lJInk8+n9rwT+o6LlUU+31fEkKwfflZ57D5I1jsxaxgnEbGzPVVx/\niR1b63uMOnZ+RPyh2gtFxCOSnpU0hySBnJk+9A/AtyLiqjSJjLWJz4ujzt2V/ivgMxGxbOIfxSwf\n7sIyG9u2mkhaQH86rY3sQtJ1NOI2ki1wkydJh1V5vVUke35PjYgN6X3T2L5fxFlVnrcROCJ97R62\n7zdxG/DdnHjeAAAAlElEQVRuSXulj82QtN+Yr2CWEycQs7GN7k66iGTbgDvYcdvSJcDRku5NC+7v\nr/J6XwAWkySSEf8M/Iuku8Y434jPA6+WdF/62j8FiIgfAxcDt0m6l2QZ9HG3dTZrNi/nbmZmmbgF\nYmZmmTiBmJlZJk4gZmaWiROImZll4gRiZmaZOIGYmVkmTiBmZpaJE4iZmWXy/wFnIugTfqGgAAAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105f690b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "scatter( true_values, heteroscedastic_measurements )\n",
    "xlabel( \"True value\" )\n",
    "ylabel( \"Measured value\" )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When these three assumptions are met, a series of increasingly sophisticated two-sample tests are available depending on the amount of prior knowledge you have about your study population.\n",
    "\n",
    "### Z-tests\n",
    "\n",
    "In the unlikely event that you've sampled your *entire* case and control populations, then there's no uncertainty in either the location (mean) or spread (standard deviation) of your samples.  This allows you to use a **z-test**, which uses as its test statistic the number of standard errors separating the case and control means.\n",
    "\n",
    "To crib from [Wikipedia's superb example on the subject](http://en.wikipedia.org/wiki/Z-test#Example), suppose that 30 students in a particular school receive the following scores on a standardized test:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[23, 81, 82, 23, 32, 84, 42, 7, 78, 41, 56, 53, 71, 47, 15, 18, 70, 88, 92, 1, 19, 25, 72, 45, 2, 28, 84, 68, 92, 88]\n"
     ]
    }
   ],
   "source": [
    "population_scores = [random.randint( 0, 100 ) for i in range( 30 )]\n",
    "print( population_scores )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50.899999999999999"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean( population_scores )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29.439599182054092"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sample standard deviation\n",
    "std( population_scores )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.3749108519738877"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sample standard error = standard deviation / sqrt( number of measurements )\n",
    "import scipy.stats\n",
    "\n",
    "scipy.stats.sem( population_scores, ddof = 0 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.3749108519738877"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "std( population_scores ) / sqrt( len( population_scores ) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first ten scores come from a particular class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[23, 81, 82, 23, 32, 84, 42, 7, 78, 41]\n"
     ]
    }
   ],
   "source": [
    "bio508_scores = population_scores[:10]\n",
    "print( bio508_scores )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we'd like to test whether this class received \"unusual\" scores on the test, i.e. whether they differ significantly from the overall population, we can measure how many standard errors apart their means are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.29767935581897431"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = ( mean( bio508_scores ) - mean( population_scores ) ) / scipy.stats.sem( population_scores, ddof = 0 )\n",
    "z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This provides us with a **test statistic** that summarizes our comparison of these two samples.  Under the assumptions listed above, a z-statistic is normally distributed, and we can determine the corresponding **p-value** by seeing how likely a statistic at least this extreme would be to arise (in either direction) from a standard normal distribution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-5. , -4.9, -4.8, -4.7, -4.6, -4.5, -4.4, -4.3, -4.2, -4.1, -4. ,\n",
       "       -3.9, -3.8, -3.7, -3.6, -3.5, -3.4, -3.3, -3.2, -3.1, -3. , -2.9,\n",
       "       -2.8, -2.7, -2.6, -2.5, -2.4, -2.3, -2.2, -2.1, -2. , -1.9, -1.8,\n",
       "       -1.7, -1.6, -1.5, -1.4, -1.3, -1.2, -1.1, -1. , -0.9, -0.8, -0.7,\n",
       "       -0.6, -0.5, -0.4, -0.3])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Separate the range from -5 to 5 into values less than and greater than our z-statistic\n",
    "x_below_z = arange( -5, -abs( z ), 0.1 )\n",
    "x_below_z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PolyCollection at 0x10bc6f6d8>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEACAYAAAC9Gb03AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYXHWd7/H3t9d0OiEhbEpCMBgNi8giRhDQwlEJeCE+\nzjgSx+W6YK4j4KMzd/CKSserjpeZkWFAR5goDo4xAwlkJ7uVvbesJKRJk5CQjZDQ6a16qeV87x9V\niZWml9PpqvrV8n09Tz9UnTq/Op+ikk9On1VUFWOMMfmryHUAY4wx6WVFb4wxec6K3hhj8pwVvTHG\n5DkremOMyXNW9MYYk+d8Fb2ITBGRBhHZIyIP9jPfB0UkIiKfGexYY4wx6SEDHUcvIkXAHuAvgCNA\nHXCPqjb0Mt8KoBP4rao+73esMcaY9PGzRj8ZaFTVA6oaAWYDU3uZ735gDvDmWYw1xhiTJn6Kfixw\nMOn5ocS000TkYuDTqvrvgAxmrDHGmPRK1c7YfwVs+7sxxmShEh/zHAbGJz0fl5iW7AZgtogIcD5w\nh4hEfY4FQETsojvGGDNIqip+Zur3BygGXgUuBcqAbcAV/cz/NPCZwY6NR8lPDz/8sOsIaZVPn8/z\nVL/8ZdWbb1adN081HFb9wQ8e1tmzVa+/XvXrX4/Pk0/y6fvrTT5/vkRvDtjjA266UdUYcB+wHNgF\nzFbV3SIyXUS+0duQgcYO+K+PMY789KewaxcsWwZTp0JpKRQXw+c+B8Eg1NTAY4+5TmnM4PjZdIOq\nLgUm9Zj2ZB/zfnWgscZko1mzYOZMqK6Gysq3vz5yJCxcCDfeCJMmwR13ZD6jMWfDzozNgEAg4DpC\nWuXD59u3Dx54ABYtgne848zXkj/fpZfCnDnw5S/DoUOZzZgu+fD99SffP58fA54wlSkiotmSxRSe\nL30JLrsMqqr8zf+970FzM/z612mNZUy/RMTXzlgrelPwdu2C226DV1+Fc87xN+att+Kbb2pr4/9A\nGOOC36K3TTem4P3oR/AP/+C/5AHOOw/uvx9mzEhfLmNSxdboTUGrr48fXfPqq1BRMbixra0wcSKs\nWQNXXJGefMb0x9bojfHhhz+EH/xg8CUP8d8A/v7v4aGHwthKislmVvSmYO3dC+vXd9LQ8NBZv8c3\nvwkLF4Z5/PHnUpjMmNSyojcF6+mnFZG5HD/+5sAz92HkSBg9eiM///nxFCYzJrWs6E1BisVg5swI\nbW3/NOT3uvDCZRw9GuCVV/akIJkxqWdFbwrSqlUQjb4BNA/5vUaM2AVUMmPGqiG/lzHpYEVvCtLM\nmTHa2p4Avjjk9xIBOMjcuSOJRqNDfj9jUs2K3hSckydh8eIYpaWnLqyaClHC4buZP39lit7PmNSx\nojcF549/hIqKjYRCn03hu3YDzTz8cGMK39OY1LCiNwXn6ae7aW39d+DTKX7n89m9exJvvfVWit/X\nmKGxojcF5dgx2LVLKS4eCZzFWVL9Go7n3cKcOctS/L7GDI0VvSkoixbBsGHVdHV9Kk1LOMmvfnUk\nTe9tzNmxojcF5bnnIrS2/h74ZJqWMIqdOy+hs7MzTe9vzOBZ0ZuCEQrFL0A2bFgI6OUWUikxAs+7\ng2XLVqfp/Y0ZPCt6UzCWL4fKygZCoY+neUkdPPFEQ5qXYYx/vopeRKaISIOI7BGRB3t5/W4R2S4i\nW0WkVkRuTnptf/JrqQxvzGA8/3yMlpbfA3eleUllrFs3ilgslublGOPPgEUvIkXAE8DtwFXANBG5\nvMdsK1X1GlW9DvgaMDPpNQ8IqOp1qjo5RbmNGZRoFBYujDFs2KvARWle2hgikTuprq5J83KM8cfP\nGv1koFFVD6hqBJgNTE2eQVU7kp6OIF7up4jP5RiTNhs3QnHxMUKhGzOyPNVyfvWruowsy5iB+Cng\nscDBpOeHEtPOICKfFpHdwELgq0kvKbBCROpE5N6hhDXmbC1apHR0zEF16sAzp0SERYu8gWczJgNS\ntqatqvNU9Qripxv+JOmlm1X1euBO4FsickuqlmmMX4sXdwGbgEkZWuKFtLffwMGDBwee1Zg0K/Ex\nz2FgfNLzcYlpvVLV9SJymYiMUdUmVT2amH5cRF4gvilofW9jq6qqTj8OBAIEAgEf8Yzp34kTsG9f\nEarnZXCpRXjeB1my5DmmTx/6FTKNAQgGgwSDwUGP81P0dcBEEbkUOArcA0xLnkFE3q2qexOPrwfK\nVLVJRIYDRaraLiKVxM9SmdHXgpKL3phU+dOfYPjwHTQ13ZbhJTfzn/95hOnTM7xYk7d6rgDPmNFn\nnZ5hwKJX1ZiI3AcsJ76p5zequltEpsdf1qeAvxSRLwFhoBP468Twi4AXREQTy/qDqi73/amMSYHl\ny2O0ts4F/j7DSy5n8+aRqCoSv2i9MU74WaNHVZfSY+Omqj6Z9PgR4JFexr0GXDvEjMYMyZIlYcrL\n9xKNnp/hJZ9LNBpg586dXH311RletjF/Zoc9mry2bx+0tMTo6prgZPmeN4EXXtjgZNnGnGJFb/La\nqlVQWrqRWCzdlz3oy0lmzRr6fWmNGQorepPXli6N0NY2F3B1VO8oGhvfSTgcdrR8Y6zoTR7zPFi5\n0qOiohkY7ihFJaqfZNOmakfLN8aK3uSxHTugqKiFUOgapzlUK/njHzc7zWAKmxW9yVtr1kAsthbV\njzlOEmLRoi7HGUwhs6I3eWvVqjAdHUuBDzhOci5Hjoylq8vK3rhhRW/ykiqsXatUVISAUsdphiES\noK7OrmZp3LCiN3lpzx7wvE5CoUxdxKx/nnc+ixZtcR3DFCgrepOX1q2DoqJaVG91HSWhhXnzQq5D\nmAJlRW/yUjAYpa1tEZCZG40MbCR7955vtxc0TljRm7y0enWEioo3gUrXURJGoHorO3bscB3EFCAr\nepN3Dh+G5malq+sS11HO4HkTWLrU7iNrMs+K3uSddeugvHw7sVi23cysmTlzmlyHMAXIit7knTVr\nPNrbF+Pu+jZ9GcbOnSNQVddBTIGxojd5Z+XKLsrKXgUyeetAP0YTjd5EY2Oj6yCmwFjRm7zS3AwH\nD5YQjY5xHaVXnnc1q1fb9elNZlnRm7xSUwPDhzcSDt/kOkof2nj++aOuQ5gCY0Vv8sqmTdDZ+Scg\nW4seamuLXUcwBcZX0YvIFBFpEJE9IvJgL6/fLSLbRWSriNSKyM1+xxqTSn/6UxexWA3wbtdR+jCG\n1tb30Nxsd50ymTNg0YtIEfAEcDtwFTBNRC7vMdtKVb1GVa8DvgbMHMRYY1LC86C+voiyMgBxHacP\nxcAt1NTY8fQmc/ys0U8GGlX1gKpGgNnA1OQZVLUj6ekIwPM71phUaWiAkpJWQqErXUfpl+pwlizZ\n5TqGKSB+in4scDDp+aHEtDOIyKdFZDewEPjqYMYakwqbNoHIZrLn+jZ9aefFFztdhzAFpCRVb6Sq\n84B5InIL8BPgE4N9j6qqqtOPA4EAgUAgVfFMAdiwIUZb2zLgx66jDGAE+/aNxvM8iorseAjjXzAY\nJBgMDnqcn6I/DIxPej4uMa1XqrpeRC4TkTGDHZtc9MYM1po13VRUvEEoNMJ1lAGMQPUmGhoauPLK\n7N7MZLJLzxXgGTNm+BrnZ3WiDpgoIpeKSBlwD7AgeQYReXfS4+uBMlVt8jPWmFRoaYHDh0uIRLLz\nRKmePO9K1q61HbImMwYselWNAfcBy4FdwGxV3S0i00XkG4nZ/lJEdorIFuBx4K/7G5uGz2EKXG0t\nDB/+KuHwZNdRfLITp0zm+NpGr6pLgUk9pj2Z9PgR4BG/Y41JtfiJUmuAj7uO4pudOGUyxfYEmbwQ\nP1FqIzDRdRSf7MQpkzlW9CbnqUJ9vVBWpmTviVI9FQMfpra21nUQUwCs6E3O27sXRDro6Mitk65V\nz2HFCjtxyqSfFb3JeTU1UFS0HdVc2RF7SisvvtjuOoQpAFb0JufV1Hi0t68EbnAdZZAqaGystDtO\nmbSzojc5b82aLsrKXgNy4xj6PzuHaPR6Dh48OPCsxgyBFb3JaeEwNDSUIlLhOspZEDzvBjZtshOn\nTHpZ0ZuctmMHDB9+jI6Oa1xHOUtRFi/e5zqEyXNW9Can1dZCLFZH/IrYuaiLNWsirkOYPGdFb3La\nxo1RQqFVQK6u0Y/i0KHziEajroOYPGZFb3La+vVhKiqagGGuo5ylCuDDvPzyy66DmDxmRW9yVksL\nHD1aQjh8vusoQ+J5l7NhQ73rGCaPWdGbnFVfD5WV+4hEcu34+Z7aWLDArmRp0seK3uSs2lro6lpP\n7u6IPcWjrs7+Kpr0sT9dJmetW9dNOLwBeK/rKEN0Lk1N4wmFQq6DmDxlRW9yVm2tUlERIff/GJci\nchNbt251HcTkqVz/G2IK1JEj0NHh0dFxiesoKeF5Y1m7drvrGCZPWdGbnFRXB+Xlu/G8D7qOkiLN\nLFp00nUIk6es6E1Oqq1VOjrWAPlS9CW89FKZ6xAmT/kqehGZIiINIrJHRB7s5fXPi8j2xM96EXl/\n0mv7E9O3iojdTsekxPr1XcA2YJzrKClyLqHQe2hqanIdxOShAYteRIqAJ4DbgauAaSLS81Y++4CP\nqOo1wE+Ap5Je84CAql6nuXdnCJOFVGHLliJKSyF3bh04kCLgRurr7cQpk3p+1ugnA42qekBVI8Bs\nYGryDKparaotiafVwNikl8XncozxZd8+gC46OnLlRuD+qI5m9Wq7taBJPT8FPBZIvjPCIc4s8p6+\nDryY9FyBFSJSJyL3Dj6iMWeqq4Pi4pdQzZft86e02K0FTVqUpPLNROQ24CvALUmTb1bVoyJyAfHC\n362q63sbX1VVdfpxIBAgEAikMp7JE7W1Sii0Gvhb11FSrJw9e3LxBiomU4LBIMFgcNDj/BT9YWB8\n0vNxiWlnSOyAfQqYoqqnjxNT1aOJ/x4XkReIbwoasOiN6cvatZ2UlOwhGs3ti5m93WjC4Ws4cuQI\nF198seswJgv1XAGeMWOGr3F+Nt3UARNF5FIRKQPuARYkzyAi44G5wBdVdW/S9OEiMiLxuBL4JLDT\nVzJjehGLwa5dpRQXp/SX0SwhqH6Q2to610FMnhmw6FU1BtwHLAd2AbNVdbeITBeRbyRm+yHxOzP/\nqsdhlBcB60VkK/GdtAtVdXnKP4UpGLt3Q3n5SUKhq1xHSQvVYpYta3Qdw+QZX6tFqroUmNRj2pNJ\nj+8F3rajVVVfA64dYkZjTqurA9Wt5M+JUj2FWLmy03UIk2fssEeTU2pqYrS3rwKudx0lTSrZv/8c\nVNV1EJNHrOhNTlm7tpvy8sPAOa6jpMlIPO8G9u7dO/CsxvhkRW9yRnc3vPpqGVDpOkpaed611NTY\nDlmTOlb0Jmds3w6VlUfp7LzGdZQ0i7BkyQHXIUwesaI3OaOuDqLROvJ3R+wpXaxdG3UdwuQRK3qT\nMzZtitLRsRrI9zX6URw5ch7RqJW9SQ0repMzNmwIU1FxEih3HSXNKoAb2b17t+sgJk9Y0Zuc0NYG\nR46UEomMdh0lIzzvCjZtsksWm9Swojc5YcsWqKzcTzj8AddRMiTEwoVHXYcwecKK3uSEujro7t5E\n/u+IPSVKTY3rDCZfWNGbnLBxY5ju7nXAla6jZMi5nDhxMd3d3a6DmDxgRW9ywqZNUYYN6wSKXUfJ\nkDJEbmL79u2ug5g8YEVvst6JE3DyZBFdXe9wHSWjPO9drF+/1XUMkwes6E3Wq6+H4cMbicVucB0l\nw1pYsOC46xAmD1jRm6xXWwudnWuI35yskAjbtuXjDVZMplnRm6y3dm0XsVgtMMF1lAw7l9bWd9PS\n0uI6iMlxVvQmq6lCfb1QXq6AuI6TYSXAh9m8ebPrICbHWdGbrHbgAMRi3YRC73YdxQnVMaxa9ZLr\nGCbHWdGbrFZTAyUlO1AttO3zp7SyeHGr6xAmx/kqehGZIiINIrJHRB7s5fXPi8j2xM96EXm/37HG\n9KemRhO3DiyUM2J7KuOVV4a7DmFy3IBFLyJFwBPA7cBVwDQRubzHbPuAj6jqNcBPgKcGMdaYPq1d\n20lJSSNwgesojowmHH4/hw8fdh3E5DA/a/STgUZVPaCqEWA2MDV5BlWtVtVThwZUA2P9jjWmL9Eo\n7NxZSnFxqesoDgmqk6mttVsLmrPnp+jHAgeTnh/iz0Xem68DL57lWGNO27ULKipOEAq9f+CZ85iq\nsGRJo+sYJoel9GwMEbkN+Apwy9mMr6qqOv04EAgQCARSksvkptpa8LzNFN6JUj11sHq1XdzMQDAY\nJBgMDnqcn6I/DIxPej4uMe0MiR2wTwFTVPXkYMaeklz0xmzaFKO9fSXwM9dRHBvJgQOj8TyPoiI7\nUK6Q9VwBnjFjhq9xfv7U1AETReRSESkD7gEWJM8gIuOBucAXVXXvYMYa05d167qpqDgOFPpRJ5Wo\nfog9e/a4DmJy1IBFr6ox4D5gObALmK2qu0Vkuoh8IzHbD4ExwK9EZKuI1PY3Ng2fw+SZ9nZ4/fVS\notFRrqNkBc+7mg0bal3HMDnK1zZ6VV0KTOox7cmkx/cC9/oda8xA6uuhsvI1Tp4slFsHDqSd+fMP\n87Wvuc5hcpFt8DNZqboaurvXAze5jpIlPDZudJ3B5CorepOV1qzpprt7LWDn18WdS1PTuwiFQq6D\nmBxkRW+yjipUVysVFRHsj+gppYjcQn19vesgJgfZ3yKTdQ4cgGg0XLBXrOyL553PihV2D1kzeFb0\nJutUV0NJyXZUb3QdJcu0sHChXcnSDJ4Vvck6mzZ5tLcvBz7kOkqWGcYrr1Siqq6DmBxjRW+yTjDY\nRWnpXuA811GyzCgiket5/fXXXQcxOcaK3mSV7m5oaCgFKl1HyUKC593Axo01roOYHGNFb7LKli1Q\nWXmYzs7rXUfJUt3Mn/+a6xAmx1jRm6xSXQ3h8CbAdsT2LsLatTHXIUyOsaI3WWXdujBdXauAq11H\nyVKjeeONi+nutssWG/+s6E1WWbfOo6KigxTfKiGPlCPyEbZu3eo6iMkhVvQma7z+OnR2RunouNR1\nlKzmee9k+fLNrmOYHGJFb7LGhg1QWroNz7vZdZQs18Lzzze7DmFyiBW9yRrr1nm0tS0BPuw6Spar\nYPfukXbilPHNit5kjdWruygvf434PWxM30YRjX6IvXv3DjyrMVjRmyzR1gavvVZCLDbadZSc4HnX\nEAzaBeqNP1b0JitUV0Nl5V66u+34eX/amDPnsOsQJkdY0ZussH49dHauAmxHrF/V1cWuI5gc4avo\nRWSKiDSIyB4RebCX1yeJyEYR6RKR7/Z4bb+IbE++abgxPa1a1UUstgmwa9D7M4bW1stpampyHcTk\ngAGLXkSKgCeA24GrgGki0vP+bm8B9wP/1MtbeEBAVa9T1clDzGvyUDQKW7YUU1ZWDIjrODmiGLiV\nDRtsO70ZmJ81+slAo6oeUNUIMBuYmjyDqp5Q1c1AtJfx4nM5pkDt2AHl5ScIha5xHSWnqBYxf36D\n6xgmB/gp4LHAwaTnhxLT/FJghYjUici9gwlnCsOGDRCLbcS2zw9WB8uWhV2HMDkgExcUuVlVj4rI\nBcQLf7eqru9txqqqqtOPA4EAgUAgA/GMa6tXR+joWAr80nWUHDOaI0fG0t3dTXl5ueswJgOCwSDB\nYHDQ4/wU/WFgfNLzcYlpvqjq0cR/j4vIC8Q3BQ1Y9KYwqEIw6FFR0UZ7e5nrODmmAtWPUVtby623\n3uo6jMmAnivAM2bM8DXOz6abOmCiiFwqImXAPcCCfuY/vTdNRIaLyIjE40rgk8BOX8lMQWhoAM8L\n0dFxhesoOUl1DC+8sMV1DJPlBix6VY0B9wHLgV3AbFXdLSLTReQbACJykYgcBL4DPCQirycK/iJg\nvYhsBaqBhaq6PF0fxuSeNWsANuF5H3UdJUe1Mn9+h+sQJsv52kavqkuBST2mPZn0+BhwSS9D24Fr\nhxLQ5LeVKyOEQguBR11HyVGj2L//HYTDYcrKbNOX6Z0d9micUYXVqz0qKpqBCtdxctRwVG+jvr7e\ndRCTxazojTONjRCNdhIKvdd1lJymegELFljRm75Z0Rtn1qyBoqJqVG37/NC0MnduyHUIk8Ws6I0z\nK1dGaW9fCNzkOkqOO4d9+y4gEom4DmKylBW9cSK+fT5KRcUJYLjrODmuEriNLVvsMEvTOyt648Te\nvdDVFaajY6LrKHnB897JggV1rmOYLGVFb5wIBqG4uAbPC7iOkieamTOn3XUIk6Ws6I0TS5dGaG9/\nAbsReKqMYu/edxIO20XOzNtZ0ZuM8zxYscJLbJ+vdB0nT1TieZ9g06Zq10FMFrKiNxm3bRsUFTUT\nCtlJ06mkWskf/2g7ZM3bWdGbjFuxAqLRlah+3HWUPBNi/nzbdGPezoreZNzixd10dS0GPuA6Sp45\nn2PHLqe5udl1EJNlrOhNRnV2Ql1dEeXlJcTve2pSpwzV21i1ao3rICbLWNGbjFq/HoYPf41QyG4b\nmB5d/O53e1yHMFnGit5k1PLlSii0APiE6yh5SgkGM3GHUJNLrOhNRi1a1IVILXCZ6yh56nxCockc\nOHDAdRCTRazoTcYcPw779wuq57mOkseKUL2eRYuCroOYLGJFbzJm6VKoqNhMd/dfuI6S504yc+Yx\n1yFMFrGiNxkzb16E1tY/EL9HvEmfEbz00jvscgjmNF9FLyJTRKRBRPaIyIO9vD5JRDaKSJeIfHcw\nY01hiERg2TKlouI4cI7rOHnuHDzvTtauXe86iMkSAxa9iBQBTwC3A1cB00Tk8h6zvQXcD/zTWYw1\nBWD9ehg27DDt7XZYZSaown/8x3bXMUyW8LNGPxloVNUDqhoBZgNTk2dQ1ROquhmIDnasKQwLFyrt\n7XOA/+E6SoGIsWSJnZBm4vwU/VjgYNLzQ4lpfgxlrMkjc+d2U1S0CbAbjWTGhXR0fITGxkbXQUwW\nyKozK6qqqk4/DgQCBAIBZ1lM6uzZA01NESIRO3Y+cwTPew+zZv2Bhx9+j+swJkWCwSDBYHDQ4/wU\n/WFgfNLzcYlpfgxqbHLRm/yxeDEUFa0iGv2U6ygFppnf/a6Vhx92ncOkSs8V4BkzZvga52fTTR0w\nUUQuFZEy4B5gQT/zyxDGmjz0/PPddHQ8C9ziOkqBOY8DB95La2ur6yDGsQGLXlVjwH3AcmAXMFtV\nd4vIdBH5BoCIXCQiB4HvAA+JyOsiMqKvsen6MCb7NDVBfb1QVlYMlLqOU2CGofoxFixY6TqIcczX\nNnpVXQpM6jHtyaTHx4BL/I41hWPBAhg+vI6mpjtdRylQzTz66H6+8AXXOYxLdmasSatZsyK0tPwW\nO6zSlVFs2zaezs5O10GMQ1b0Jm2am2HdOmXYsA5gpOs4BWoknjeFefOWuQ5iHLKiN2mzYAFUVm4m\nFLK1ebeaefTRfa5DGIes6E3azJoVobn5N8BdrqMUuFFs3jzONt8UMCt6kxYtLbBmjTJsWAi7iJlr\nI/G8O1m0yI6+KVRW9CYt4pttthIK2UlS2aGFf/kXuxxCobKiN2lhm22yzQjq6y+hq6vLdRDjgBW9\nSbk33zy12aYVGOU6jgFgFLHYHTz77FLXQYwDVvQm5WbNgoqK1YRCn3MdxZzhLX7609ddhzAOWNGb\nlHvqqW7a2n4N2Pb57HIRjY0f5Ngxu59sobGiNym1bRscOdIJvAsoc5zGnGkYqlfyi18sdh3EZJgV\nvUmpp59WIpH/JBL5susopldtPPlkGFV1HcRkkBW9SZlwGJ55JkL8YqXXuo5jenUxra13sXnzVtdB\nTAZZ0ZuUWbIESkr20dn5cc68LYHJHkWoFlNVtdF1EJNBVvQmZZ54IkpLy2Ooft51FNOvEpYtu8Qu\niVBArOhNSjQ0QE1NmJKSduAi13FMv84nGr2NX/5yoesgJkOs6E1KPP64ovpbOju/6TqK8eUkP/1p\nm+2ULRBW9GbIWlriO2FVFwM3uY5jfLmE5uZPs2qVbasvBL6KXkSmiEiDiOwRkQf7mOffRKRRRLaJ\nyHVJ0/eLyHYR2SoitakKbrLH009DefkGOjqmYTthc0UR0MZ3v7vddRCTAQMWvYgUAU8AtwNXAdNE\n5PIe89wBvFtV3wNMB/496WUPCKjqdao6OWXJTVaIxeAXvwjT1vbPgF3yILdcyM6dt/D66wddBzFp\n5meNfjLQqKoHVDUCzAam9phnKvAMgKrWAKNE5NQeOfG5HJODFi2Czs6jRKMfAMpdxzGDMhzVi/i7\nv7MLneU7PwU8Fkj+J/9QYlp/8xxOmkeBFSJSJyL3nm1Qk31U4Uc/itDa+kM8z3bC5ibh+eevpLm5\n2XUQk0aZWNO+WVWvB+4EviUit2RgmSYDFi+Gw4ePo3ou8E7XccxZuRDVidx//zzXQUwalfiY5zAw\nPun5uMS0nvNc0ts8qno08d/jIvIC8U1B63tbUFVV1enHgUCAQCDgI55xQRUeeihCW9v3iET+0XUc\nMwSqMGvWRB5/vJnRo0e7jmP6EQwGCQaDgx7np+jrgIkicilwFLgHmNZjngXAt4D/FpEbgWZVPSYi\nw4EiVW0XkUrgk8CMvhaUXPQmu525Nt9zS57JLRehKtx//zx+//v/6TqM6UfPFeAZM/qs0zMMuOlG\nVWPAfcSvVLULmK2qu0Vkuoh8IzHPEuA1EXkVeBL428Twi4D1IrIVqAYWqupyvx/KZKcz1+b/wXUc\nkwKqyqxZE21bfZ7ys0aPqi4FJvWY9mSP5/f1Mu417DKGeefZZ+HIkWOojsbW5vPFRah63HvvIp57\n7guuw5gUs8MezaCEQvCd70Robf0mkchDruOYFFItYu7cG9mz54DrKCbFrOjNoPzsZ0okso5o9GPY\nxcvyTXyt/u6717oOYlLMit74tncvPP54mFDox3je27bUmbwwnlde+RTPPtvrgXEmR1nRG19U4dvf\njuF5/0Zn5w+AUteRTFoMA47z1a82E41GXYcxKWJFb3z5r/+CmpoTeF498HHXcUxavZeOjuv44hcX\nuA5iUsSK3gxo/3544IEIbW3T6Ox8zHUck3aCahGzZ9/CypU7XYcxKWBFb/oVi8Hf/E2UaPQXdHc/\nALzDdSRxGxELAAAIxklEQVSTEe8EjnH33U10dna5DmOGyIre9OuRR6CxcS+RyKvAp13HMRn1Prq6\nxjJlyiLXQcwQWdGbPi1aBI880kFb29fp7v6F6zgm4wTVMaxd+1F+/ON1rsOYIbCiN73asQO+8IUI\nHR1/RVfXr4GRriMZJ84Fmnj44UnMn7/LdRhzlqzozdscOwZ33hkhHH6AcPgB4jcWM4VrEnCQz3ym\nnJdfPuo6jDkLVvTmDG++CbfdFqW9/Um6u68CpriOZLLCB1A9zgc+0MT+/S2uw5hBsqI3px09Crfc\nEuXIkScJhd6ws1/NGVRvJBx+k8svP0pj40nXccwgWNEbIH6s/M03Rzl27HFCoSai0Z+4jmSyjuB5\nASKRI7zvfcfZseMt14GMT1b0huXL4YYbIhw//n8JhcJEoz90HclkLcHzbiMafY3rrovyzDOvuQ5k\nfLCiL2CeB//4j/C5z3XQ3v5XtLe/j1jsQdexTNYTPO92PG8HX/5yJV/5yjZUXWcy/bGiL1A7d8KN\nN0Z59NE9dHXdTnf3z4DPuo5lcsongGM88wxcfHEjW7a0uw5k+mBFX2Cam+F731M+/OEudu78Pi0t\nj9HVtQg7hNKcnavxvPGcOFHPDTeEmTbtFVpbXWcyPVnRF4iTJ6GqCi69NMLMmS8Sjd5OZ+dUwuFf\nAqNcxzM5bQzR6DRUG3juuQbOO+8k06cfsMLPIr6KXkSmiEiDiOwRkV434orIv4lIo4hsE5FrBzPW\npIcqbNgAX/qSx7hxYR5//EUikSm89dYJOjtXATe7jmjyyoeJxe4kGl3Jb36zjTFj2rj99gNs3Bi1\nbfiODVj0IlIEPAHcTvz3+2kicnmPee4A3q2q7wGmA7/2O7YQBIPBjC2rsxNWrID77/cYO7aLu+46\nxrPP/hiRz9LU1ERn5zLgS/i8L7xPwRS+V/bJ5PfnRjCF71UKfJZY7A5isSUEg0u49dZDjBnzFp//\n/BusXu3R3Z3CxfmQ/9/fwPz8bZ8MNKrqAQARmQ1MBRqS5pkKPAOgqjUiMkpELgIm+Bib94LBIIFA\nIOXv290NDQ3w0kuwdauyenUnL79cyogR+2lvf5aSknq6uq7C875Id3dVypf/Z0EgkMb3dytd31/2\nCJL6768M+BzhMMBO2trmMXduMc8+ewsiVzBhQjOBQBkf/ei5vP/9wqRJUFaW4ggJ+f/9DcxP0Y8F\nDiY9P0S8/AeaZ6zPsSbB8+Jr5G1t0NoKLS3Q1BT/efNNOHpU2b+/m717oxw4UERTUxkjRhxH9WVC\noQ2Ulu6muLiSpqYbgM8TDj/k+iMZA7yPWOx9xGIAe4Dfc/DgcX73u1J++9uJlJRcTyRyMaNHdzBu\nXJgJE4qZMGEYEyZUcOGFwpgxMGYMjBoFI0fGfyoqoMj2MPqWyt/fk0ma3nfIIhG4664oR44cAXrf\ncNjX9kRVUBVAkx6fmh5//ufH8eeeBydPtjJz5hFiMcHzhGi0CM8rIhotJhaL/0SjpXheCUVFYYqL\nQ0A70EosdgLPOwYcB44CbwAHKCkJoVpOS8sE4P3AjcRin0pK25T4yYQjwOYMLSvVDqTofRqBC1L0\nXpmW6e/verpO38vkBLHYc8DLNDe309xcxI4dw4GLid/85HzgPIqKLkBkFDASzxuBailFRRGKi6OU\nlEQpLo5SXOwl/ShFRUpxsdLa2sIzzxxEBEQ08d/kHwXk9Gun+Hnc08iRMVatuphhw4al5P9UqogO\nsJdERG4EqlR1SuL59wBV1f+XNM+vgT+p6n8nnjcAHyW+6abfsUnvYbtrjDFmkPTUGmc//KzR1wET\nReRS4quU9wDTesyzAPgW8N+JfxiaVfWYiJzwMdZ3WGOMMYM3YNGrakxE7gOWEz9K5zequltEpsdf\n1qdUdYmI3CkirwIh4Cv9jU3bpzHGGPM2A266McYYk9uyar+1iNwvIrtF5CUR+bnrPOkgIn8nIp6I\njHGdJZVE5JHEd7dNROaKyDmuMw1VPp/sJyLjRGS1iOxK/H17wHWmdBCRIhHZIiILXGdJtcRh7M8l\n/t7tEpEP9TVv1hS9iASAu4CrVfVq4J/dJko9ERlH/EpQqTrUI5ssB65S1WuJH4LyfxznGZICONkv\nCnxXVa8CbgK+lWef75RvAy+7DpEmjwFLVPUK4Bqgz83iWVP0wDeBn6tqFEBVTzjOkw6PAv/bdYh0\nUNWVquolnlYD41zmSYHTJwqqagQ4dbJfXlDVN1R1W+JxO/GSGOs2VWolVqzuBGa6zpJqid+Yb1XV\npwFUNaqqfV5dKJuK/r3AR0SkWkT+JCI3uA6USiJyN3BQVV9ynSUDvgq86DrEEPV1EmDeEZF3AdcC\nNW6TpNypFat83BE5ATghIk8nNk09JSIVfc2crhOmeiUiK4CLkicR/xJ+kMhyrqreKCIfBJ4FLstk\nvqEa4PN9n/hmm+TXcko/n+8hVV2YmOchIKKqsxxENIMkIiOAOcC3E2v2eUFEPgUcU9Vtic3COff3\nbQAlwPXAt1S1XkT+Ffge8HBfM2eMqn6ir9dE5H8Bzyfmq0vssDxPVXPmxpR9fT4ReR/wLmC7iAjx\nzRqbRWSyqr6ZwYhD0t/3ByAi/5P4r8ofy0ig9DoMjE96Pi4xLW+ISAnxkv+9qs53nSfFbgbuFpE7\ngQpgpIg8o6pfcpwrVQ4R30JQn3g+B+jzgIFs2nQzj0RBiMh7gdJcKvn+qOpOVX2Hql6mqhOIf0nX\n5VLJD0REphD/NfluVc3w9QnT4vSJgiJSRvxkv3w7cuO3wMuq+pjrIKmmqt9X1fGqehnx7251HpU8\nqnoMOJjoSoC/oJ+dzhldox/A08BvReQloJv4tXTzVfziGvnlceKXLFwR/6WFalX9W7eRzl6+n+wn\nIjcDfwO8JCJbif+Z/L6qLnWbzAzCA8AfRKQU2EfiRNXe2AlTxhiT57Jp040xxpg0sKI3xpg8Z0Vv\njDF5zoreGGPynBW9McbkOSt6Y4zJc1b0xhiT56zojTEmz/1/YR5VmX3WthgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10baed048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_between_z = arange( -abs( z ), abs( z ), 0.1 )\n",
    "x_above_z = arange( abs( z ), 5, 0.1 )\n",
    "x = concatenate( [x_below_z, x_between_z, x_above_z] )\n",
    "# Calculate the probability density function for a normal distribution over this range\n",
    "y = scipy.stats.norm.pdf( x )\n",
    "plot( x, y )\n",
    "fill_between( x_below_z, scipy.stats.norm.pdf( x_below_z ) )\n",
    "fill_between( x_above_z, scipy.stats.norm.pdf( x_above_z ) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.LineCollection at 0x10bae5898>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGX1JREFUeJzt3XmUVOWZx/HvgwiDETyGRNFuEBUXXEliEMclnWiGxSCo\nMYKMDhhncMsYM2fiMpmRk2TiJBMTFxRDBBOcKIZVRFREqZlgwuKGJDYCMmKDwgQFUWRpmmf+uIW0\nHbq7uvtWvXXv/X3OqWNX1du3nrKrf7z93Pvea+6OiIgkX7vQBYiISDwU6CIiKaFAFxFJCQW6iEhK\nKNBFRFJCgS4ikhLNBrqZTTCzDWb2ahNj7jazlWb2ipn1ibdEEREpRCEz9AeB/o09aWYDgaPd/Rhg\nNHB/TLWJiEgLNBvo7r4A2NTEkCHApPzYRcBBZnZoPOWJiEih4uihVwA19e6vyz8mIiIlpJ2iIiIp\n0T6GbawDute7X5l/7C+YmU4cIyLSCu5uzY0pdIZu+du+zAKuADCzfsBmd9/QRFGpvd12223Ba9D7\n2/etkM9ec+8vyZ/fxt7brl3OkiXOT37ifP3rTs+eTufOTt++zt/9nfOjHzmPPOL8/vdOTY2zc2f4\n99Kan13Sb4VqdoZuZg8DVUBXM3sLuA3oEH22fby7zzGzQWa2CtgKjCr41UWk5DZvhieegBkz4Nln\n4fDD4ctfhqFD4Yc/hGOOgXZqxiZSs4Hu7pcVMOb6eMoRkWLYvTsK8QceiEK8qgouvBDGjoVu3UJX\nJ3GJo4cueVVVVaFLKCq9v+T58EO4/34YN66Kp5+Gv/97+PWvoUuX0JXFK40/u9awlvRn2vxiZl7K\n1xPZw8xa1Iss1jZKZetW+NnP4J57onbKzTfD5z4XuipprfxnL7adoiKSAO7w8MNw/PFQXQ0LFsCj\njyrMs0ItF5GUWL0aRo2KZueTJ8OZZ4auSEpNM3SRhHOHCRPg9NNhyBBYtEhhnlWaoYsk2Nat0ax8\nxQqYPx9OOil0RRKSZugiCfXWW3DWWXDAAdGsXGEuCnSRBFq8GPr1g8svhwcfhI4dQ1ck5UAtF5GE\n+d3v4OKLo7754MGhq5FyokAXSZB58+Cyy6JDE887L3Q1Um4U6CIJkcvB8OEwbRqcc07oaqQcqYcu\nkgDLlsE3vhEtElKYS2MU6CJl7q23YNAguPtu+MpXQlcj5UyBLlLGPvwwCvMbb4Rhw0JXI+VOJ+eS\nTEjiybncoxDv3Dk67a1kV6En59JOUZEyddddsGoVPP986EokKRToImVowQK4/XZYuBD+6q9CVyNJ\noR66SJl5/30YMQImToQjjwxdjSSJeuiSCUnqoY8cCZ06wbhxRX8pSQj10EUS6LHHoqX9S5eGrkSS\nSIEuUib+/Ge4+mr47W/hwANDVyNJpJaLZEISWi4jRkC3bnDHHUV7CUkotVxEEiSXi45see210JVI\nkukoF5HAamvhuuvg5z+HT30qdDWSZAp0kcDuvBN69IALLwxdiSSdeuiSCeXaQ1+7Fvr0iRYQ9eoV\n66YlRQrtoSvQJRPKNdBHjYLDDoMf/SjWzUrKaKeoSJlbtgzmzIEVK0JXImmhHrpIIDffDLfeCgcd\nFLoSSQvN0EUCyOWguhqmTw9diaSJZugiJeYO3/0u/Pu/Q8eOoauRNFGgi5TY7NmwcydcemnoSiRt\nFOgiJeQO3/8+/Nu/QTv99knM9JESKaGnnoLt22Ho0NCVSBop0EVKZM/s/Hvf0+xcikMfK5ESefZZ\n2LwZvv710JVIWhUU6GY2wMyWm9kKM7tpH893MbNZZvaKmS0zs5GxVyqScD/4QTQ732+/0JVIWjW7\n9N/M2gErgHOBt4ElwDB3X15vzC1AF3e/xcw+A7wOHOruuxpsS0v/JYjQS/8XL4ZvfANWrYL2Wv0h\nLVTo0v9CZuh9gZXuvsbda4HJwJAGYxzonP+6M/BuwzAXybI77oBvf1thLsVVSKBXADX17q/NP1bf\nWOAEM3sbWArcEE95Isn35ptR//yb3wxdiaRdXPOF/sDL7v4VMzsaeMbMTnH3DxsOHDNmzMdfV1VV\nUVVVFVMJIuXprrvgyiuhc+fmx4oA5HI5crlci7+vkB56P2CMuw/I378ZcHf/cb0xs4Hb3f35/P1n\ngZvc/YUG21IPXYII1UPfvBmOOgpefRUqK9v08pJhcfbQlwC9zOwIM+sADANmNRizBjgv/8KHAscC\nq1tWskj6/PKXMGiQwlxKo9mWi7vXmdn1wFyifwAmuHu1mY2OnvbxwA+BX5nZq/lv+667v1e0qkUS\noK4O7r0Xpk4NXYlkha5YJJkQouUye3Z07PmiRW16WZFYWy4i0gr33QfXXhu6CskSzdAlE0o9Q3/j\nDejXD956Czp1atPLimiGLhLSL34BI0cqzKW0NEOXTCjlDH3bNujRA/7wB+jVq00vKQJohi4SzJQp\n8PnPK8yl9BToIjF74AEYPTp0FZJFarlIJpSq5bJiBZx9NtTUQIcObXo5kY+p5SISwMSJcPnlCnMJ\nQzN0yYRSzNB37YLu3eG556B37za9lMgnaIYuUmJz5sCRRyrMJRwFukhMJk6Eq64KXYVkmVoukgnF\nbrmsXx/NzGtq4MAD2/QyIn9BLReREnr4YRg6VGEuYSnQRWIwaRJccUXoKiTrFOgibbRsGbz3Hnzp\nS6ErkaxToIu00UMPwYgR0E6/TRKYdopKJhRrp2hdXXQirmeegRNOaNPmRRqlnaIiJTB/PnTrpjCX\n8qBAF2mDSZOipf4i5UAtF8mEYrRctm6Figp4/XU49NC2VijSOLVcRIps9uzoMnMKcykXCnSRVnrk\nERg+PHQVInup5SKZEHfLZfNmOOKI6CLQBx0UR4UijVPLRaSIZsyAc89VmEt5UaCLtILaLVKO1HKR\nTIiz5bJhAxx3HLz9NhxwQEwFijRBLReRIpkyBb72NYW5lB8FukgLTZ6sdouUJ7VcJBPiarnU1Din\nnBJd0EIXgpZSUctFpAimTYMhQxTmUp4U6CItMGUKXHJJ6CpE9k0tF8mEeFoulRx88Fq1W6Tk1HIR\nid3FardIWVOgixTsErVbpKyp5SKZ0NaWy7p1UFn5Ljt2dNUMXUou1paLmQ0ws+VmtsLMbmpkTJWZ\nvWxmfzSz+S0tWKScTZ0KMEthLmWt2UA3s3bAWKA/cCIw3MyObzDmIOBe4GvufhKgP0wlVaZNA5ga\nugyRJhUyQ+8LrHT3Ne5eC0wGhjQYcxkwzd3XAbj7xnjLFAln/XpYtgxgXuhSRJpUSKBXADX17q/N\nP1bfscCnzWy+mS0xM11lUVJj5kwYOBBgZ+hSRJoU11Eu7YHPAwOBAcC/mlmvmLYtEtS0aXDxxaGr\nEGle+wLGrAN61LtfmX+svrXARnffDmw3s/8BTgVWNdzYmDFjPv66qqqKqqqqllUsUkLvvguLFkWz\ndJFSyeVy5HK5Fn9fs4ctmtl+wOvAucA7wGJguLtX1xtzPHAP0ey8I7AIuNTdX2uwLR22KEG09rDF\nX/0KZs2C6dPjWW0q0hqFHrbY7Azd3evM7HpgLlGLZoK7V5vZ6OhpH+/uy83saeBVoA4Y3zDMRZJo\n2jQYNix0FSKF0cIiyYTWzK63bIHKSqipia4dqhm6hKJzuYi00Zw5cNZZuhC0JIcCXaQRM2bARReF\nrkKkcGq5SCa0tF2yfTt06wYrVsAhh7RuGyJxUctFpA3mzYNTTtkb5iJJoEAX2Qe1WySJ1HKRTGhJ\nu2TXLjjsMFiyBHr2bN02ROKklotIKy1YAN27fzLMRZJAgS7SgNotklSFnMtFJDPco/O2zJkTuhKR\nltMMXaSel16KLgJ9wgmhKxFpOQW6SD0zZ8KFF4I1u/tJpPwo0EXq2RPoIkmkQBfJW7UKNm6E008P\nXYlI6yjQRfJmzoQLLoB2+q2QhNJHVyRP7RZJOq0UlUxobpXnhg1w3HHRfzt2bN02RIpFK0VFWmDW\nLBgwoPEwF0kCBboIardIOqjlIpnQVLvkgw+gomLvpeZasw2RYlLLRaRATz4JZ56pS81J8inQJfPU\nbpG0UMtFMqGxdsnOnXDooVBdHV1yrjXbECk2tVxECjB/PvTu3XyYiySBAl0ybeZMGDo0dBUi8dD5\n0CWzdu+Gxx6D//7v0JWIxEMzdMmsxYvh4IPhmGNCVyISDwW6ZJbaLZI2CnTJJPfo2qE6XFHSRIEu\nmVRdDdu2wRe+ELoSkfgo0CWTZsyI2i261JykiQJdMkntFkkjBbpkzpo18OabcPbZoSsRiZcCXTJn\n5kwYPBjaaxWGpIwCXTJH7RZJK52cSzJhz4m1/vxn6NUL1q+HTp1atw2RUtPJuUT24fHH4atfbXmY\niyRBQYFuZgPMbLmZrTCzm5oY90UzqzWzi+IrUSQ+06bBxReHrkKkOJptuZhZO2AFcC7wNrAEGObu\ny/cx7hlgGzDR3afvY1tquUgQZsbmzU737rB2LXTp0rpt6PMrIcTZcukLrHT3Ne5eC0wGhuxj3LeA\nqcD/tahSkRJ54gk455zWhblIEhQS6BVATb37a/OPfczMDgeGuvs4QGvvpCyp3SJpF9dO0TuB+r11\nhbqUmQOYNw8uuCB0HSLFU8jSinVAj3r3K/OP1XcaMNnMDPgMMNDMat19VsONjRkz5uOvq6qqqKqq\namHJIq3Rn759oWvX0HWINC+Xy5HL5Vr8fYXsFN0PeJ1op+g7wGJguLtXNzL+QeBx7RSVcmL2G+67\nbwTXXNOWbWinqIRR6E7RZmfo7l5nZtcDc4laNBPcvdrMRkdP+/iG39KqikWKZMcOgEFaHSqpp5Wi\nknqzZ8Pgwb/DvW1n49IMXULRSlGRvClTAKaELkOk6DRDl1TbsQMOOww2barAveG+/JbRDF1C0Qxd\nBHjmGTjxRIgWOYukmwJdUm3KFLjkktBViJSGWi6SWnvaLcuWQWVl29slarlIKGq5SObtabdUVDQ/\nViQNFOiSWmq3SNao5SKpVL/dUlERT7tELRcJRS0XybSnn4aTTlK7RbJFgS6p9MgjMHx46CpESkst\nF0mdrVvh8MNh1Sr47Gejx9RykSRTy0Uy6/HH4Ywz9oa5SFYo0CV11G6RrFLLRVJl0ybo2RNqaj55\n7VC1XCTJ1HKRTJo+Hc47TxeClmxSoEuqTJ6sdotkl1oukhrr1sHJJ0f/7dTpk8+p5SJJppaLZM7D\nD8NFF/1lmItkhQJdUsEdJk2Cyy8PXYlIOAp0SYWlS2HLFji7bZcNFUk0BbqkwkMPRbPzdvpES4Zp\np6gk3q5d0L075HJw3HH7HqOdopJk2ikqmTFvXhTojYW5SFYo0CXxtDNUJKKWiyTapk1w5JHwxhvQ\ntWvj49RykSRTy0Uy4Te/gQEDmg5zkaxQoEuiTZgA3/xm6CpEyoMCXRLrpZeilsu554auRKQ8KNAl\nsSZMgFGjdOy5yB7aKSqJtG0bVFbCyy9Djx7Nj9dOUUky7RSVVJs+HU47rbAwF8kKBbok0rhxcPXV\noasQKS8KdEmcpUthzRoYPDh0JSLlRYEuiXPfffAP/wDt24euRKS8aKeoJMr770cXga6uhm7dCv8+\n7RSVJNNOUUmlX/8a+vdvWZiLZEVBgW5mA8xsuZmtMLOb9vH8ZWa2NH9bYGYnx1+qZJ171G659trQ\nlYiUp2YD3czaAWOB/sCJwHAzO77BsNXAOe5+KvBD4JdxFyry7LOw//66KpFIYwqZofcFVrr7Gnev\nBSYDQ+oPcPeF7v5+/u5CoCLeMkXgjjvgxhvBmu0kimRTIYFeAdTUu7+WpgP7KuDJthQl0tCf/gSv\nvAIjRoSuRKR8xXrgl5l9GRgFnNXYmDFjxnz8dVVVFVVVVXGWICn1s5/BdddBx46hKxEpvlwuRy6X\na/H3NXvYopn1A8a4+4D8/ZsBd/cfNxh3CjANGODubzSyLR22KC22fj2ccAKsXNn6857rsEVJsjgP\nW1wC9DKzI8ysAzAMmNXgxXoQhfnljYW5SGvdey8MG6aLWIg0p9mWi7vXmdn1wFyifwAmuHu1mY2O\nnvbxwL8CnwbuMzMDat29bzELl2z48EP4xS/g+edDVyJS/rRSVMraT38KS5bAo4+2bTtquUiSFdpy\nUaBL2froIzj6aJg7F05u41I1BbokmZb+S+KNHw9//ddtD3ORrNAMXcrS9u3R7PyJJ6BPn7ZvTzN0\nSTLN0CXRJkyIrkgUR5iLZIVm6FJ2PvoIjj0WZs6MQj0OmqFLkmmGLol1551R7zyuMBfJCs3Qpaxs\n3AjHHw8LF0KvXvFtVzN0STIdtiiJ9O1vw65dMHZsvNtVoEuSKdAlcVavhi9+Mbq83CGHxLttBbok\nmXrokjg33QQ33BB/mItkha6bLmVh7lx48UWYNCl0JSLJpRm6BLd9e3Su83vugU6dQlcjklwKdAnu\nP/8TTjoJzj8/dCUiyaadohLU6tXQt2/UbjniiOK9jnaKSpJpp6iUvd274coro52hxQxzkaxQoEsw\nd98dHXP+ne+ErkQkHdRykSCWL4ezzop/RWhj1HKRJFPLRcpWbS1ccQX84AelCXORrFCgS8ndcgt8\n5jNw9dWhKxFJFy0skpKaOhWmTYMXXgBr9g9IEWkJBbqUTHU1XHMNPPUUdO0auhqR9FHLRUpi0ya4\n6CK4/Xb4whdCVyOSTjrKRYpu+3bo3z+6nNxdd4WpQUe5SJLp9LlSFurq4NJLYb/94JFHoF2gvwkV\n6JJkhQa6euhSNO7wrW/Be+/Bk0+GC3ORrFCgS1HU1UU7QP/4xyjMO3YMXZFI+inQJXa7dsGoUbB2\nbXSe8wMPDF2RSDYo0CVWW7bAiBFRqM+Zo/Obi5SSupoSm1Wr4IwzoLISZs1SmIuUmgJdYjF7Npx5\nJlx/PYwbB/vvH7oikexRy0Xa5IMPotPfzpsXLes/++zQFYlkl2bo0mpz5kSLhXbvhqVLFeYioWmG\nLi1WXR3Nyv/3f2HsWBg4MHRFIgKaoUsLLFsWHcFyzjnwN38Dr76qMBcpJwp0aVJtLTz2GJx/fhTi\np54Kb7wBN94IHTqErk5E6iuo5WJmA4A7if4BmODuP97HmLuBgcBWYKS7vxJnoVI6dXXw+9/DjBnR\n+VeOPhquuira6alDEUXKV7MzdDNrB4wF+gMnAsPN7PgGYwYCR7v7McBo4P4i1Fr2crlc6BJaxR1W\nr4YJE+Bv/xYOPzw6/LBLF3juOViwAEaOhEWLcqFLLaqk/vwKkeb3Bul/f4UqpOXSF1jp7mvcvRaY\nDAxpMGYIMAnA3RcBB5nZobFWmgBJ+FB98AG8+CL813/BrbfCgAFwyCHRBZvnzYv643/4Q3TUypgx\n0Lv33u9NwvtrizS/vzS/N0j/+ytUIS2XCqCm3v21RCHf1Jh1+cc2tKk6aZQ77NgBH30EW7dGQb1l\nC7z/Prz7bnSGw40b4Z13YP366Lwqb74J27bBMcdEQd27N1x7LZx2WjQrF5FkS+1hi089Bffe27rv\nbckpr+uPXbECFi/+5GPue+/v+XrPbffuvf/dc6uri86DUle39+va2ui2c2cU4tu3R7f27eFTn4pu\nnTtHLZIuXaLLu3XtGl2IuU8f6NYNKiqgZ0/47Gd1LU+RtGr2Ahdm1g8Y4+4D8vdvBrz+jlEzux+Y\n7+6P5u8vB77k7hsabEtXBxARaYW4LnCxBOhlZkcA7wDDgOENxswCrgMezf8DsLlhmBdakIiItE6z\nge7udWZ2PTCXvYctVpvZ6OhpH+/uc8xskJmtIjpscVRxyxYRkYZKek1REREpniArRc3sW2ZWbWbL\nzOw/QtRQbGb2T2a228w+HbqWOJnZT/I/u1fMbJqZdQldU1uZ2QAzW25mK8zsptD1xMnMKs3sOTP7\nU/737R9D1xQ3M2tnZi+Z2azQtRSDmR1kZlPyv3d/MrPTGxtb8kA3sypgMHCyu58M/LTUNRSbmVUC\nXwXWhK6lCOYCJ7p7H2AlcEvgetqkkIVzCbcL+I67nwicAVyXsvcHcAPwWugiiuguYI679wZOBaob\nGxhihn4N8B/uvgvA3TcGqKHYfg78c+giisHd57n77vzdhUBlyHpiUMjCucRy9/V7TsPh7h8ShUFF\n2Krik588DQIeCF1LMeT/Aj7b3R8EcPdd7r6lsfEhAv1Y4BwzW2hm883stAA1FI2ZXQDUuPuy0LWU\nwJXAk6GLaKN9LZxLTeDVZ2Y9gT7AorCVxGrP5CmtOwOPBDaa2YP5ttJ4M2v0jEpFWVhkZs8A9Zf+\nG9H/8O/lX/Ngd+9nZl8EfgscVYw6iqWZ93crUbul/nOJ0sT7+xd3fzw/5l+AWnd/OECJ0kJmdiAw\nFbghP1NPPDM7H9jg7q/kW7mJ+10rQHvg88B17v6Cmd0J3Azc1tjg2Ln7Vxt7zsyuBqbnxy3J7zjs\n6u7vFqOWYmjs/ZnZSUBPYKmZGVE74kUz6+vu/1fCEtukqZ8fgJmNJPoz9yslKai41gE96t2vzD+W\nGmbWnijMH3L3x0LXE6MzgQvMbBDQCehsZpPc/YrAdcVpLdFf/C/k708FGt1xH6LlMpN8EJjZscD+\nSQrzprj7H929m7sf5e5HEv0wPpekMG9O/lTK/wxc4O47QtcTg48XzplZB6KFc2k7WmIi8Jq73xW6\nkDi5+63u3sPdjyL6uT2XsjAnv0CzJp+VAOfSxA7gEOdyeRCYaGbLgB1Aqn4ADTjp+zPwHqAD8Ez0\nRwgL3f3asCW1XmML5wKXFRszOxMYASwzs5eJPpO3uvtTYSuTFvhH4Ddmtj+wmiYWbmphkYhISugS\ndCIiKaFAFxFJCQW6iEhKKNBFRFJCgS4ikhIKdBGRlFCgi4ikhAJdRCQl/h9fuZ8fKopbqwAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10bae5908>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Integrate the two shaded areas using the cumulative distribution function\n",
    "plot( x, scipy.stats.norm.cdf( x ) )\n",
    "vlines( [-abs( z ), abs( z )], ymin = 0, ymax = 1 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.76594790118137279"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "left_region = scipy.stats.norm.cdf( -abs( z ) )\n",
    "right_region = 1 - scipy.stats.norm.cdf( abs( z ) )\n",
    "p_value = left_region + right_region\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the mythical p-value, which we can of course calculate directly using a built-in test as well (probably not significant in this case):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7659479011813729"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Multiply by two since the built-in test is unidirectional by default\n",
    "scipy.stats.norm.sf( abs( z ) ) * 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T-tests\n",
    "\n",
    "Of course, most of the time we *don't* know the population distribution, and we must instead deal with a greater degree of uncertainty.  **T-tests** are a series of tests made for determining whether two samples are drawn from the same population distribution (or not) when nothing about the population is known with certainty.  This is anologous to the difference between the \"more certain\" population standard deviation (similar to our z-test above):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "27.741845648766773"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum_of_squares = sum( ( d - mean( bio508_scores ) )**2 for d in bio508_scores )\n",
    "number_of_measurements = len( bio508_scores )\n",
    "sqrt( sum_of_squares / number_of_measurements )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and the larger, \"less certain\" sample standard deviation corresponding to a broader distribution (similar to the uncertainty in a t-test):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29.242472915644843"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sqrt( sum_of_squares / ( number_of_measurements - 1 ) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A variety of t-tests can be used depending on how much you know about your population and measurements, including:\n",
    "\n",
    "* A [one-sample t-test](http://en.wikipedia.org/wiki/T-test#One-sample_t-test) when you're comparing a set of measurements to a fixed reference point.\n",
    "\n",
    "* A [paired t-test](http://en.wikipedia.org/wiki/T-test#Paired_samples) when you're comparing a sample to itself at a different point in time (and thus lose some independence).\n",
    "\n",
    "* [Welch's t-test](http://en.wikipedia.org/wiki/T-test#Equal_or_Unequal_sample_sizes.2C_unequal_variances) when you're comparing two samples that might possess different measurement errors.\n",
    "\n",
    "We'll demonstrate here a [standard (or *unpaired*) t-test](http://en.wikipedia.org/wiki/T-test#Unequal_sample_sizes.2C_equal_variance) in which you'd simply like to determine whether two samples made using identical measurement strategies are likely to have arisen from the same population.  A t-test is applicable to the same types of data as a z-test, for example test scores, here drawn from two different classes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[65, 88, 74, 75, 68, 52, 75, 53, 54, 89, 53, 86, 64, 57, 80]\n"
     ]
    }
   ],
   "source": [
    "# Sample 15 integers between 50 and 90 inclusive\n",
    "bio508_scores = [random.randint( 50, 90 ) for i in range( 15 )]\n",
    "print( bio508_scores )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[64, 46, 45, 63, 61, 45, 46, 74, 53, 68, 78, 48, 50, 70, 45, 77, 51, 79, 57, 52, 49, 63, 65, 48, 57]\n"
     ]
    }
   ],
   "source": [
    "# Sample 25 integers between 45 and 85 inclusive\n",
    "bio200_scores = [random.randint( 45, 85 ) for i in range( 25 )]\n",
    "print( bio200_scores )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[68.86666666666666, 12.85750969494309]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[mean( bio508_scores ), std( bio508_scores )]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[58.159999999999997, 11.130786135758786]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[mean( bio200_scores ), std( bio200_scores )]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To compare these two samples, we'd like to compute a summary statistic like the z-statistic that we used above - recall that this was just the difference between two sample means \"normalized\" by the variability (standard error) of the data:\n",
    "\n",
    "> $$Z = \\frac{\\bar{x}_{sample}-\\bar{x}_{population}}{\\sigma_{population}}$$\n",
    "\n",
    "However, since we can no longer assume that we know the population standard error with certainty, we replace our measure of variability with something more complicated to form the **t-statistic**:\n",
    "\n",
    "> $$T = \\frac{\\bar{x}_1-\\bar{x}_2}{S_{1,2}}$$\n",
    "\n",
    "For those interested in the gory details, our new measure of *common standard deviation* is really just a generalization of the population standard deviation:\n",
    "\n",
    "> $$S_{1,2} = \\sqrt{\\frac{(|x_1|-1)S^2_1+(|x_2|-1)S^2_2}{|x_1|+|x_2|-2}}\\sqrt{\\frac{1}{|x_1|}+\\frac{1}{|x_2|}}$$\n",
    ">\n",
    "> $$S^2_x = \\sum{(x-\\bar{x})^2}$$\n",
    "\n",
    "but don't worry about this if the parallel isn't clear from the formulas.  Much easier to see this demonstrated, since the reason this is referred to as a t-statistic is that if all of our assumptions above hold, then under the null hypothesis, it's drawn from a **t-distribution** just like our z-statistic is drawn from a normal distribution.\n",
    "\n",
    "What difference does this make?  Two important ones, for our purposes:\n",
    "\n",
    "1. A t-distribution is **heavy tailed**, which means it spreads further than a normal distribution.  This means that it samples from a wider range of values than a normal distribution - not all of the function's mass is close to its mean.  This is how the increased uncertainty captured by a t-test plays out numerically.\n",
    "\n",
    "2. A t-distribution has a parameter called the **degrees of freedom** that corresponds to the size of our sample.  For very small samples, a t-distribution with fewer degrees of freedom spreads wider and is much heavier tailed than a normal distribution - we're very uncertain what a small sample tells us about an underlying population.  For very large samples, a t-distribution with many degrees of freedom starts to look exactly like a normal distribution - we have a good estimate of what the population looks like, so we're more certain, the extra mathematical tricks become less necessary, and any test starts to look like a z-test again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x10bfc4c18>"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEACAYAAAC9Gb03AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8VFX6+PHPmZn0XiAJCSl0aVKT0BNQghRBlFVUrItl\n17rN/X5/q4L61V11q+6uXXdtuLZVRAURAgkhhd5L6J1AEtLbzPn9cWfCJJkkkzCZmSTn/XrlNTP3\nnjvzTCDPnHnuuecIKSWKoihK16VzdQCKoihKx1KJXlEUpYtTiV5RFKWLU4leURSli1OJXlEUpYtT\niV5RFKWLsyvRCyFmCCH2CSEOCCGeaKHdWCFErRBifluPVRRFUTqGaG0cvRBCBxwApgGngTzgFinl\nPhvtfgAqgXeklF/Ye6yiKIrScezp0ScCB6WUx6SUtcAyYK6Ndg8DnwHn23GsoiiK0kHsSfTRwAmr\nxyfN2+oJIXoB86SU/wREW45VFEVROpajTsb+BVD1d0VRFDdksKPNKSDW6nGMeZu1McAyIYQAwoHr\nhBB1dh4LgBBCTbqjKIrSRlJKYU+jFn8APZAPxAGewDbgqhbavwvMb+uxWihd09NPP+3qEDpUV3p/\nX38tpU4nJUg5cKCUv/+9lNOmPS3j4rRter2Uq1a5OkrH6kr/frZ05fdnzput5vFWSzdSSiPwELAK\n2A0sk1LuFULcL4S4z9YhrR3b6qePorjA1q2wcCGYTPD007B3LzzxBEycCIcOwW9+A0YjLFgA+9S4\nMaUTsad0g5Tye2Bgo22vN9P2ntaOVRR3c+oUzJ4N5eWwaJGW6IXVF2K9Hl54AQ4ehC+/hDlzICcH\nQkNdF7Oi2EtdGesEKSkprg6hQ3WF9/foo3D6NEyZAm++eTnJ7ynYQ9zVcQDodPDvf8OIEZCfD488\n4sKAHagr/Pu1pKu/P3u0esGUswghpLvEonQvmzfDmDHg7a2VaMJ6VrNs1zL+uemf5JzKAWBC7wn8\nYtwvmDtwLseO6hk0COrqYMcOGDrUxW9A6baEEHadjFWJXun2rrsOvv8efv1reO6FGq59/1rWH1sP\nQJBXEEIIiquKAbimzzV8e+u3PP6oB3//O8yfD59/7sroO058fDzHjh1zdRgKEBcXx9GjR5tsV4le\nUeywfr1WrgkIgCNH4HcbH+S1za8R5R/Fc1Of4+YhNyORvLftPZ5Z9wwFFQX8ctwv+eWwl+nTB6qq\nYNMmGD3a1e/E8cxJxNVhKDT/b6ESvaK0QkqYPBkyM2HpUoia9Sb3fXMfXnovMu7OYGz02AbtM49n\nkvqvVOpMdXy64FNy3r2Jl1/WvhF8+62L3kQHUonefVxpolcnY5Vu68cftSQfFgaTbsnl59/+HIDX\nZr/WJMkDTIydyMvXvgzA3V/dzQ2L9+LvD999B7m5Tg1dUdpEJXql23rjDe320UclT2b+glpTLQ+N\nfYi7RtzV7DGPJD3CzUNupqymjKeyH+aBB7Ttb77Z8fEqSnup0o3SLV28CL16aSNnPt64lpu/m0qo\nTyhHHz1KgFdAi8cWVRYR95c4SmtK+XjaRhZOSsbfH86eBT8/J70BJ1Clm/ZbunQp+fn5vP/++w55\nPlW6UZR2+OgjqKmB6dPhn3ufAeDx5MdbTfIAIT4hPJT4EADvH3+WceOgrAw++6xDQ1YaiY+PJyIi\ngsrKyvptb7/9NqmpqS6M6jIhWp+CxllUole6pXfe0W6Tb84g/Wg6QV5BPJz4sN3HP578OL4evnx7\n8Fum3b65wXMqziGEwGQy8Ze//KXJ9rbq6t9cVKJXup2tW2HbNm36ggzxLACPJT9GkHeQ3c/Rw68H\nD455EIDtQf+Hr682VPPgwQ4JWWnGr3/9a/74xz9SUlLSZF9WVhaJiYmEhISQlJTExo0b6/elpqby\nu9/9jokTJ+Ln58eRI0dITU3lySefZMKECQQEBDB37lwKCwu5/fbbCQoKIikpiePHj9c/x2OPPUZs\nbCxBQUGMHTuWzMxMp7zn9lCJXul23n1Xu512Ry4/Hv2BAM8AHk16tM3P86vxv8Lb4M3y/C+Zftse\nAN57z4GBujEhHPdzJcaMGUNKSgovvfRSg+1FRUXMnj2bxx57jIsXL/L4448za9YsioqK6tt88MEH\nvPXWW5SWlhIbq82m/sknn/Dhhx9y+vRp8vPzGT9+PPfeey9FRUUMGjSIpUuX1h+fmJjIjh07KCoq\n4tZbb2XBggXU1NRc2RvqICrRK91KdTV8+KF2v+7qtwC4f/T9hPiE2D7g2DHtSiobIv0juWP4HQD4\njHsP0BK90ejIiJXWLF26lFdffZWLFy/Wb1uxYgUDBgzg1ltvRafTccsttzBo0CCWL19e3+auu+5i\n0KBB6HQ6DAZtfse7776b+Ph4AgICuO666+jbty+pqanodDoWLFjA1q1b64+/9dZbCQ4ORqfT8fjj\nj1NdXc3+/fud98bbQCV6pVv54QcoLIShI6pYc/Y/ALaHU2Znww03QEIC9OkDN94Iu3Y1aXbniDsB\nWFf0IQl9jZw+DVYVgi5Lm53fMT9XasiQIcyePZsXXnjBHJvk9OnTxMXFNWgXFxfHqVOX1z3q3bt3\nk+eKiIiov+/j49PkcVlZWf3jl19+mcGDBxMSEkJISAglJSVcuHDhyt9QB1CJXulWvvpKux08bwWX\nqi8xInIEQ3oOadjo4Ydh3Dj473/B0xO8vOCLL2D4cPif/2nQdFzMOPqG9OV06WlGzV8LaIcpzrVk\nyRLefPNNTp06hRCC6OjoJnPDHD9+nOjoy0tWX8momIyMDF566SU+++wzioqKKCoqIjAw0G1P6qpE\nr3QbJhNYvrmfj9TGNy8avqhho08/hVdf1ZL7//wPHD0Khw/DQw9pk9L//veXnwQtWdw+/HYAyvpq\nz/nVV47pqSr269u3LzfffDN/+9vfALjuuus4ePAgy5Ytw2g08sknn7B3717mzJnjkNcrKyvDw8OD\nsLAwampqeOaZZygtLXXIc3cEleiVbiMnB86dg5gBF9lw/lt0QsfCoQsvNzh5Eu6/X7v/xz/C889D\nZKR2ZdUrr8Af/qDt++lPoaCg/jBLos+8+DlhkeXk52urUykdq3GP/KmnnqKiogIhBKGhoXzzzTe8\n/PLLhIeH8/LLL7NixQpCQkJsHtvctuakpaWRlpbGgAEDSEhIwNfX12YpyG3Ys96gM37owmvGKu7h\niSe0qnDKr/4hWYKc/v70yzuNRimnTtUazJwppcnU9AmMRimnTNHazJ/foM24t8ZJliAn/fx9CVI+\n/3zHv5+Opv4m3Udz/xY4as1YRekqLPX581E2yjavvw5r1kCPHtqVT7Z6dzqdNqzG31+r2S9bVr/L\n8lylCdpzqzq94k7sSvRCiBlCiH1CiANCiCds7L9eCLFdCLFVCJErhJhgte+o9T5HBq8o9jpwQFvQ\nOyD2CHtKN+Ln4ccNg27QdhqN8OKL2v1XXwWrkRZNxMfDn/6k3X/mGa3wD9w89GY8dB7sKF+NV+h5\ncnO1pQkVxR20muiFEDrgVSANGAIsFEIMatRstZTyainlSOBe4C2rfSYgRUo5UkqZ6KC4FaVNLL35\nAbO1E6mzBszCz9M8A9nXX2snXS3DKFtz110QE6N9cvzwAwChPqFM6zMNkzRx1fXfAQ3O2SqKS9nT\no08EDkopj0kpa4FlwFzrBlLKCquH/mjJ3ULY+TqK0mEsib424RsA5gywGn3x5z9rt48+qo2saY2H\nB/xcm7uev/61fvPs/rMB0F+lvYYq3yjuwp4EHA2csHp80rytASHEPCHEXmA5cI/VLgn8IITIE0Is\nvpJgFaU9ioq0i5gMfqXsrUxHIJjRb4a2c/NmyMiAwEC4+277n3TxYvDx0VYd2bcP0L4lAOyvWwn6\nGtLTtaUGFcXVHNbTllL+V0p5FTAPeM5q1wQp5ShgJvBzIcRER72mothj7VqtlD4gbTW1plrG9R5H\nuG+4ttPSI7/3Xm3hWHuFhcHt2rBKXnkFgPjgeIb2HEpZbSl9UjOoqoINGxz4RhSlnQx2tDkFxFo9\njjFvs0lKmSmE6COECJVSFkopz5i3FwghvkQrBdmc5m3JkiX191NSUkhJSbEjPEVp2erV2q33cK2k\nYimxcOaMNnJGp9Ouhm2rRx/VlpZ67z147jkICWF2/9nsOr+LkMRvYPU0Vq+GadMc8z4UJT09nfT0\n9LYf2Nr4S0AP5ANxgCewDbiqUZu+VvdHASfM930Bf/N9P2ADML2Z13HAaFNFaWrAACkRRhn6fIRk\nCXLH2R3ajhdf1MbE33BD+5982jTtOV5/XUopZeaxTMkSZNQLfSWY5JgxDngDLqL+Jt1Hc/8WOGoc\nvZTSCDwErAJ2A8uklHuFEPcLIe4zN7tRCLFLCLEFeAX4iXl7BJAphNgKZAPLpZSr2v5xpCjtc/y4\nNrTSt+9mCmvOERsUy9CeQ7WdliWhFi1q/glac+ut2u2nnwKQHJNMmE8YZ6oP4RG1n82btUnUFPfx\nr3/9i0mTJtU/zsrKYsCAAQQGBvL111+7MLKOY1eNXkr5vZRyoJSyv5Ty9+Ztr0sp3zDff1FKOVRK\nOUpKOUFKudG8/YiUcoTUhlYOsxyrKM7y44/abcy0FYBWthFCaNMP5+Zqi7zOmNH+F5g3DwwG7UTA\nhQvodXpm9p8JQO9rvkFKbZfiOAEBAQQGBhIYGIher8fX17d+28cff2zXc1hPd/DUU0/xyCOPUFJS\nwvXXX++wOJ966imGDx+Oh4cHzzzzTJP9H330Uf2UyPPnz6e4uNhhr92YGvaodGmW+nxljFaft4yM\n4fPPtdtZs7TRM+0VGqoV4Y3G+vGUswdo5wBMfbXXNA+1VxyktLSUkpISSkpKiIuLY8WKFfXbFi5c\n2PoTNHLs2DEGDx7s8Dj79+/PSy+9xOzZs5vs2717Nw888AAffvgh586dw8fHhwcffNDhMVioRK90\nWVKaE71vASeMm/E2eJMab1442lK2uemmK38hy3OYyzfT+05HJ3ScFFngUV7/YaM4nrx8jq9ZhYWF\nXH/99QQFBZGcnMyhQ4fq9/Xr148jR44we/ZsAgMDqa2tdVhsixYtIi0tDX9//yb7PvroI66//nom\nTJiAr68vzz77LF988QXl5eUOe31r9oy6UZROadcuOH8eQiakUwRM6D0BHw8fbZbKjRu1nvzMmVf+\nQvPmwQMPaHWiixcJDgtjdNRo8k7n4XtVJod2pHHkiLaGSVchll7hGoBW5NMdO6fzz372M3x9fTl3\n7hyHDh0iLS2NPn36AJCfn09CQgLvvPMOqampNo+fM2cOmZmZCCGQUja4nThxYrvq+rt372bChPqZ\nYujTpw9eXl4cOHCAkSNHtu+NtkD16JUuy9KT7pG0BoBpCeZxjpayzcyZWo3+SoWHw9SpWvnGfAnu\n1ISpAERP1F7bcq5AcS6TycQXX3zBs88+i7e3N0OGDOHOO+9s0q6lbwXLly+nqKiIwsLCJrftPXlb\nVlZGUFDDxegDAwM7bE571aNXuixLci0JWwO1l5OvQ8s2FgsWaMX4Tz+Fe+5hWsI0/rDhD1T30hL9\n6tXaNPZdRUf3wh2loKAAo9FITExM/ba4uDgyMjJcGBX4+/tTUlLSYNulS5cIaMtFe22gevRKl2Q0\najMbEHiSs7UHCPAMYHSv0dpFUhs2aCtIzZrluBe84QZtnpzVq6G4mAmxE/DQeXCibjN4F5GRoVad\ncoUePXpgMBg4ceLyLC7Hjx9v03PMnDmzwUgf659Z7fw/NGTIELZv317/+NChQ9TW1jJgwIB2PV9r\nVKJXuqSdO6GkBMLHaGMbp8RPwaAzwMqVWsa95pq2TXnQmvBwGD8e6upgzRp8PXwZ13scEon/0HWc\nPg1Hjjju5RT76HQ65s+fz5IlS6isrGTPnj3861//atNzfPvttw1G+lj/rFixotnj6urqqKqqwmQy\nUVtbS3V1NSbztNa33XYby5cvZ8OGDZSXl/PUU09x44034ueIUqINKtErXZLlm3nQCK10MjXeXLZZ\nuVK7TUtz/ItantP8GpbX7JG4pkFMiuPYs/zfK6+8QmlpKVFRUdxzzz3cc889DfZfySLhLVm8eDG+\nvr4sW7aM559/Hl9fXz744AMABg8ezGuvvcatt95KZGQklZWV/P3vf++QOABEa0OTnEUIId0lFqXz\n+8lP4NNPJaHPxlNoPM62+7dxdY+h2qIiFy/C/v3g6K/JmzbB2LEQFwdHjpBxPJPJ700mQgzh3NO7\nuPdeeOut1p/GXVhGlyiu19y/hXl7q59UqkevdDlSmnvPIYcpNB4nzCeMYRHDYMsWLcnHx0P//o5/\n4VGjtBLOsWNw4ABJMUn4evhyTu4G/7OqR6+4jEr0Spdz6BCcPQv+w7WSSWpCKjqhg1XmaZbS0myv\nCXuldDq49lrt/sqVeOo9mRSrzaniOXAtBw7AuXOOf1lFaY1K9EqXY+k5BzuzPm/RuE5vHtIZPkbV\n6RXXUYle6XK0ZCopCV0HQEp8ijYEZ+NGbQjk1Kl2Pc/Jqiru3rePhXv2cMLepaKmT9du09Ohupop\ncVMAqI7KsIpNUZxLJXqly1m/Hgg5Qok8Q5hPGIPCB8GaNdrQx+RkaHRFYmM1JhMvHj/OoNxc3jt7\nlmXnzzMkL483T59u/eRkVBQMHw4VFZCZyaioUfgYfLjIfvAtUIlecQmV6JUu5cwZrUbv1V9bxGxi\n7ERt+Jx1fb4Vt+3dyxOHD1NuMjE/PJy5YWGUGo3cd+AA9+7f33oQVuUbD70HyTHJAOjiN7B9u/bl\nQlGcSSV6pUux9JjDR2t3LCdD7a3Prykq4rOCAvz1er4fPpzPhw7ly6FD+fiqq/DR6Xj37FmyLl1q\nOYhGdfqJsdoyyRFjMzCZICur7e9LUa6ESvRKl5JpXo24OuJyj55jx+DwYQgOhtGjmz3WJCW/NE9h\n+9vYWNJCQwFtrPItERH8qndvAH5z6FDLJZwJE7QpFnbsgIsXL3/YxGoxqQXDFWdTiV7pUjZuBHwL\nuMA+fAw+jIwaCeu0k7JMnqydjG3G++fOsa2sjBgvLx63mgTL4le9exPu4cGGkhK+vnix+SC8vbVz\nAQAZGSTHJKMTOs7rt4BHuRaj4jJqKUFF6cQqK2HbNhBxWpc5OSYZT72nNgIGYMqUZo+tMBr5f4cP\nA/B8QgK+Nj4QAg0GnoqLA+C3hw9TZ563xKaUFO02PZ0ArwBGRo7ESB3E5JCTo026prRPZ1lKcOrU\nqfTs2ZPg4GBGjhzp0g8RuxK9EGKGEGKfEOKAEOIJG/uvF0JsF0JsFULkCiEm2HusojjK5s3awJqw\nkVZlG7jco28h0f/j1ClO1dQwyt+f2yIimm13f69e9PH2Zl9FBe+dPdt8MJZEb35tSyxBwzIpK4Pd\nu+17T0pTnWUpwb/+9a+cOnWK4uJiXn/9dW6//XbOueiKuVYTvRBCB7wKpAFDgIVCiEGNmq2WUl4t\npRwJ3Au81YZjFcUhLCURXbx2InZi7EQ4cUKrzwcGwogRNo+TUvKOOWk/HR+ProWrZj11OpbGxwPw\nz9Onmw8mKQk8PWH7digqqk/03gO12LKz2/LOlOa481KCw4YNw8PDo/5xXV1dg+mSncmeHn0icFBK\neUxKWQssA+ZaN5BSVlg99AdM9h6rKI6ycSPgUc5Fzy3ohI5xMeMu9+YnTWq2Pr+trIy9FRWEe3hw\nnfkEbEtu6tGDQL2eLWVl7GtujU8fH61Ob554x5Loi/w2gq6u89fphXDcTwezXkrw7bff5p133qnf\nl5+fT+/evVmxYgUlJSUNErPFnDlzCAkJITQ0tMlta6WeOXPm4OPjQ3JyMqmpqYwZM8bh788e9iT6\naMD6Y+ikeVsDQoh5Qoi9wHLgnrYcqyhXSkpzoo/OxUgdIyJHEOAVcLk+byml2PCB+ev0LT174qFr\n/U/CW6/nxh49APjw/PnmG1pKRenpRPpH0i+0HzWUQ8T2zp/oOwlXLyW4fPlyysrK+O6777jWMg+S\nCzhsKUEp5X+B/wohJgLPAW1+V0uWLKm/n5KSQkoLf5yKYu34cW0iM++0TKqAib3tq88bpeRjc7K+\nvYXafGO3R0Tw7tmzfHTuHM/Ex9ue0zwlBZ59tkGdPr8wH0PfDPZnjqawEOz4AuGeOsn0xe6wlKBe\nryctLY2//OUv9OvXj9mzZ7f7udLT00m3dF7awJ5EfwqItXocY95mk5QyUwjRRwgR2tZjrRO9orSF\npYfsf1UWVcCE2Alw6hTk52srSY0cafO4tUVFnKmpoZ+PD4ltWHFqSnAwvTw9OVxVRXZJCeNsTauQ\nnKzV6bduheJixseM571t7xE8dCMXMh8jO1tbn1zpONZLCVqW6WvPUoIZGRk2P8wnTZrU4ipT1urq\n6hqcH2iPxh3gpUuX2nWcPaWbPKCfECJOCOEJ3AI0+L4ihOhrdX8U4CmlLLTnWEVxhI0bAWGiLFg7\ny9mgPj9xIhhs92ksZZvbIyLatNKQXggW9uzZ4Dma8PWFxESt95uZybje4wCo7ql9KqkTsh3PVUsJ\n7t+/n++//56qqirq6ur44IMPyMjIYEoLI786UquJXkppBB4CVgG7gWVSyr1CiPuFEPeZm90ohNgl\nhNgCvAL8pKVjO+B9KN1cdjYQtp8qiokOiKZ3UO9W6/MVRiOfX7gAwG3mpN0WlmGY/ykooLa5MfVW\n4+kH9xhMoFcgpboTEHBK1ekdwF2XEpRSsmTJEiIiIujZsyevvPIK//nPfxjRzMivjqaWElQ6vaoq\nbfRk7dB3YO693DT4Jj5d8CkMGqQtGbhx4+UrVa18dv48C/bsISkggOwWpkZojpSSoXl57Kmo4Pvh\nw+unTGhg9WptMZLERMjJYfr70/nh8A/wn08JOHETRUUtXqzrUmopQfehlhJUur3Nm6G2FkKGaV3k\ncTHjoKBAS/I+PtoSfzasKCwEYL55BE1bCSGYFx4OwLfNTYmQlKStPLVlC1RUaLEBAYM3UloKe/a0\n66UVpU1Uolc6vZwc853eVoneMkWk5cKlRkxS8r050dszdr45lmO/Mz9XEwEB2oVadXWQm1tfp/fs\no8Wam9vul1YUu6lEr3R6ubmA1yWKPfbgqfdkVNSoy9NYTpxo85gdZWWcrakh2tOToX5+7X7t5MBA\ngg0GDlZWcqiy0nYjSwyZmfVz0xf7bgZ99eUPKUXpQCrRK51ebi4Qk4NEMipqFF4Gr8uJfsIEm8dY\neuDXhYVd0ck4g07HtSEhAPXfEJqwSvTB3sEM7jEYIzUQtVX16BWnUIle6dQKCuDIEfBI0EohydHJ\n2jSWmzdrl9ePG2fzOEuin+GAK5bqyzfN1ektHzZZWWA01tfpRexGdu2C5mZRUBRHUYle6dQsPWL/\nq8z1+d7jIC9POzs7fLjN9WGLa2vJunQJPXCNuTd+JSwfFmuKi6myNf9wr16QkAClpbBzZ32iDxqy\nEaNRO0+rKB1JJXqlU8vNBYSJilCt2D0uZlyr9fkfi4sxAuODgghq5kKqtojy8mKEvz+VJhPrm1tm\n0Kp8YzkhWxupTsgqzqESvdKp5eYCYfupFlYXSrVWnzeXWK5ktE1jrY6+sST6DRsYFD6IYO9gyvUn\nIfCkSvRKh1OJXum0pDQn+t5WZRvr1bdt9Oilg4ZVNmZ3nT4jAx2CpOgk7XFMtkr0TqaWElSUTuTw\nYSgsBJ9+WtkmOTpZW7rp0iWIjQXzYt7WdpeXc6qmhkhPT67293dYLOMCAwnS69lfWcnxqqqmDa66\nCkJCtInWjh+vT/QeCdkcPQotzXasNNRZlhJ86qmnGD58OB4eHjzzzDMOe972UIle6bQsPWGPBG12\nsKSYpFbr82uKiwGYFhzs0DlODDodk4ODAVhrfo0GdLrLvXqr8fS+/bUPKdWrt19nWUqwf//+vPTS\nS1c0LbGjqESvdFo5OYBnGaW+uzDoDNqFUhu0hcGbq8+nm5NwqgNG2zSWakn0RUW2G1hi2rCBxOhE\nAMqDNoOuViX6dnLnpQQXLVpEWloa/g785theDlt4RFGcLTcX6LUJiYnhESPw9fBtMdGbpGSdJdGb\nk7IjpVr16KWUTb8xjB+v3WZlEeYbRv/Q/hwsPAgRO8nJsT0fj7sS7Vj8ojmygxcYsl5K8NChQ6Sl\npdGnTx9AW0owISGBd955h9TUVJvHz5kzh8zMzPqJxaxvJ06c2Cnq+irRK51Sba15/PkYq/r86dNw\n9Kg2v8zQoU2O2VFWRmFdHbFeXiR4ezs8puH+/oQaDByvruZIVRV9fHwaNhg7VpsXf+dOKCkhKSZJ\nS/TROeTmjkJKpyyh2q1YlhLcvXt3g6UEG68w1dpSgp2dSvRKp7RzJ1RXg9+gbMox1+ctE7wnJ9uc\n+9dSO09xcH3eQicEU4KD+fLCBdYWFzdN9JaZNHNzISeHpOgkPtjxAd79syne9CD5+dC/v8PD6hAd\n3Qt3FHdYStAdqBq90inl5QFIjFHmE7HRSZfLNpYSSSPpHVi2sbC7Tp+VVX9CVh+nfSvR3pPiSNZL\nCVq0ZylB65E+1j+zZs1ydMgdQiV6pVPKywOCTlBlOEuIdwj9w/pfHj9vI9EbrevzHXAi1qJxnb4J\nqzr98IjheOm9KPfeD95FKtF3AFctJQjaGrFVVVWYTCZqa2uprq7G1NxKZB1MJXqlU8rLA6K1nnBi\ndCK6qmqtaC+EzdWktpWVccloJMHbm7gOqM9bDPbzI9zDg9M1NeTbmrbYkug3bsQTPaN7mVe2is5V\nib4d3HUpQYDFixfj6+vLsmXLeP755/H19eWDDz7okNdqjarRK51ORYV2XZS4JhsJWglk06bLE5kF\nBjY5xlJK6ciyDWh1+pTgYD4rKGBtcTH9fX0bNujVC+LjtZPGu3eTFJ1E1oksiMlhS24adXXNrmOu\n2HD48OFW24SHhzc5obp06dI2PUd7vPvuu7z77rsd8txtZVePXggxQwixTwhxQAjxhI39twohtpt/\nMoUQw632HTVv3yqEUKOFlSu2dSsYjeAzQOvRJ0UntVi2gcv1+ZQOTvTQsHxjkyXGDRvq6/Q+/bOp\nrFRLCyqGgAxoAAAgAElEQVQdo9VEL4TQAa8CacAQYKEQYlCjZoeByVLKq4HngDes9pmAFCnlSCll\nomPCVrqzvDxAV0t16GZAK920NH6+zmQiwzyrZEf36K1fI725Or3VCVnLVAjGyBxAqvKN0iHs6dEn\nAgellMeklLXAMmCudQMpZbaU0jI/azYQbbVb2Pk6imKXvDwgYgdGUUW/0H6E+YS22KPfXl5OidFI\nX29vYjqwPm8xyNeXHh4enG2tTp+VRWxQLBF+EdQYCiE0XyV6pUPYk4CjgRNWj0/SMJE39lPgO6vH\nEvhBCJEnhFjc9hAVpSHL0oFgrs8fPAgXL0JkpLbARyPrzSWUyU7ozYN2cm+yecETm/PTDx0K/v5w\n+DDi3Ln68g0xOSrRKx3Coad9hBCpwN2A9YxSE6SUZ4QQPdAS/l4pZaat45csWVJ/PyUlhZROclGG\n4jxFRZCfD7obczDRaPz8uHE2Ly21JNvJNlab6ihTgoP5/MIF1hcXc29UVMOdBoM2Mmj16vryzVf7\nv4KYHHasup2qKnDCFw+lE0pPTye9HdNP2JPoTwGxVo9jzNsaMJ+AfQOYIaWsv1pESnnGfFsghPgS\nrRTUaqJXFFs2bdJuPftkU4W5R//Ga9rGZua3cXaP3vq11rV0Qnb1atiwgaSHtItuvPtlU/UtbN8O\nSUnOilTpTBp3gK1HD7XEntJNHtBPCBEnhPAEbgEazOIjhIgFPgcWSSkPWW33FUL4m+/7AdOBXXZF\npig25OUB3kVU+R3AS+/F8Ijhl+vzNhL9nvJyCuvqiPb07JD5bZoz1M+PYIOBY9XVHLM1P73VTJZj\neo1BIKgJ2Q6GKrcp38TFxSGEUD9u8BMXF3dF/5at9uillEYhxEPAKrQPhrellHuFEPdru+UbwJNA\nKPAPIYQAas0jbCKAL4UQ0vxaH0opV11RxEq3pl0opY3SHRU1Cs9LZbB3r1brGNV0BkhL2WZKB81v\n0xy9EEwMCuKbixfJKC4mLjKyYYPkZG2O+i1bCDR5MLjHYHYX7IbIreTljXNanC05evSoq0NQHMSu\nGr2U8ntgYKNtr1vdXww0OdEqpTwCjLjCGBWlXl4e0M/qRKylNz92LHh6NmnvirKNxWRzol9/6RK3\nN070gYEwbJhWp8nLIzkmWUv0MTluk+iVrkMNe1Q6jTNntJX4DPH2TWQmpXTJiVgLy4fLejsunLKM\npxe9s9m3D0pKnBGh0l2oRK90GpYZK0WMVrpJiklqsT6fX1nJmZoawj08GNR4KgInGOXvj59Ox/7K\nSs7V1DRtYH3hVIxlDdkcpITNm50YqNLlqUSvdBp5eUDoIWo9LtLTrydxPlGXF1u10aO37s07sz5v\n4aHTMd78TSLDVq/eKtEPCbsKPw8/anyPgt85tzkhq3QNKtErnYb1jJXJMcmIbdugqgoGDYKwsCbt\nXVmft7CUjNbZunAqLk6b5KywEP3BfMZGj9W2qwunFAdTiV7pFKQ0J/oYq4nMWlloxDKG3RX1eYsW\n6/RC2KzTE60SveJYKtErncLhw1BY2OhEbAv1+WNVVRyrriZIr2e4v78zQ20gMSAALyHYWV5OYW1t\n0wZW4+ktiV4Xl82xY1BQ4MRAlS5NJXqlU8jLAwxVGHtsQyAY22tMizNWWnrQk4KD0btwxW1vvZ6k\nwEAk1M+g2YCNpQVFdB4Io+rVKw6jEr3SKeTlAZFbkbpahvQcQuDpi3D2rFabHzCgSXtXDqtsbEpL\n5ZsRI8DXFw4cIKpST2xQLEZDKfTYqxK94jAq0SudQl4e0HsjAMnRyZCRoe2YONHmRGaW+vwUF56I\ntag/IWsr0Xt4XJ7YxmohEmKyVaJXHEYlesXtGY3acrDEaPX5cb3HQaZ5XryJE5u0P1NdzcHKSvx0\nOka5sD5vMS4oCIMQbC0r41JdXdMGkyZptxkZjIsxXxVrTvS21i1RlLZSiV5xe3v3Qnk56OO0RJ8c\nY9WjtyRJK5ayzYSgIAw61/8X99PrGRMQgAnIslWnt3xYZWbW9+j1cdmcPw8nTjRtriht5fq/AkVp\nRV4eEHAKo/8JAr0CGSTDYP9+8PGBkSObtHenso3FlJbKN8nJoNfDli2M9O+Pp94TY+ge8LqkyjeK\nQ6hEr7g9bfz85WGVug3mYZXJyS1OZOZWid4yP72tHn1AgHZS1mjEa/M2RkaOBCEhOlclesUhVKJX\n3J62dKCNso2N+vyFmhp2V1TgrdMxNiDAiVG2bEJQEDpgU2kp5UZj0waWElRmZoM6vWWGB0W5EirR\nK26tqkqbybf+RGyM1YlYG/V5y1j1cYGBeLpBfd4i0GBgpL8/dVKy0Vav3uqErPXIm02btJPRinIl\n3OcvQVFs2LYN6ky1iBhtDcHEoMHaEBy9XivdNJLuhmUbC0tM6S1NcJadTXLEaAB0sdmUlkr273dW\nhEpXpRK94tZyc4HI7Uh9FQPCBhC2M1/r4o4YodW2G1lrTqKpbpjoLTGttZXoIyK0C7/Ky4k9Ukik\nfyQm70IIzVflG+WKqUSvuLVm6/M2yjYFNTXsLC/HW6cjKTDQiVHaZ1JwMDogt7SUMlvj6c3nHMSG\nDapOrziUSvSKW2uQ6KOTW7xQylISmRgUhJcb1ectggwGRgcEUCclmfbW6XtvVIleuWJ2/TUIIWYI\nIfYJIQ4IIZ6wsf9WIcR280+mEGK4vccqSnMKC+HgQaC3NpxyXOQY2KhNg2Ar0a9x47KNxdSWyjfW\nV8j2Mk+L0DuL7du1k9KK0l6tJnohhA54FUgDhgALhRCDGjU7DEyWUl4NPAe80YZjFcWmTZsA/zMQ\ncgR/T3+GHa+GigptoZGIiCbt1xYVAZeTqTtKDQkBLn8oNdCnD0RHw4ULjC32w0PnARE7qdOXsG2b\nkwNVuhR7evSJwEEp5TEpZS2wDJhr3UBKmS2ltHwXzQai7T1WUZqTm0t9bz45Jhn9enN9PiWlSdvT\n1dXsr6zEX69ntBuNn29sonnemy2lpU3nvRGi/r15Z+UwKmoUCBNE56jyjXJF7En00YD1jBsnuZzI\nbfkp8F07j1WUejk51Cf68THjIT1d2zFlSpO2a61Wk/Jww/q8hZ9eT5J53hub0xZb3lt6OuN7m1ef\n6p2lEr1yRQyOfDIhRCpwN9C0gGqHJUuW1N9PSUkhxUbPTekepDT36K/XFheZGJUEG/6o7bSR6NeY\nyzbuXJ+3SA0JYUNJCWuKi5kTHt5wp+X//Lp1TFjyD/7Mn7VEn+P0MBU3lJ6eTrqlw9MG9iT6U0Cs\n1eMY87YGzCdg3wBmSCmL2nKshXWiV7q348fhfGElRG1BIBh3zlObwnLgQIiKatLe0qOfaq6Bu7Op\nwcE8d+xY/TmFBvr1097fmTNMqjB/CMRkc/BDI4WFekJDnRur4l4ad4CXLl1q13H2fMfNA/oJIeKE\nEJ7ALcDX1g2EELHA58AiKeWhthyrKLbk5gK9NoO+lqE9h+K/Ubsy1lZv/khlJUeqqgg2GLjaDeaf\nb824wEC8hGB7eTkXamoa7rSq0/fM20N8cDx4l0DP3WqCM6XdWk30Ukoj8BCwCtgNLJNS7hVC3C+E\nuM/c7EkgFPiHEGKrECK3pWM74H0oXYxWn9fKNhN6T4B167QdNsp5lhEsU4KCXLo+rL289XrGm6ct\ntjnM0vJhtm6dqtMrDmHXWSsp5fdSyoFSyv5Syt+bt70upXzDfH+xlDJMSjlKSjlSSpnY0rGK0hrr\nE7ETopIuXyhlo0e/qrAQgOmdqK4x3VxiWmWrfGP5MEtPZ0LM5USfne2c2JSux32HJyjdVm0t5G2S\n9Yk+tSgIysqgf3/o1atBW6OUrDYny+mdoD5vYflQWlVYiGy8XuCAAdp1AufPk1ptfr+9N5CdrZYW\nVNpHJXrF7WzfDtV+B8HvAhF+EfTafEDbYaNss6W0lMK6OhK8venr4+PcQK/ACH9/wj08OF5dzYHK\nyoY7rer0A3aext/TH0IPU1hzlvx858eqdH4q0StuJzuby+Pne49HtDB+fqWlbBMSgugE9XkLnRBc\naynfmN9DA+ZEr09fZzXvTVb9DBCK0hYq0StuJzsbiNVq8pMik2D9em3H1KlN2lpq3J2pPm/RYp1+\n2jTtds0aJkab6/SxmapOr7SLSvSK28nOBuK05D7jQrA2v82QIU3Gz5fU1bGxpAQ97j2/TXMsH05r\ni4qoMZka7uzXD2Jj4eJFZpSb33fcepXolXZRiV5xKwUFcOjcWQg7iJ+HHwO2Htd2XHNNk7bpxcXU\nSUlSYCDBHh5OjvTK9fLyYqifH+UmE1mNpy0Wor5XP2J3IQadASK3sn1fKeXlLghW6dRUolfcSk4O\nEKtNXjau9zj0a9ZqO2wk+pWdcFhlYy2Wb8zv2St9PaOjRoPOhKlXFps3OzNCpStQiV5xK1rZRkv0\n14QlapfI6vW2x893wmGVjVkPs2zCUqdfv56USPOKU3EZqnyjtJlK9Ipbsa7Pzzztp60Pm5zcZH3Y\nw5WV5FdWEqTXM9aNpyVuzaSgILyEYEtZGecaT4cQEQHDhkFlJdcX9tC2qTq90g4q0Stuw2iEnO3F\nELEDD50HV203z39no2zzzcWLAKSFhmJw42mJW+Or1zM1JAQJfGt+Tw2Y3/vI3eZ90blk5VSrC6eU\nNum8fyFKl7N3L5SFbAAhGRs9FsOadG2HjUS/3JwUZ4eFOTHCjmF5D9/YSvTm8o3Pug0M6zkMDNWc\nM+Rx/LgzI1Q6O5XoFbeRlUV9fX6W70jYswf8/SEpqUG7kro61hUXowNmdqFEv6qoiOrGwywnTwaD\nAfLymB5qnkIqbr32u1IUO6lEr7iNDRuoH3Ez84S3tnHKFGg0dHJVYSG1UjI+KIiwTjissrFYb2+G\n+/lRZjSyrvFslgEB2jkKk4m5Z4PMB2RovytFsZNK9IrbyMiuhOg8BILBm5sfP28p28zpAr15C8t7\nWW6rfHPttQCM2n5eexy7gcwNRmeFpnQBKtErbuHsWThSkw36Wkb0GI7n6jXajpkzG7QzSsm35qGI\nXaE+b2Fdp28ym6X5d+D343oSAhPAq5QdBVspLXV2lEpnpRK94hY2bAAStIujFlUPhIsXoU8fbWpi\nKzklJVyoraWPtzdX+fq6INKOkRgYSA8PD45WVbG78aWvo0ZBz55w/Di3e4wEQMatVcMsFbupRK+4\nBS3Ra734mfnmWSivu06bCsCKddmmM81W2RqdEMxqbvSNTgczZgAw74iXti1hrarTK3ZTiV5xC+uz\nyyE6Bx06+mab559vVLYBWH7hAtC1yjYWlvf0ta06/XXXATDEcu4ibj0ZWbXOCk3p5FSiV1yuogK2\nXswEfR1TfYZj2LIVvLyaLDSyv6KC3RUVBOn1TO6Es1W2Ji0kBG+djo0lJZyqrm64c/p00Onwysrl\nau/+4FnOxmN51NW5Jlalc7Er0QshZggh9gkhDgghnrCxf6AQIksIUSWE+EWjfUeFENutFw1XFGu5\nuWCK1co2iy9GaxtTU6FRDf6zggIA5oaH49mJr4Ztjr/BwHXmuW8+N7/XeqGh2jDL2loeLO0LQGXE\nWnbudHaUSmfU6l+LEEIHvAqkAUOAhUKIQY2aXQQeBl6y8RQmIKXxouGKYmFdn5+0u0zbaC5VWPv0\nvDa8cEGPHs4KzeluMr+3zxoneqj/nVx7wNyNT1ij6vSKXezpFiUCB6WUx6SUtcAyYK51AynlBSnl\nZsDWF0lh5+so3VR6djFEbcHTqCcya4e2sVGiP1hRwfbycgL1eq7txNMSt2Z2WBheQpB56RJnGpdv\nzOcs4jbuBQnEbmB9VpXzg1Q6HXsScDRwwurxSfM2e0ngByFEnhBicVuCU7o+kwk2nl4POhOLKocg\nioq01ZUaDau09HCvDw/HqwuWbSwCDQbSQkORwBfmE8/1RoyAiAj0J08xu6I/GKpJz1djLJXWGZzw\nGhOklGeEED3QEv5eKWWmrYZLliypv5+SkkJKo5NxStezaxeU99TKNned8NM2zprVpN2n5kR/Uxcu\n21gs6NGDry9e5LOCAn4ebdWn0ulg9mx4+20ePBvON30PUuC/hqNHU4iPd1m4ihOlp6eTnp7e5uPs\nSfSngFirxzHmbXaRUp4x3xYIIb5EKwW1muiV7iE9Ha0+L2FUjnno4A03NGhzqLKSrWVlBOj1pHXi\nRUbsNSc8HE8hWF9czLmaGiI8PS/vnDcP3n6b8ZvPQ18gYQ3p6c9w112uilZxpsYd4KVLl9p1nD3f\ngfOAfkKIOCGEJ3AL8HUL7euvYhFC+Aoh/M33/YDpwC67IlO6hZWZ5yFiJ1cXeOJ79BSEh8OECQ3a\nWMo2c8LC8NbrXRGmUwUZDEwPDcUEfNn4pOw114CfH8F7DhFbLCA6hx/Wl7gkTqXzaDXRSymNwEPA\nKmA3sExKuVcIcb8Q4j4AIUSEEOIE8Djw/4QQx80JPgLIFEJsBbKB5VLKVR31ZpTOxWSC9ae0/w4/\nO2X+0jhnjjYtr5Vl5tE23aFsY2EZWfSR+b3X8/auP1H94Ik40Nex+tBaZ4endDJ2ndWSUn4vpRwo\npewvpfy9edvrUso3zPfPSSl7SymDpZShUspYKWWZlPKIlHKEeWjlMMuxigLadPNlESsBuH6fefRI\no7LNjrIytpWVEWwwdIm55+11Q3g4vjodGZcucbiysuHOefMA+Mkh7eH5gJVqIRKlRV13+ILi9tam\nm6DvKmIuQeT+k9oFUo2mJf732bMA3NKzZ5cebdNYgMHADeHhAHxw7lzDnTNngsFAws4ThFYAfVey\nbp3zY1Q6j+7zl6O4na9zt4H/eRYeME9nMGMG+PjU768zmfjQXLq4MyLCFSG61B2RkYD2Yddg6uKQ\nEEhJQRiNzD/gB6GHWb4h30VRKp2BSvSKS0gJGwu0ss2iI+bk3qhss7qoiLM1NfT38SEpMNDZIbrc\ntJAQojw9OVRVxcaSRidczeWbO49qH5Jrj690dnhKJ6ISveIS+/ZBeeRKQitgyP5z2gnYRuPn/20u\nWdwREdGlpiS2l14Ibjd/k7GUsOrN1S5OT95bgF81XAheySm7Bz0r3Y1K9IpLfL+2BHpv4KY9Ap3R\nBNOmaSUJs0t1dXxpvjJ0kbmE0R3dYU70nxQUUGW0Wj4wJgbGj8dQXcOcA0D8Wn5Mr3FNkIrbU4le\ncYkvtqwFfR137Q7QNixc2GD/p+fPU2UykRIcTJy3twsidA9D/f0Z6e9PcV0dXzWep/6WWwC4c1cA\neJXxaXaWCyJUOgOV6BWnkxI2l6ykVwkkHS3V5p4315wtXj9zBoC7unFv3uIe8+/gtdOnG+5YsAB0\nOq7JLye4EjLPqjq9YptK9IrT7dwpqez1HQt2g05KbbhgUFD9/rySEjaVlhJiMPCTbnSRVHMWRUbi\nq9ORXlzMXuv1ZCMjITUVg9HEDXuhOPxbjh51WZiKG1OJXnG691fuhpCj3LbTfAWsuQRh8U9zz/We\nyEh8usGUB60JMhi4zVyrb9KrN//ubt2ph8gdfPL9MWeHp3QCKtErTvfVvq/pUwhjT9eBn582I6NZ\nYW0tH5vHzj/Qq5erQnQ7D5p/F/86e5Zy65Oy8+eDhwepR430LINPti13UYSKO1OJXnGq6mrIN3zF\nzZap7ebObbBk4L/OnqXKZGJ6SAj9Gi0l2J2NDAggKSCAS0YjH1tfKRsaCmlp6CXctAd21X6FyeS6\nOBX3pBK94lTL155BRuVy607zuHir0TYmKevLNj+LbsvaNt2D5Xfyz9OnG14pay7f3LYDaqPTyci7\n5IrwFDemEr3iVO9s+IYxp2FogdSmJJ4+vX7fD0VFHKysJMbLi1ldeLnA9vpJjx6EGgxsKStreKXs\nvHng78/4kzCwsI7Xf/zedUEqbkklesWpsi5+zT1bzQ8WLQKrRTVeNE/B+PNevTB0ownM7OWt19ef\nt3jRerpKP7/6Xv09W2HtqZaWi1C6I/XXpDjNiXPl1AT8wK07zRvuuad+36aSEtYUFxNglcyUph6O\njsZLCL66eJF91kMtzb/LO7bDBd8VlJTVuihCxR2pRK84zasrVjP/YDVB1UBiIgwdWr/vxRPa+vP3\n9+pFsIeHiyJ0f5FeXtxpvoDqZfPvDIDkZLjqKiLL4bqTl3j9e5urdSrdlEr0itP8d/9/L5dtrHrz\n+RUVfF5QgIcQPBYT45rgOpFf9e6NAN4/d47T1dXaRiHqf6f3bIWPt37pugAVt6MSveIU1XU11JV8\nztSjUOfl3eAiqZdPnMAELIqIINrLy2Uxdhb9fX2ZHx5OjZT87eTJyzsWLcKk0zP7AJy/9AkmqcZZ\nKhqV6BWneOPHH7hrTykA+gUL6qc8OF1dzXvmKXh/1bu3y+LrbH4dq62x+4/Tp7lYa67HR0TArNkY\nJNy6/zz/yVblG0VjV6IXQswQQuwTQhwQQjxhY/9AIUSWEKJKCPGLthyrdA/vb/iIxZu1+2LxT+u3\nP3fsGNVScmN4OFf5+bkous4nKTCQa0NCKDUaeclqBI7u/sUAPLgJXlu7zFXhKW6m1UQvhNABrwJp\nwBBgoRBiUKNmF4GHgZfacazSxVXVVTFo+xdElsOF2IEwaRIARyorefPMGQTwTEKCa4PshJ4z/87+\nduoUZy21+uuuo7BHDAnFEL71I4wmYwvPoHQX9vToE4GDUspjUspaYBkw17qBlPKClHIzUNfWY5Wu\n75NN3/PwpioA/H/za+3EIbD06FHqpOT2iAgGq958myUGBjI3LIxKk4nnLb16nQ6Ph7Uv1Q9svsTK\nfetdGKHiLuxJ9NGA1TguTpq32eNKjlW6iIwP/8HY01Dk5Yv3PbcCsLe8nPfPncMgBEvi410bYCf2\nbEICAm1Wy2NV2odpwCP3UG7w4Joj8NWHr7k2QMUtGFwdgLUlS5bU309JSSElJcVlsSiOUVlbybS1\nawDYPXERE320hcCfPHIEE3BfVBR9zNuUthvm788tPXvy8fnzLDl6lHcHDYKgIDaPvoHJOf9h9Mqv\nqXu2DoPOrf7UlXZKT08nPT29zceJBpMj2WogRDKwREo5w/z4t4CUUv7BRtungVIp5Z/acaxsLRal\n8/lq9ZvMTLsPnYRT648ROzGWdcXFpGzbho9Ox8GkJDWk8godrKhgcF4eRinJHTWKMYGB7PtyD4Pm\nD6HcAzZmfcY1Y250dZhKBxBCIKUUrbWzp3STB/QTQsQJITyBW4CWJtOwftG2Hqt0McUvvoiHCb6L\nGk7sxFjqTCYeOXgQgP+JjVVJ3gH6+/ryWEwMEngkPx8pJQPnDWZNeB/8auHE88+6OkTFxVpN9FJK\nI/AQsArYDSyTUu4VQtwvhLgPQAgRIYQ4ATwO/D8hxHEhhH9zx3bUm1Hcy7mT+5m3Ph+AfVNfAOCN\nM2fYUV5OvLe3GjfvQE/GxRHh4cHGkhI+PHcOISB3/NMAzFq5neLC0608g9KVtVq6cRZVuul6MhbP\nYNJbK/mxZziB3xSQcHUNA3JzKaqr44shQ7hBrQfrUO+dOcPd+/cT5enJ/sRE8tbp8b85iMSLpWz4\nxU+Y8MdPXB2i4mCOLN0oSpvJ0lKGfrQagL9HPMyYMfC/R45QVFfHNSEhzAsPd3GEXc8dkZEkBgRw\npqaGpUePMnmK4E89tPlv+r39X215L6VbUole6RAnXvodIRVGsnoZiLv2t6wpLuLNM2fwEIK/9uuH\nEK12QpQ20gnB3/v3Rwf8+eRJNleU4D/5OXaF64i4VMOZvzcZA6F0EyrRK45XVUXg398C4P8iZ3H9\nAh0/3b8fgKfi4tTFUR1oTGAgv+rdGxNw9759zF/gy/O9JwPg+fKfoa7xNY1Kd6ASveJwNa/+jeDC\nCrZGwpa6Z/g89AhHq6oY6e/PE+bJuJSOsyQ+ngE+PuytqCAz4Sjf1yzhYCiEnSnG+M7brg5PcQF1\nMlZxrOJiquNj8LpUzswZ/fCflMen47dhEIJNo0dztb+/qyPsFjZcusSkrVvRAbO+G4nPtqtYtuoM\nlT1D8TlyAnx9XR2i4gDqZKziEvIPf8DrUjnpcfBd6f+ydvweAP43NlYleSeaEBTEozExGIG8GXv5\npOq3bI4Cn/OF8Ne/ujo8xclUj15xnFOnMPbri76qmsTbg9gxOp3qEcVMCAwkfcQIteC3k1UZjYzb\nupVtZWV4ZIUwaWcyPy6rwhjgj/7IUQgLc3WIyhVSPXrF+ZYsQV9VzaeDIS/291SPKCbEYOCjwYNV\nkncBb72eTwYPxl+vp3Z8EWvin2NVH9CXlsH//Z+rw1OcSP31KY6xdSvynXeo08EvbhoOqdqyA+8N\nGkSst7eLg+u+Bvj68tqAAdqD1NE8skC7L199Ffbtc2FkijOpRK9cOZMJHnwQYTKxdHoEJ5NeAAM8\nHhPD9erCKJe7LSKCn0ZFgSfsn/QSL08ORdTWwkMPgSqXdguqRq9cuTffhPvuI7+nLwPf+humgL5c\nVRbCjpnDVMnGTdSYTAz8ZjtHAy9hKNrD8bt/QdSlavj44wYLtSudi701epXolStTUAADB1J36RJx\nbz3D6YQJcMKHwzNGkdDDw9XRKVZ2HKvh6o2bIbKavntXc+Dnz6OLjNRKOIGBrg5PaQd1MlZxjt/8\nBllUxNylv9GSfLmJGeuHqSTvhobHeTJ+xTCoMnHoqmtY9NtHkGfOwO9+5+rQlA6mEr3SfsuXI997\nj1/+/Gd8OzEN6mrgtyN5+AZ1MY67emimPzw5DIy1fDR9Hs/ecQe88gqsXevq0JQOpEo3SvsUFMDQ\noTw3fTpP3nsvmGrhtVKiMudz/DgY1Mp1bqmyEqKjoWjuR3BHBAg9f33lFR7ZtAl27lQlnE5GlW6U\njiMl8oEHWHLddVqSlya8dr4Ln83lwQdVkndnPj7w058C79+I5753AXj04Yf5Y1ISPPaYa4NTOoxK\n9Eqbyfff57fh4Sy96y6QJtj3AtXvTcfTQ89997k6OqU1P/sZ6KQXde8nwoE/AfCrn/2M52pr4Ysv\nXDYOwaUAAByySURBVByd0hFUolfapG7HDh7ctYsXFy5EbzLBnmcIOXMcdtzGLbdARISrI1RaEx8P\nc+eCKe8+gs5sh32/R0jJk/feyy/Xr8d04ICrQ1QczK5EL4SYIYTYJ4Q4IIR4opk2fxNCHBRCbBNC\njLTaflQIsV0IsVUIkeuowBXnKy0s5Pp163h95ky86urwOfgHuLCO6m9eApMHjzzi6ggVez38MFDn\nDT8+D+dWEnTkVQxGI3+aN4+ffPstlWVlrg5RcaBWE70QQge8CqQBQ4CFQohBjdpcB/SVUvYH7gf+\nabXbBKRIKUdKKRMdFrniVCcqK5m8ejXfDRtGeFkZP6nOouzsKvp7TKFi6xwmTIDRo10dpWKvlBQY\nOhQuZS4kwWs0xSe+4K7abQRVVPD5iBFM/fZbzqqlB7sMe3r0icBBKeUxKWUtsAyY26jNXODfAFLK\nHCBICGH5Ei/sfB3FTa0uLGRURgbbevak/6lTfBHpzydbtUmxqr/+IyBUb76TEQLt30zq0P34MgAf\nb3mab0J9iT1/nuyePRm1bh0bLl1ybaCKQ9iTgKOBE1aPT5q3tdTmlFUbCfwghMgTQixub6CK85mk\n5IVjx0jbvp0Lnp6k5eaSFRzMnw79lRpjDRMDFnE8ZzR9+sANN7g6WqWtbr8doqLg0OoUEgOvp7y2\nnNfPvE2OTsfk7ds54+lJypYt/PXkSdTQ587NGT3tCVLKUcBM4OdCiIlOeE3lCp2oquKa7dv53yNH\nMAnBk//+NysCA1mXUMV/9/2XAM8ATv37eUC7sNJDXQjb6fj4wG9/q90v++KPeBu8+WDHB2wfEcDq\n2lp++ckn1AnBY/n5zN65k3M1Na4NWGk3e0Y8nwKsF/qMMW9r3Ka3rTZSyjPm2wIhxJdopaBMWy+0\nZMmS+vspKSmkpKTYEZ7iSFJKPjl/ngcPHqS4ro4excW8+4c/MCs1laI7b+ahfwwG4MagP/De9hj6\n9NF6hkrntHgxvPAC7Mnsx513LOFfp3/L/d/cz66Hd/LyL/+HcU8/zeJf/YpvgWF5ebw5cCBz1Yyk\nLpOenk56enrbD5RStvgD6IF8IA7wBLYBVzVqMxNYYb6fDGSb7/sC/ub7fsAGYHozryMV1zpeWSln\n79ghWbtWsnbt/2/v3MOjqq4F/tuTZB6ZPElCEgiQRCC8QaA8qrxFEGlQUcEHCNoLRVv9akv12muv\nVr23WhVQe0stYHnYCkU0IiLvhyIgz5DwCCEkkEDer0kmk5lkZt8/9uSBJBBMwkA4v+/bX3Jm9jmz\nzsw566y19tpry8lvvilzgoOlnDFDSpdLPpXwlOQV5B1L75TdujslSLlsmael1mguCxdKCVLePsgh\nByweIHkF+euvfy1ldbWU998vM0ND5biFC2uvi4eSk2V2ZaWnxdaQUrr15tX1eJM6wUQgBUgFXnS/\nNheYU6/PB+4HQiIw0P1ajPvBcARIqtm3kc+4Ll+MxuU4nE757vnz0m/3bsmOHTJwxw754dSp0gVS\nTp8uZVWV3Hxms+QVpP41vXxz6UkJUsbGSulweFp6jeZSUSFlRITSBu/866DUvaqTuld1cs/5PVLa\n7VL+7GfSKYRc9Pjj0uxW9kHffCP/kpUlq5xOT4t/S9NURa/VurnF2VhYyPNpaZyqqABgqsvF+7Nm\nEZmZCdOmwapV5FQWMGDxAHKtufz3nW+wbPZLZGbCsmUwe7aHT0CjRVi0SFVA6N4dfvbeC7yz7y06\nB3bmyNwjtNOZ4cEH4csvOdetG/M+/JCN7v36ms0s6tqVMcHBHpX/VkWrdaNxRb63WBifmMikpCRO\nVVTQzWTiy/x81k6YoJT8o4/CqlU4dYLH1j1GrjWXMdFjqN71ApmZcPvtMHOmp89Co6WYNw/i4uD0\naQg99hpDOg7hfOl5nkx4EqnXw9q1cN99dElNZcPEiawtKyPaaCTJamVsYiL3HjvG0bIyT5+GRiNo\niv4W43BZGfcnJzP08GG2FhcT4OXF27GxJG/Zwr0PPwzV1TB/PqxcCd7e/M83/8P29O2E+Ybxv4M/\n5u23vABV2dbLy8Mno9Fi6PWwcKH6/40/6nlvxCcEGgJJSEng/e/fB4NBKft58xB2O1OnTOHE3r28\nHh2Nn5cXXxUVcfuhQ0w/fpxkbVbtjUdT4jvXo6HF6FsNl8sldxcXy4mJibUDaqZdu+QLZ87IwoIC\nKadMUQFaIdTInJsNpzdI3as6KV4RctOZTfK++1S3xx7z4MlotCrx8eo3fuIJKdceXyt5BenzRx+5\nI32H6uBySfnGG6oTSPnIIzKvuFj+OjVVGnburL2+7ktKkvtLSz15KrcEaDF6DYfLxZq8PBZmZXHI\nbWX56nT8okMHftupE5EpKSr2mpoKQUGwahXcey8AR7KPMOKjEVirrPxh5B8YVvkqkyaB2azc+w4d\nPHlmGq1FWhr06gUOB+zaBQm23/DuvncJMgax96m99Ah1Vz/59FOYNQvKy6FvX1izhszoaN7KzGRJ\ndjaVLhcAPw0I4LmoKB4IDdXWD24FtDVjb2HSbDb+fvEiH+XkkFdVBUCItzdPd+zIsx07EurtDe++\nC7//vbqj+/VT5Wlvuw2ALEsWQ5cM5WLZRR7t+ygLR6yiXz9BTg78+c/w29968uw0Wps//AFee01V\nuTx8xMnsr6eSkJJATFAM+36+j/bm9qrjyZNqSnRKipp99ec/w9NPk1tVxYLMTP6WnU1JdTUAHfR6\nnoqM5KnISLoYjZ47uTaGpuhvMSzV1Xyan8+K3Fx2lpTUvt7XbOa5qCgebd8ek5eXMtl+/nOomXQx\ndy68844y1YF8az5jV4wlOS+ZkV1GsumxzTw23cC6dTBihFpxTovNt20cDhg+HA4fVkb7B3+zMnr5\naA5ePMjgDoPZMmMLQcYg1dliUUVzli9X2xMnwuLF0KULVqeTFTk5LMrKIsVmA1Thq3HBwTwRHs79\nYWGYtYupWWiK/hagwunkq8JC1uTn82VhITa3u2zU6ZgWFsacDh0YHhCAEALsdnjrLXjjDfV/+/Yq\nP9IdqgHIs+YxbsU4kvOS6RHagz1P7uGL1e2YPRv8/eHYMWXlabR9Tp6EgQOhslJFaX56dw7Dlw4n\noySDwR0Gs/nxzQSb6qVUrl2rjIaiIvD1hVdfheeeAx8fpJTsLi3lbxcv8ml+Pg73fW7W6YgPDeXh\nsDAmtmuHUVP614ym6NsoBQ4HXxYWklBYyOaiIircyh1gVGAgMyIieDAsjMCa9fykhC++gN/9TgXX\nAWbMUFZ8WFjtvrnluYxdMZYT+SfoGdqTbTO3UXohkiFDoKwM/vEPeOKJ63iiGh7n/feVsR4SAgcO\ngC74HGOWjyG9JJ1BkYPYMmPLpco+O1sp93//W2336aPCORMmqHKZQFFVFWvy8liZm8t3Fkvtrmad\njntCQpgSEsK9ISEEa8WTmoSm6NsI1S4Xh8rL2VRUxFeFhXxfVkb9b2mIvz/T2rfnwbAwOv8w9rl3\nL7zwAnzzjdqOi4O//hXGjLmk26mCU0z+52TSitPoHdabbTO34VUZzrBhKtLz8MPwySe196rGLYLL\nBZMnw8aN0Ls3fPcdlMpMRi8fzdnis/QK68X6R9YTGxx76Y4bN8Izz0B6utq+6y74058uW7Ag3Wbj\n3/n5rM7L43C9lEwdMDwggHtCQpgQHMzt/v54aRdfg2iK/ibFKSXHysvZVVLCzpISdpSUYHE6a9/X\nC8HooCCmhIYSHxJCVEMDW7t2weuvw9atajs0VI2wzZ2rEqbrsfXsVh5c8yCl9lIGRg5k42MbCfJp\nz/jxsHu3ct93764N4WvcYpSWqnj9yZNwzz2wfj1kW7OYuGoix/OPE+obyufTPueOzndcumNlJXzw\ngQoV1owZTZoEL78Mw4Zd9jkZNhtfFBaSUFDA7tJSquvpgmBvb8YEBTEmKIiRQUH0MZvRaYof0BT9\nTUNJVRUHysrYa7HwXWkpey2WSxQ7QDeTibuCg7mnXTvGBAXh591A0dGqKhVMXbQI9u1Tr/n7K997\n/nwIDLyku0u6WLB3AS9sfQGndHJ/j/tZef9KTN5mnnxSja116ADffw8df7j6gMYtRVoaDB0KhYXK\nUH//fShzWJi2dhpfn/kavZeeBRMWMG/wPDUeVJ/CQmXN/9//gbvMBiNGqHoLU6Y0OLJvqa5mW3Ex\nG4uK2FJcTEZl5SXvB3t789OAAH4aGMjwgAAG+/vj39A9cQugKfobkNLqao6Wl3O4rIzD5eUcsFhq\nsxHqE200MiowkFFBQYwLDr48JFOfjAz46CNYuhQuuKtHBwerWOmzz6r/f0B2WTZPfP4EW85uAeDF\nO17kjXFvIF06/uM/1OFMJhXx0ZYH1AB1LYwbp+yJX/1K2RNOWc3zm55XM2eB+Lh4lsYvJdS3gTLG\n+fmwYIGy8mtKJXTpojLAZs2CqKhGP/uszca24mJ2l5ayq6SEzB8scSiAXr6+DPb3Z6C/PwP9/Ojv\n53dLKH9N0XuQSqeTVJuNZKuV41YrSVYrieXlnGtgDU69EAzw82N4PQul09XyjEtK4LPP4J//hG3b\n1IArQM+eSrnPmNFgrEVKycpjK/nN5t9QUFFAqG8oS+OXEh8XT3W1ut8+/lgp+fXr1Y2toVHDl1/C\n1Kkq/fIXv4C//AV0OlhzfA1z1s+h1F5KpF8kCycu5KFeD11u3YNKx/zHP+C995SrAOogEyao+kpT\npihPtBGklJy32/mutJQ9paXsLysjsbycqgZ0R6zRSH8/P/qYzfQ2m+ljNtPNZELfhiZuaYq+lXFJ\nSZbdzhmbjVSbjdMVFZy22ThptZJeWYmrgX0MQtDHbGbQDyyPJl14eXlK+yYkwKZN6m4DVYNk6lR4\n6ik1yNpI7DI5L5mnNzzNN+fVwOz42PEsv285kf6RWCzq2fDFF+DnBxs2wMiRP/KL0WjTbNoE992n\nQvDTpilH0myGcyXnePyzx/n2vFpTaHzseD6Y9AHdQ7o3fCCXC7ZsUQf4/HPlKgAYjWowYMoUNRIc\nEnJVmewuV62nfMjtLR+3WmvTOOvjBdxmMtHD15fuvr50N5no7utLV5OJSL3+pov9a4q+mbikJM/h\n4LzdzrnKSs5VVpJev9ls2BuRV4e6mGqsiN6+vvT386ObydT0aeBVVSqnbdMm2LwZ9u+vs9yFUEp9\n+nRVwuAKJWLTitL44+4/surYKlzSRZhvGG/f/TYz+s1ACMGJE/DAA2pyY2CgSpgYPvwavyyNW4pt\n25QetlovnVTtki6WHF7Ci1tfpLiyGC/hxewBs3l51Mt0Duzc+AELClRK5r/+VZchBsrSHz5cWfsT\nJqg4YhNz7atcLlIqKjhmtZLsbsfdRlhjWsak0xFtNBJrNBJjMhFtNBJtNNLFYKCz0UiYj0/DXooH\n0RT9FbA6nWTb7eQ4HGQ7HFyw27no/ptVrzVkEdQnQq/nNqORriYTcW4LoYfbOjBcq3tYUaEU+549\nKmtmzx51J9Wg18PYscqcio9XqzpfgaM5R1m0fxErE1filE68dd7MHTSX18a8RrApGCmVB/3ss3Xl\nStatg65dr01sjVuTEyfUpVhTJumvf1UWvhBqdvVL215i2dFluKQLvZeeWf1n8dyw5+gV1uvKB87K\nUl5rQoKahu0uoQBAQIAayB01Sj0ABg1SccZroNLp5LTNRorbA0+pqOCMzcYZm438Gq+iEQxCEGUw\n1LaOBgMdDAY66vVE6PVEGgxE6vX4XseJX5qib4Tnz5xhQVZWk/qGeHvTqd5TvYvRSKzJpJ74RmPD\n2S9NoaJC3SmHD8PBg3DoECQmwg+ybejWDe6+W1kzo0dfMXYJUFFVQcKpBBYfWszuc7sB0AkdM/vP\n5OWRL9fmO6emqkzLHTvUftOnw5IlWgqlxrVRWqpCfuvXq+1Jk1RyTZcuajulIIVXd73KJ8mfIN12\n9PjY8cwZNIfJ3Sdj9L7KWFRpqbpIN21SYZ6amH4NPj7Qvz8MHqyU/sCBqiLbj6ylY6muJr2ykrM2\nG+luLz7D/TfTbqeo/kPnCrx72238ulOnq3dsATRF3wivZ2Tw+rlztU/fCL1ePZn1ejoYDHRyP6k7\nGQzNfzJbLEqrnj6tFPuJE5CcDGfOqBhlfXQ65QcPH64C5CNHNqlEpK3Kxvb07aw9uZZPT3xKmUNl\nNPjr/Xny9if55ZBf0rWdMtOzs+HNN1UpErtdpde/+65a3PsG80g1bhJcLhVmnz9f6WVfX5WCOX9+\n3cTrUwWneG//eyxPXE5FlUqxDDIG8VCvh5jacyqjo0dj8DZc/cOyspS3u3u3SiFOTr78PvLyUstk\n9emjkhN69lQTBbt1UwNQzaC8upoLDgdZdjuZlZW1UYAL9aIDOQ4HS+LimBER0azPaiqaom+EKpcL\nbyFaJtZmtaqLLzMTzp1TLT0dzp5V1kdubsP7eXlBjx6XWyNNuBCllJwsOMn29O1sPbuVLWe31N48\nAEM7DmVm/5k83u9xAgwBgHq+LF4Mf/+7GkQDtTrUO+8oZa+h0Vyys1Vq/Jo1attsVpk5c+YovQtQ\nbCtmeeJyViSu4EjOkdp9/fR+3H3b3dwVcxdjY8bSPaR70+5Pi+Vyrzgl5XLlX0NkpBpMiI2FmBjo\n3Fm5H507q8kivr7N/BbU/emU8rqVZG5RRS+EmAgsRI0zLpVSvtlAn/eAewArMEtKebSp+7r73RiD\nsTabGhwqKFCZLvn5SmHXtIsX1VV94ULdjL/GMBiUJdGtm3Ipa1rPnuq9qyClJLs8m2O5xzh48SD7\nL+xnf9Z+8ivyL+k3KHIQ8XHxTO8zvTbL4eJF5VKvWKGmrtfwwANqkmz//tf8zWhoXJWDB+GVV1Tm\nVg2jRqkQz+TJEB6uXjued5zVx1eTkJLAsdxjlxwj3BzOsKhhDO04lEEdBtEvvB8Rfk20kG02NY23\nxoM+cUJ51WfO1GWqNUa7dsqLjoxUf8PDISJC/Q0LUy00VGUCXePYQGvRYopeCKEDTgPjgIvAAWC6\nlPJUvT73AL+UUt4rhBgKLJJSDmvKvvWOcX0U/YYNqjRASYlqxcWq4l5xsZrF18AEpkbR69VEj06d\n6iyDmBhlMcTGqvd0Onbu3Mno0aMbPISUkiJbERklGWSUZHC2+CwphSmcLjzNyYKTFFQUXLZPhF8E\nY2PGMiZ6DBO7TiQqIIriYlXa5ttv1ekdOFDX399fpSg//bSKDrU0Vzq/toB2ftfOwYMqXr96dd2E\nWCFU9YNx4+DOO1WUMiAAMkoy2Jy2me3p29mevv0yQwagvbk9PUJ7EBcSR1xIHDHBMcQExRAdFE2Q\nMeiKHsDOnTsZPWKE8rzT05W3nZFR54VnZal2lcHYSzCZlMJv10614GA1Kh0UpKypO++8xm/sx9FU\nRd+U0cQhQKqU8pz7wJ8AU4D6ynoKsAJASrlfCBEohAgHYpqw7/Xl22/rFsdsCINB/YChoXVP8fDw\nuqd7ZGTdEz809JLgtku6sDqsWOwWLHYLpRe/p9hWzJLVS0gyJVFoKyTfmk+uNZdcay7ZZdlcKLtA\nZXVlo+IEGYPoH96fAREDuL39UKK9h+IsiCEjQ5D2KTyTDElJdfWjajCZYPx4tS7EQw+17kCrpghv\nblrj/AYPVlWwFyxQyv7zz1Va5t69qoG6dWJjoV+/aHr3nsPE2+Ywb4hEBqeRXrWPw7n7OZp7lGO5\nx8iz5pFnzatNMqiPydtEVEAUkf6RhJvDCTeHE+obSohvCCGmEDas24Cpq4lAv0ACBscRcMdPMPuY\nL304uFzKe8/OrvPa63vy+fmq1Xj7NlvdA+KHdO9+3RR9U2mKou8IZNbbzkIp/6v16djEfa8r33bt\nTPHM+7H6GqjwNWD19aHM14DF7IPF7IPVB6pxUuVyUO2qolo6cLjyqXJl4bDZsZ+pxHG6ErvTpprL\nhs1ppdJpxea0NvyhJ2Hd1+salclIAEGiCwGuGPyqYjDZuqG3xCGK4rBmR5FdJEjMg0VXiBTp9fCT\nn6jra8QIlWbfAiFHDY1mERio4vRz5qjKB9u3q1T5b79V4fW0NNU++6xmDwF0BboSHPw4YWHQK0Ti\nG3ke2S4Fh38KFaZUyn3SsejSKZHnsVWXkVqUSmpRasNCHIOPl3582csmLzNGLzMmLzMGnS9GLxMG\nLxN6nREfnQF9OwP6UAM+fX3x0XXFW9cLb+GDj/DGzyEJsFbjX1GFf4Udc0UV5opKzBV2IiLCGNxK\n3+ePpbWKQdywORy/yzjP3tjPLn3R4W7FLfABDjPYA8DuD5VBquVlwYFRUBGqWnk4WMOhPAIsUVQ6\n/MgBcq5yaB8f5VR06qTGlGJiVMi/b181DKCV8Na4kfH3VxOtpkxR2w6HGjtNSlJh9ZqoSlYW5OSo\naGpxMSh10sXd7r78wAYL+F8Avxx3ywVTIfgWgG8hFB+CCyFgLAF9GRjKQK8MM5vT2vzb3hsIcDdg\netrd/Ku5x2xhmhKjHwa8IqWc6N5+EbXy+Jv1+iwGdkgpV7u3TwGjUKGbK+5b7xg3wEishoaGxs1F\nS8XoDwBdhRBdgGxgOvDID/p8ATwDrHY/GEqklLlCiIIm7NtkYTU0NDQ0rp2rKnoppVMI8UtgM3Up\nkieFEHPV2/JDKeVXQohJQogzqPTK2Vfat9XORkNDQ0PjMm6YCVMaGhoaGq3DDVWYWQjxKyHESSFE\nkhDiT56WpzUQQvxGCOESQrTztCwtiRDiLfdvd1QI8akQIsDTMjUXIcREIcQpIcRpIcQLnpanJRFC\nRAkhtgshjrvvt2c9LVNrIITQCSEOCyG+8LQsLY07jf3f7vvuuHsOU4PcMIpeCDEa+BnQV0rZF3jb\nsxK1PEKIKGA8cM7TsrQCm4HeUsoBQCrwnx6Wp1m4J/t9AEwAegOPCCF6eFaqFqUaeF5K2RsYDjzT\nxs6vhueAE54WopVYBHwlpewJ9AcaDYvfMIoemAf8SUpZDSClvHxK6M3PAmC+p4VoDaSUW6WUNUVG\n9gGNrw13c1A7UVBKWQXUTPZrE0gpc2rKlEgpy1FKok2tDuw2rCYBSzwtS0vj9phHSCk/ApBSVksp\nLY31v5EUfXdgpBBinxBihxDiRptz0CyEEPFAppQyydOyXAeeBDZ6Wohm0tgkwDaHECIaGADs96wk\nLU6NYdUWByJjgAIhxEfu0NSHQohGC/Bc19VzhRBbgPD6L6F+hP9yyxLsrpHzE2ANEHs95WsuVzm/\nl1Bhm/rv3VRc4fx+L6Vc7+7ze6BKSvlPD4iocY0IIfyAtcBzbsu+TSCEuBfIlVIedYeFb7r77Sp4\nAwOBZ6SUB4UQC4EXgf9urPN1Q0o5vrH3hBC/ANa5+x1wD1iGSCkLr5uAzaSx8xNC9AGigUShCmxE\nAYeEEEOklHnXUcRmcaXfD0AIMQvlKo+9LgK1LheA+uvfRblfazMIIbxRSn6llDLB0/K0MHcA8UKI\nSYAJ8BdCrJBSzvSwXC1FFipCcNC9vRZoNGHgRgrdfI5bQQghugM+N5OSvxJSymQpZYSUMlZKGYP6\nkW6/mZT81XCXo54PxEsp7Z6WpwWonSgohNCjJvu1tcyNZcAJKeUiTwvS0kgpX5JSdpZSxqJ+u+1t\nSMkjpcwFMt26ElSF4EYHna+rRX8VPgKWCSGSADvQZn6UBpC0PVfyfUAPbHFXBdwnpXzasyL9eNr6\nZD8hxB3AY0CSEOII6pp8SUr5tWcl07gGngU+FkL4AGdxT1RtCG3ClIaGhkYb50YK3WhoaGhotAKa\notfQ0NBo42iKXkNDQ6ONoyl6DQ0NjTaOpug1NDQ02jiaotfQ0NBo42iKXkNDQ6ONoyl6DQ0NjTbO\n/wM4DBzbkfFqGAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10bc36b38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot( x, scipy.stats.norm.pdf( x ), lw = 2 )\n",
    "plot( x, scipy.stats.t( df = 10 ).pdf( x ), lw = 2 )\n",
    "plot( x, scipy.stats.t( df = 3 ).pdf( x ), lw = 2 )\n",
    "plot( x, scipy.stats.t( df = 1 ).pdf( x ), lw = 2 )\n",
    "legend( [\"Normal\", \"T df = 10\", \"T df = 3\", \"T df = 1\"] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus given two samples, a t-test answers the following question:\n",
    "\n",
    "> Suppose that each of these two samples was drawn from a normally distributed population.  What's the probability that we'd get two samples this different *if* the underlying population is actually the same?\n",
    "\n",
    "For our BIO508 and BIO200 classes, we can \"see\" the data the same way a t-test will:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x10c223ac8>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEACAYAAABVtcpZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VFX6+PHPM6m0hN4SCL2qFBFBBAM2QAVUVLAgrLq4\nuyquu4q6/ihf115W1HXtIIpioymISImKFLHQewmQEHoLgfTz++NMZiYxkECmJfO8X6+8MufOnXvP\nTCbzzD3lOWKMQSmlVGhyBLoCSimlAkeDgFJKhTANAkopFcI0CCilVAjTIKCUUiFMg4BSSoWwUgUB\nEekrIhtFZLOIjD7NPq+KyBYRWSkiHT22x4rI5yKyQUTWicjF3qq8UkqpsikxCIiIA3gduBpoDwwV\nkTZF9ukHNDfGtARGAm963D0BmGOMaQt0ADZ4qe5KKaXKqDRXAl2BLcaYncaYHGAqMLDIPgOByQDG\nmOVArIjUE5EYoKcxZqLzvlxjzHHvVV8ppVRZlCYIxAG7Pcopzm1n2ifVua0pcFBEJorIbyLytohU\nKkuFlVJKeY+vO4bDgc7Af40xnYGTwKM+PqdSSqlSCi/FPqlAY49yvHNb0X0anWaf3caYX5y3vwBO\n17GsSYyUUuosGWOkLI8vzZXACqCFiCSISCQwBJhVZJ9ZwDAAEekGHDXG7DPG7AN2i0gr536XA+tP\ndyJjjP4Yw9ixYwNeh2D40ddBXwt9Lc784w0lXgkYY/JE5D5gHjZovGeM2SAiI+3d5m1jzBwR6S8i\nW4EMYITHIR4ApohIBLC9yH1KKaUCqDTNQRhj5gKti2x7q0j5vtM8dhVw0blWUKlzlZ2dzeLFS8jN\nzfPJ8bt06UTNmjV9cmyl/KVUQUD5V2JiYqCrEBTK+jqkpKTwzjuriI7u7J0KeThyZDsPPhjmt7+V\nvifc9LXwLg0CQUjf5JY3XocqVWoQH9+z7JUpwhjfXF2cjr4n3PS18K6gDwJNmjRh586dga5GhZSQ\nkEBycnKgq6GUCqCgDwI7d+70Wi+4KkykTCPLlFIVgGYRVUqpEKZBQCmlQpgGAaWUCmEaBJRSKoRp\nECiDJk2aULlyZWJiYqhVqxbXXXcdqak2ZdKIESMYM2aMa9/s7Gwee+wxEhISqFKlCq1bt+bFF18s\ndLwRI0YQFRVFTEwM1apVIyYmplCn+MqVK+nSpQtVqlThoosuYtWqVYUe/8QTTxAfH0+NGjXo06cP\n69efNkOHUkoBGgTKRESYPXs2x48fJy0tjbp163L//fcXu+/gwYNZtGgRc+fOJT09nQ8//JC3336b\nUaNGFdpv9OjRHD9+nPT0dI4fP+4awZOTk8OgQYMYNmwYR48eZdiwYQwcOJDc3FwAPvvsMyZNmsRP\nP/3E4cOH6datG3fccYdvXwClVLmnQaCMCr6pR0ZGMnjw4GK/fS9YsID58+czbdo02rZti8PhoGvX\nrnz00Uf897//Zfv27SWeJykpiby8PB544AEiIiK4//77McawcOFCAJKTk7n00ktJSEhARLj99tvZ\nsEEXcVNKnVn5DgIi3v0pg5MnT/Lpp5/SvXv3P9w3f/58Lr74Yho2bFhoe9euXYmPj2fBggWubW+8\n8Qa1a9fmoosuYtq0aa7t69at44ILLij0+A4dOrBu3ToAhgwZwrZt29iyZQs5OTlMmjSJfv36lek5\nKaUqvqCfLBbsBg0aRHh4OCdOnKBu3bp8++23f9jn4MGDNGjQoNjHN2jQgIMHDwIwatQoXn75ZWJj\nY/n222+55ZZbaNCgAd27d+fEiRPExsYWemxMTAzp6emu4/To0YPWrVsTHh5Oo0aNXFcJSil1OuX7\nSiAIzJw5k8OHD5OVlcVrr71Gr1692L9/f6F9ateuTVpaWrGPT0tLo3bt2gB07NiRGjVq4HA46Nev\nH7fddpvraqBq1aocP154eeZjx45RrVo1AMaPH8+KFStITU0lMzOTMWPG0Lt3bzIzM739lJVSFUj5\nDgLGePfnnKpgHyciXH/99YSFhbF48eJC+1xxxRUsX77cNXKowPLly0lJSaFPnz7FHltEXMdv3749\nq1evLnT/6tWrOe+88wBYtWoVQ4YMoUGDBjgcDu68806OHDmiI4SUUmdUvoNAkJk5cyZHjx6lbdu2\nhbZffvnlXH755dx4442sX7+e/Px8li1bxh133MFf//pXmjdvDsCXX35JRkYGxhjmzZvHlClTGDBg\nAGAzJ4aFhfHaa6+RnZ3Nq6++isPhoHfv3gBcdNFFfP755+zfvx9jDB9++CG5ubm0aNHCvy+CUqpc\n0T6BMrruuusICwtDREhISGDy5Ml/CAJgP+DHjh1L3759OXToEHFxcdxzzz08/PDDrn0mTJjA3Xff\njTGGpk2b8u6779KrVy8AIiIimDFjBnfddRePPvoobdu2ZebMmYSH2z/h6NGjOXDgAB07duTkyZO0\naNGCadOmERMT458XQilVLkmwZOgUEVNcXTybRJR3VfTXdvv27Tz99GLi44d5/djJyUkMH6657VVg\nOf+Hfb7QvFJKqQpKg4BSSoUwDQJKKRXCNAgopVQI0yCglFIhTIOAUkqFMA0CSikVwjQIKKVUCNMg\noJRSIUyDQBl4e3nJhx9+mFatWhEbG0u7du348MMPC91f0vKS//nPf2jQoAHVq1fn7rvvJicnx0fP\nXClVUZQqCIhIXxHZKCKbRWT0afZ5VUS2iMhKEenksT1ZRFaJyO8i8rO3Kh4MvL28ZNWqVZk9ezbH\njh1j0qRJjBo1imXLlgElLy/57bff8vzzz7No0SJ27tzJtm3bGDt2rO9fBKVUuVZiEBARB/A6cDXQ\nHhgqIm2K7NMPaG6MaQmMBP7ncXc+kGiM6WSM6eq1mgcJby4vOXbsWFq2bAnYVcd69uzJ0qVLAVi0\naNEZl5ecPHkyd911F23atCE2NpYxY8YwceJEf7wESqlyrDRXAl2BLcaYncaYHGAqMLDIPgOByQDG\nmOVArIjUc94npTzPWQui1SW9trxkgVOnTrFixQrXegHr168/4/KS69ato0OHDoXu279/P0eOHCnb\nE1NKVWilSSUdB+z2KKdgA8OZ9kl1btsHGOA7EckD3jbGvHPu1Q0+3lxe0tO9995Lp06duPLKKwFK\nXF6y6P0xMTEYY0hPT6dGjRrn/PyUUhWbP9YT6GGMSROROthgsMEYs7jER5UTM2fOpHfv3hhjmDFj\nBr169WLDhg2F9qlduzZbt24t9vGey0sWePjhh1m/fj2LFi1ybStpecmi9x87dgwRcd2vlFLFKU0Q\nSAUae5TjnduK7tOouH2MMWnO3wdEZDr2KqLYIDBu3DjX7cTExBJztQdDKvyiy0uOHDmy2OUlJ0yY\nQGpqKnFxca7txS0vOXbsWL799lt++OEHqlat6trevn17Xn755ULHXb16NQ888IDr/lWrVjF48GDA\njiSqV6+eXgUoVYEkJSWRlJTk1WOWJgisAFqISAKQBgwBhhbZZxbwN+BTEekGHDXG7BORyoDDGHNC\nRKoAVwHjT3cizyBQHnkuL/nVV1+5tnsuL/n+++/Tpk0bfv75Z4YNG1ZoeclnnnmGTz75hMWLF1O9\nevVCx/ZcXnLkyJG8+eabhZaXHDZsGCNGjODWW2+lfv36/Pvf/2bEiBH+e/JKKZ8r+uV4/PjTfpyW\nWolBwBiTJyL3AfOwHbzvGWM2iMhIe7d52xgzR0T6i8hWIAMo+PSpB0wXEeM81xRjzLwy1zqIeHN5\nyX/9619ERUXRokULjDGICI8//jiPPvpoictLXn311TzyyCP07t2bzMxMBg8eXO6DqlLK93R5yRBW\n0V9bXV5SVXS6vKRSSqky0SCglFIhTIOAUkqFMA0CSikVwjQIKKVUCNMgoJRSIcwfaSPKJCEhASlr\ndjdVrISEhEBXQSkVYEEfBJKTkwNdBaWUqrC0OUgppUKYBgGllAphGgSUUiqEaRBQSqkQpkFAKaVC\nWNCPDlIqWJ04cYLt27f79BwxMTF/WHlOKW/SIKDUOVo4YwZ7v/iCyHDf/Bvl5+ezv3p1HimyopxS\n3qRBQKlzlJ+VxYD69albpYpPjp+Vm8vLhw755NhKFdA+AaWUCmEaBJRSKoRpEFBKqRCmQUAppUKY\nBgGllAphGgSUUiqEaRBQSqkQpkFAKaVCmAYBpZQKYRoElFIqhGkQUEqpEKZBQCmlQpgGAaWUCmGl\nCgIi0ldENorIZhEZfZp9XhWRLSKyUkQ6FrnPISK/icgsb1RaKaWUd5QYBETEAbwOXA20B4aKSJsi\n+/QDmhtjWgIjgTeLHGYUsN4rNVZKKeU1pbkS6ApsMcbsNMbkAFOBgUX2GQhMBjDGLAdiRaQegIjE\nA/2Bd71Wa6WUUl5RmiAQB+z2KKc4t51pn1SPff4DPAyYc6yjUkopH/HpymIicg2wzxizUkQSATnT\n/uPGjXPdTkxMJDEx0ZfVU0qpciUpKYmkpCSvHrM0QSAVaOxRjnduK7pPo2L2GQwMEJH+QCWgmohM\nNsYMK+5EnkFAKaVUYUW/HI8fP77MxyxNc9AKoIWIJIhIJDAEKDrKZxYwDEBEugFHjTH7jDGPG2Ma\nG2OaOR+38HQBQCmllP+VeCVgjMkTkfuAedig8Z4xZoOIjLR3m7eNMXNEpL+IbAUygBG+rbZSSilv\nKFWfgDFmLtC6yLa3ipTvK+EY3wPfn20FlVJK+Y7OGFZKqRCmQUAppUKYBgGllAphGgSUUiqEaRBQ\nSqkQpkFAKaVCmAYBpZQKYRoElFIqhGkQUCpIGM2zqwLAp1lElVKnd+hkJb7Z2pLlKXH8vCeOlXvr\nExWWR3zMceJjjtOy5n4cTRYGupqqgtMgoJSfHc+K4uWl3XlpaXdOZEcVui87L5wNB+uw4WAdvtve\nHH7pzsqecP/9cMMNEK7/scrL9C2llJ/kG+G15V158ofLOHSqcqkft3ix/enSBaZOhebNfVhJFXI0\nCCjlB+lZkdw+/QZmbSq0PDft6uznlvbr6BqXSpeGe3CIIeV4DMlHqzNlTTu+2HAe+flhAPzyC3Tq\nBG+/DUOGBOJZqIpIg4BSPpZ8tDoDPhnKmv31XNuaVD/C+MQkbjt/NWGOwj3CNSud4oJ6+7i6+Tqa\ndf6YsMbP8fzzkJMD6ekwdCgsWABvvAEREf5+Nqqi0dFBSvnQ8pQ4ur5zT6EA8I/uS9j4t9cZ1mHV\nHwJAUTFVjvPvf8PSpYWbgd59F4YNg7w8X9VchQoNAkr5yLr9deg75XYOnKwCQIQjj4kDZ/DiVfOI\nCj+7T+8LL4TffrNXAQWmToURIzQQqLLR5iClfCDleAx9p9zO0cxKANSunMGMW6bSo/Hucz5mTAxM\nmQK1asHrr9ttH34IkZG2n8ChX+nUOdC3jVJeduRUNH0/up2U47EAVI3MYu5tH5UpABQQgQkT4J57\n3Nveew+eeKLMh1YhSoOAUl6Uk+fg+k+HsO5AXQDCHXlMu/lTLmyY5rVzOBzw5pswfLh72zPPwMyZ\nXjuFCiEaBJTyovHfJ/L9ziau8sSBM7my+Xavn8fhsJ3Dffu6t915J2zd6vVTqQpOg4BSXrJwR1Oe\n/rGnq/xk74XcfsFqn50vLAw++ggSEmz52DG48UY4edJnp1QVkAYBpbzgQEZlbp92AwYB4Ipm23i8\n548+P2+tWvDFF7ZzGGD1arjvPp+fVlUgGgSUKiNjYMTMQaSdqAZAncoZTB40HYf4Jy1oly7w2mvu\n8sSJ8PXXfjm1qgA0CChVRm//eiGzt7RylT8YNJ0G1U74tQ733FN4DsG999rmIaVKokFAqTLYd6Ia\no+df6So/1G0J/Vr6v3dWxF4N1LWDkkhNhYcf9ns1VDmkQUCpMvh/iwZwLCsagJY1D/HU5YHL/+85\niQzgnXdsjiGlzkSDgFLnKDn1AmZu6ugqv3nt10SH5wawRjB4sF13oMA990BGxjkcyBj7wD17IDvb\na/VTwUeDgFLnICfHwcKfh7vKwzqspE/THYGrkJMI/Pe/UKOGLe/YAc8+W8KDjIHNm+HVV6F/f4iP\nh0qVoGpViIuD6Gho3BgSE+HBB+3iBvn5vn4qyk9KFQREpK+IbBSRzSIy+jT7vCoiW0RkpYh0dG6L\nEpHlIvK7iKwRkbHerLxSgbJ6dRPSM+oAUKvSSV68cl6Aa+RWvz68+KK7/MILkJxczI6ZmXbqcZs2\n0Lo1jBoF33xjOxSystz7GQO7d8P339ucFT172qDw0EOnObAqT0oMAiLiAF4HrgbaA0NFpE2RffoB\nzY0xLYGRwJsAxpgsoLcxphPQEegnIl29+xSU8q/Dh2H9+nhX+cWr5lGnSnDN0Bo+3A4dBft5XqiT\n+ORJ+M9/oFkz+Mtf7FVAcaKioHZte3lRVGqqPUbr1vD3v8PBg95+CspPSnMl0BXYYozZaYzJAaYC\nA4vsMxCYDGCMWQ7Eikg9Z7ngvyMKm7XUP4OnlfKRBQsgP9/+63RpmMydHVYGuEZ/5HDY1p0CX3wB\nSUnATz9Bhw72W3yaRz6jqlVh4EB7ZbBpE5w4AadOwYED9vemTfDVV/DnP9se6ALZ2fDKK3axgxdf\n1Gaicqg0QSAO8Ex/mOLcdqZ9Ugv2ERGHiPwO7AW+M8asOPfqKhVYu3bB+vXu8vjEr4r9ohwMuneH\n225zlx+8eQ95l15WOMFQgwb2G/3evTBjBowcCa1aQZUq7iuAqCi77dpr4a23bPCYM8eeoMDx4/Zy\n46qrbGeyKjd8vp6AMSYf6CQiMcAMEWlnjFlf3L7jxo1z3U5MTCQxMdHX1VOq1IyBeR5N/60SltGl\n4S6gSsDqVJJnn4Xp0/I5ecrBqgMNeY8/8WfesYsTPPUU3H237fg9GxER0K+fzV43cyY89hhs3Gjv\nW7DAXmlMmgTXXOP15xPqkpKSSEpK8uoxSxMEUoHGHuV457ai+zQ60z7GmOMisgjoC5QYBJQKNuvW\n2aZwAIcjn0s6fQaEBbROJYnfvZRH5XvG8CgA4xjH7X3SqDzpDWjUqIRHl0AEBg2yVwjjx9ugYozt\nH7j2Wnj+eZ2x5mVFvxyPHz++zMcsTRBYAbQQkQQgDRgCDC2yzyzgb8CnItINOGqM2ScitYEcY8wx\nEakEXAmUNGBNqaCTmwvz57vLbdumEFv1AFDfJ+cbs2gVO47ksS4ri03Dx53TMS5KXseff5zOP/Mj\neJM72EMcaTTkOh5hQVkDgKfwcHjySejTB26/3d0c9MgjtpnphRd02bMgVmIQMMbkich9wDxsH8J7\nxpgNIjLS3m3eNsbMEZH+IrIVyABGOB/eAPjAOcLIAXxqjJnjm6eilO/8+qs7F0/lynDBBbt8er5d\nxwwJ1R9kz8mTNGly7Vk//qIVb3DN9/ZbYgR5jI54mVE5LwGweHEXjh6F6tW9WmXo3RtWrbL5rH/4\nwW57+WXYtw/ef9+d6lQFlVL1CRhj5gKti2x7q0j5DwlsjTFrgM5lqaBSgZaTAz96ZIXu2RMiI3Mh\nSBd4v/DXt7lmzt9c5YO1WpN7y1+pOdUOb83OrsQLL9jWG6+rXRu+/db2SE+bZrdNmQLp6XaIUkSE\nD06qykKv0ZQqwc8/u1MvVKvmHn8fjDqunMR1X490lXfHd+O9u5ZwvE5zPMdZvPKKbanxieho+Owz\nm8q0wKxZMGKEDiENQhoElDqDrCw7tL5Ar162CTwYnb/mYwbO/JOrnNqwCx/dNpdTlWoCcN55UK+e\nve/kSR9dCRQIC4M33ijcMTxlCtx/v+08VkFDg4BSZ7B8uZ0rBbYNvVOnwNbndJrsWMSgGXcizrmY\nafU78uHt88iKjnXtI2L7bgu89ZbNBuEzIvDcc3buQYE33oAnnvDhSdXZ0iCg1GmcOgVLlrjLl11m\nv+AGm1oHN3HLZzcSlm8zmO6rex4f3vEdmZVq/GHfli2hTh3bqZ2TU4rkcmVVkNHOc8Wbp5+2HcUq\nKGgQUOo0li1z51GrVQsuuCCw9SlO5ZMHue3ja6iUeQSA9KoNmHLrHE5Wrl3s/iLQocP3rvK777rn\nPvhMWBh88EHhyWP33muzkaqA0yCgVDEyM21TUIHLLgu+oe5huVnc8un11DyyDYDsiMp8PPQrjsee\neQ5Aw4bb6NbN3s7Oti02PhcRAVOnuiNpTo5d+GDnTj+cXJ1JkHZxKRVYP/9c+CqgffvA1qc4V837\nJwm77LdpgzDthimkNbywxMeJwJgxdukAgLffhkcfhYYNfVfXV8aM4eiuXcS2bMmfN2+mSmYmHDjA\n3k6deL9fP7K9MHR00/bttG7WzAu1Pb3qjRvz4P/9n0/P4W8aBJQqIjvbNgUVuPTS4LsKOG/tVC5e\n4V5Lcv4Vz7KxzaBSP75vXzvU9ZdfbLB74QWbR85Xju7axbgmTWwhJsY2D+XnU//IER5fswauv774\nlNVnYdDixYzz7Pn2gXEVcP2EIHtrKxV4v/xSeETQ+ecHtj5F1T6wgQGz7naV17UbzE+XnF2OHhEY\n67HE05tv+nDeQFGNG9vcQgXWrIHff/fTyVVRGgSU8pCTU3hE0KWXBteIoMjsE9zy2Y1E5tjZa4dq\ntmTWgPfO6Vv0NddAZ+d8/sxM314J/EGnToXH237zjU0vofxOg4BSHn77zT07OCbGZkUOJv3n3Eed\ngxsAyAmvxKc3f0lWVMw5HUuk8JD9//0PjhzxRi1LqV8/qFvX3s7Nhc8/L7yspfILDQJKOeXlFb4K\nuOSS4Jod3H7dZ3Rc9YGr/PU1/2N/vbK1VQ0cCG3b2tvp6XZIv99ERMBNN7nzCR06ZBerUX6lQUAp\np9Wr7QJZYBfW6hxEqQ9jju3mWo+cQKsuuINVHe8s83EdDjsyqMCECTalhN/Url24f2D1artwg/Ib\nDQJKYdPZeOYI6tYteBJeSn4e188YRqXMowAcqd6EOf1fL+FRpTd0KCQk2NsHD9oJZH51wQWF292+\n/todjZXPaRBQCrs64qFD9nZUVHBlCr1k6Us0TU4CIF8cTLv+o3PuByhORAT885/u8gsv2GGyftW3\nL8Q68xxlZtqso5pozi80CKiQV/QqoEuXs19211eandxBn4Xu3tsfe/6L3Y17eP08d93l7qNNSbEJ\nP/0qOtouVVlg2zZYscLPlQhNGgRUyEtOdufPCQuDiy8OaHVcHPm5PJL8CmH5OQCkxHXl+17/zyfn\nqlQJHnzQXX7uuQCk/m/SxPbGF/juO/flmfIZDQIq5HnmMevY0S4cEwx6pXxCy1PbAcgNi2LGoA/I\nD/NdR8Vf/+p+7ps2wVdf+exUp9e7t3vRg9xcbRbyAw0CKqSlpcF2+zmLSOEvooEUf+oYibs+cpUX\n9n6Sg7Xb+PScsbGFFwN77rkAfP6Gh9tmoYI8Hbt22UROymeCaBS0Uv7n2RfQrh3UrFn6xy5duYv7\n8w9TycvDiMJMPv/cvoJwY9cH2B3fjaXdH/LqOU7nwQftMNHsbFi61F4l9exZuseOGfMKu3YdLfa+\nlYtTSF5Z/H3Fub5OCwbu2wxA1rzveONQHn/vHyQRuoLRIKBC1pEjsH69u9zjLPtb0zPCaFx/FFUi\nI71ar547P6Jtrp22nC0RzBw4EePwT+6Khg3hjjvgvfds+fnnSx8Edu06SpMm44q9L3nlDJpUr17q\neqyOyaHbiZHUy9hBVH4efdf/Dv26lznJnPojbQ5SIWvpUndzR7Nm0KBBYOsDUOvkLi7bOdlVntjw\ndp83AxX18MPuz9qvv4a1a/16egDyHBHMbPUI+c6PqLYZB+HXX/1fkRCgQUCFpIyMwokrz/YqwBfE\n5HPd5pcIN3Y0UGrVVnxRb6Df69G6deHRmi+84PcqALAnpg1LGt3i3jB/vs1tobxKg4AKSStW2MEn\nAPXrQ9Omga0PQOe02TQ5thqAXGB6y3+QL4FJYfrII+7bH3/s4wXpzyCpyXAOVYq3hawsm21UeZUG\nARVysrMLDzjp0SPwTc3Vsg5y5fa3XOV3o+PZW7VFwOrTrRv06mVv5+bCK68Eph65jki+avUP94YN\nG+z0buU1GgRUyPn998KLxrRrF9j6APTb+hrRec41AirF8Vp04wDXCEaPdt9++20/p5n2kFy9I9/X\n9Hg95szRlNNepEFAhZT8fNshXKB798AvHdnq4BLaHfzBVf6q5T/IClAzkKd+/eC88+ztEyfs6mOB\n8lmD9ja1K9h+gfnzA1eZCkaDgAop69bBsWP2duXKhRe3CoTIvFP03/qqq/x7/b4k1whwpZxE7Eih\nAhMm2NxugZARHmmTzBX45Rd3rg9VJqUKAiLSV0Q2ishmERl9mn1eFZEtIrJSRDo6t8WLyEIRWSci\na0TkAW9WXqmzUTRRXNeugU8XnZg8iepZdlnFjIhY5jW7t4RH+NeQIRDv7Jfdtw8mTz7z/j7Vvj20\nbOkuf/11ABIcVTwlBgERcQCvA1cD7YGhItKmyD79gObGmJbASKDgwjEXeMgY0x7oDvyt6GOV8pft\n293L2EZEwEUXBbY+9U9spVvKF67yvGb3cioiNoA1+qPISPj7393lF1+0K7AFhAj07+9e7m3vXli+\nPECVqThKcyXQFdhijNlpjMkBpgJFBy8PBCYDGGOWA7EiUs8Ys9cYs9K5/QSwAYjzWu2VOgueVwGd\nOtnmoEARk8e1m1/Cgf0muyO2I6vqXR24Cp3BPfe4U/1v2QIzZgSwMtWrw2WXucuLFrnb99Q5KU0Q\niAM8Rwmn8McP8qL7pBbdR0SaAB0BDd3K7/bsgR077G0R2yEcSBemfU18uh3qmCsRfN3q74Efp3oa\n1arZDKMFApJYzlP37u7FD3JydO5AGfkld5CIVAW+AEY5rwiKNW7cONftxMREEhMTfV43FRo8rwLO\nO89+oQyUqtmHuWL7O67y4sZDOVQ58ENCz+SBB+Dll+3IzBUrICnJZn0OiLAwuy7x++/b8qZNITN3\nICkpiaSkJK8eszRBIBXwfIfGO7cV3adRcfuISDg2AHxojJl5phN5BgGlvOXQITvHqECg00Vfte1/\nheYELG58W2ArVAr168Pw4fCWcz7bs88GMAgANGoEnTvDb7/Z8ty5RJ1NCthyquiX4/Hjx5f5mKVp\nDloBtBDurHVVAAAfaElEQVSRBBGJBIYAs4rsMwsYBiAi3YCjxhhnFxzvA+uNMRPKXFulzsGSJe7m\ni+bN7QdaoDQ98isX7HePcZ/d4kFyHd7NQuor//yne07FvHmFcy8FxBVXuDt2jh3jlkDNZivnSgwC\nxpg84D5gHrAOmGqM2SAiI0Xkz8595gA7RGQr8BbwFwAR6QHcBvQRkd9F5DcR6VvsiZTygZMnK7Fq\nlbt86aWBq0t4fjbXbnHnX1hTpw/bawbRivYlaNECBg92l59/PnB1AeyamFde6SoOOHoU9u8PYIXK\np1LNEzDGzDXGtDbGtDTGPOvc9pYx5m2Pfe4zxrQwxnQwxvzu3PaTMSbMGNPRGNPJGNPZGDPXN09F\nqT9au7ata0hjfDwkJASuLj12fUKtUykAZIZV4dvmfy3hEcHHM5XEZ5/Z9eADqkMH1x81HGD2bF2O\n8izpjGFVYR0/7mDjxlauciATxdU8lUrPXVNc5YVN7+JEVK3AVKYMOne2rTBg52m9+GJg64MIXHNN\n4eUoV64MbJ3KGQ0CqsKaMiWGnBzb3l67ts2THxDG0H/LK+51Aqq1ZkXDAQGqTNk9+qj79sSJdp3m\ngKpTp3Bv/3ffwcmTgatPOaNBQFVIp07BxIkxrnIgrwLaH1hEiyO/AJCPg69b/h0TBAnizlWfPu7Z\n1llZduhowPXqxb6CmcSnTtlAoEpF1xhWFdJ778GhQ/btHRMD558fmHpE5Z6g77Y3XOUVcQNJq1b6\nS5K01FSSvDxF9+iJxYwbPtxV3rR9O62bNTurYzSr2pkV2FRgE17JxJHyD1au2ETyyuLreiAtzbeT\nMyIieLt2bf7f3r22vHIldOwY2E6gckKDgKpwsrPtrNYCl1xi5xcFQp/k96mWfQiA9MhaLGzyp7N6\nvMnOJtHLH57JVGVckyau8qDFixnXp89ZHSM/4Qhr1+5n3YG65ORGUyntJjpWfeW0i8lP3bWrLFUu\nlV+rVIG2bd2TQmbPhpEjA/fHLye0OUhVOJMnQ4odhEN09Ck6dw5MPRoe30jXVPc347nN/0ZWeNXA\nVMbLHGJ47NIfXeVXl19MTl50AGvk1LevzXoHcOCAnSSizkiDgKpQcnPhmWfc5fPPXx+QdNEOk8d1\nW15GsMMVt9a4iHV1Ev1fER+65bx1NKtxGIAjmZXYdCgIpgDFxIBnupkffgjckmjlhAYBVaFMnWpT\nRgNUr55HmzabAlKPrqnTaHBiCwA5jkhmt3wwaBPEnatwRz6je7iTMq07MJCcvCD4SLn4Yve08Nxc\nnTtQgiD4iynlHfn58NRT7vLw4ceIjMz1ez1iM/fRZ8f7rvL3CcM4Uqmh3+vhD3d2WElcteMAZObW\n4Le0BgGuEXbOwLXXusvbttkl5VSxNAioCmPaNHcyyZgYuPPO4wGpR7+trxGZb9dh3F+5CUvjbw5I\nPfwhKjyPRzyuBhbvbkxufhB8rMTF2aXjCsyda4eOqj8Igr+WUmWXnw+eCRXvuw9iYvy/9GDbAz/Q\n5pD7Q/GrVg+R5wjwGpY+dk/nX2lQNR2AE9lR/BoMVwNgJzRUq2ZvZ2To4vSnoUFAVQhffglr19rb\nVasWXhLRX6JzT9B/qztZ7q8NrmF3bIAmKPhRpYhcRvdY7Cr/tCtIrgaioqBfP3f5t99g587A1SdI\nBcFfSqmyycsDz6Uo7r/fponwtyu2v0W1bDtaJj2yJt8F2aLxvvTnC3+lUrjzuWdHBUffAECbNoXz\nhXz1le0sVi4aBFS59/nnsH69vV2tGvzjH/6vQ8LRVXRJ+9pV/qbFA2RWkDkBpVEpIpfz6k53lRcH\ny9VAweL0BXMHDh2yw0aVSxD8lZQ6d3l5hfsCHngAavk5OWd4fjbXbX7JVd5Yqwfra/fybyWCQKta\n31I1MguwVwNB0zcQE+NOfQp2rdF9+06/f4jRIKDKtalTC48Ieugh/9eh184PqX1qNwBZYZWZ3XJU\nhZsTUBrhjmx6NNrtKv+4M4HsYJg3ANClCzR2rpKbnw+zZtnfSoOAKr9ycgpfBTz4IPh7mdm4E1u4\ndNfHrvL8pveQHlXHv5UIIl0a7iEmyg6PzciJ5OfU+ADXyEkErrvOnUdozx5YvjywdQoSGgRUuTVx\nImyxk3KJjfX/iKAIk8/N217Cgf1GmRx7Ab+U43UCvCHckU+vBPcInJ92NyIzN0jyVNauDb08mukW\nLrR9BCFOg4Aql06eLDwiaPRo32YqLs49R3fS8OQOAHIcUcxq/QhG9F+qY7291KxkF3XJzI1gye5G\nAa6Rhx49oF49ezs3F2bODPlmIX3HqnLp1VfdK1o1aACjRvn3/PGHtzHyqPsb74Kmd3G4Upx/KxGk\nwhyGxCbJrvKylHhy8ysFrkKewsJg0CD3cpS7d4d8s5AGAVXuHD4Mzz7rLo8dC5Ur++/8YXnZ3P3T\nc0Q4M4TujmnP8rgb/FeBcuC8OvupV+UEADn5YRzKvCjANfJQvz707Okuh3izkAYBVe48+ywcO2Zv\nt2wJfzq7dVrKrNf3T9LksDNDqEQws/Uj5Xq5SF8QgT5Nd7jKR7PO59DJILkaABsEtFkI0CCgypmU\nFHjtNXf5qafw63oB8SnL6Ln4aVf5m8Z/4mDlxv6rQDnSsuYhGscedZbCmL/j7Jaw9KnimoVCdAEa\nDQKqXHn0Uci0IxDp0gUGD/bfuSNyTnL99GE4jP3G+HN0dRY3GOS/CpQzInB1822u8saDddh5NDaA\nNSqifv3Co4UWLXJ3NIUQDQKq3Fi2DKZMcZdffNG/c7Ku+G40tZzNQKciKvN47TY6GqgEDaulc35d\n9+zcb7e1CK71XXr2tGmnwTYHTZ9uJ6CEEH0Hq3IhP99OBitw441w2WX+O3/LLXO4eMXrrvKUi+5j\nT0QQtXEHscubbkewSdvSTlRjzf56Aa6RB4cDrr/e3aZ44AAsWBDYOvlZkMziUOrMPv7YPZIvMhKe\nf95/566ansagGcNd5Y2tB7C4RV9IW+G/SnjR73v3MXzGSlf5t32VCpXP+bhph2lSzFyN2OgsakT/\nzmHnCKEFO5rStvYBIsKCpCO2Vi246iq7DCXYN1qLFvYnBGgQUEEvI8P2BRR46CFo5qc+RjH5XD/j\nTqqcPABAetUGzBrwHuxf658K+EBGdgRNqrsvqypHrKZJ9QvKfNzFu8ac9r5a0b+QldeBjJxIjmdF\n88OuBC73GD0UcBdeaKefb95syzNmwL332sUpKrhSNQeJSF8R2Sgim0Vk9Gn2eVVEtojIShHp5LH9\nPRHZJyKrvVVpFVqefRZSU+3tevXg8cf9d+7uS1+m+fbvADAI067/kJOVA7BYQTnnkBwub7rdVV6y\nu1FwDRkVgQED3B/6GRl2vdKg6sDwjRKDgIg4gNeBq4H2wFARaVNkn35Ac2NMS2Ak8D+Puyc6H6vU\nWdu4EZ57zl1+6in3ioG+1jB1BZcveMxVXtxjNDuaXe6fk1dAHevvJT7GTvDINw7mbG0ZXJ+xVarY\n/oECO3bA4sWn37+CKM2VQFdgizFmpzEmB5gKDCyyz0BgMoAxZjkQKyL1nOXFwBHvVVmFCmPsFXnB\nYI1u3WDECP+cu9LJQ9z8+U2E5dsOzZS4rizq/X/+OXkFJQL9W2xBnDOttx+pyYaDQZZxtVmzwrOJ\nFy2CXbsCVx8/KE0QiAN2e5RTnNvOtE9qMfsodVY+/BC+/97eDguDt95yz+3xJTH53DD9Dqofs7mB\nMqNi+fKGj8kPq9gLxvtDg2on6NIw1VWeu7UF2XlBNts6MREaOZPeGQNffAEnTgS0Sr6kQ0RVUDp0\nqPAykX//O1xQ9r7LUun549O03PqNqzx90AccqdncPycPAX2aJlMlIhuwK5At3NE0wDUqwuGwY5Ar\nOfss0tNtIKigaSVKMzooFfCcFx/v3FZ0n0Yl7FOicR65gRMTE0lMTDzbQ6gKYvRoOHjQ3m7UyCaJ\n84dm2+fTe5F7lMviHqPZ1KZo66cqi+jwXK5svo0ZG9sCsDw1jnZ19tM49niAa+YhNtYGgo8+suWd\nO+G77wovWh8ASUlJJCUlefWYpQkCK4AWIpIApAFDgKFF9pkF/A34VES6AUeNMZ6LeIrz54w8g4AK\nXd9+C++95y6//rp/RurVOLyNwV/c4mqz3tEkkYV9/u37E4egC+ruY82+emw7UhMQZm1qw8gLfwme\nuQMAzZtD7962XwBg2TLaFyxYHyBFvxyP91xa7xyV2BxkjMkD7gPmAeuAqcaYDSIyUkT+7NxnDrBD\nRLYCbwF/LXi8iHwMLAFaicguEfFT154qj44cgbvucpdvvNGO3PO1qKzj3PrJdVQ+dRiw8wG+vPET\n8h06lcYXROC6VpuIDLMd74dOVSYpuUlgK1Wcnj2hVStXceCSJfD77wGskPeV6h1ujJkLtC6y7a0i\n5ftO89hbz7l2KuSMGuWeE1CnDrzxhu/PKfl53PjlrdQ5uAGA3LAopt4ynRNV6/v+5CEsNjqLq5pt\n4+st9qNlaUoj2tY5QHxMeoBr5kHEDht95x04fJjI3Fy7VvHPP0PDhoGunVdox7AKGtOn2xFBBd58\nE+rW9f15L1/wOK22zHaVZw54j9T4i31/YkXnBmk0rW5HkBuEGRvbkp0XZB9L0dEwZAhERdlyaqq9\nPD15MrD18pIge7VVqNq3D0aOdJdvvx1u8MNiXV1+eZNLl7gTES3uMZo1F9zm+xMrwN0sFOHIA2yz\n0NytLQNcq2LUqQM33UR+QdraX3+FYcMqxIghDQIq4PLy7If+AZueh7g4u4awr7XZOIP+c/7mKm9q\ndR0L+jzl+xOrQmpUyqR/yy2u8u97G7B2f5BNIgNo3pw5Xbu6y19+accxB9W057OnQUAF3FNPwfz5\n9rYITJwINWr49pyNdv3EjV8OdS0Qk9qwC1/e+DHGEWQTl0JEh3p7C6078PXm1hw5FR3AGhXvlzZt\n4IEH3BteeaVwXpNySIOACqiFC8FzZPDjj8OVV/r2nHUOrOfWT64jItcuUXaoZgs+vnU22ZEVP2Nk\nsBKBa1pupkb0KQCy8sL5ckM78vL9uGpQab38sh22VuCxx+DddwNXnzLSIKACZu9euPVW99X0ZZcV\nDgi+UOvQZoZNvpxKmbYzMqNyHT66bS4ZVfzQA63OKCo8jxvbrschzquz9Bi+Ccb+gbAwu8Rd797u\nbSNH2uahckiDgAqIrCy7PvA+ZwtA3brwyScQ7sNh+TUOb+POD/pQ7cReW4fIqky5bY6mhAgicTHp\nhVJO/5rWkF/2BOFQzKgou+ZAJ2fW/Px8O4Jo+vTA1uscaBBQfmcM3H03/PSTLYvY2fkNGvjunLFH\nd3Ln5D7EpNtJCNkRlZly6xz2NOziu5Oqc9I9PoXzPPoHvtnagn0n2gWwRqcREwPffAMtnVcrublw\n883lLhBoEFB+9/TT7pQsYBeM92U/QM1DWxgxqRfVj9mUwDnh0Xwy9Ct2JfQs4ZEqEERgQKtNNKhq\nJ43lGwdJOx9h59HYANesGPXq2bQS5TgQaBBQfvX55/DEE+7yPffYDKG+En9oF3+aeKkrAOSGRTL1\nlhnsaNrHdydVZRYRls8t7ddS2ZltNDO3Old/dAcHMioHuGbFiIv7YyC46SaYNCmg1SotDQLKb+bP\nt/MBCvTpA//9r/3m5wtRv/7Kw3OeoWrGfgBywisxdchMtrXQhe7Kg9joLG5ut44wZ0fxpkO16f/x\nbaRnBTaJW7EKAkHB4vR5eXYFpBdeCGy9SkGDgPKLxYth4EDItl/saNXKXhVE+GqdlmnTaDBsGFWy\n7dT+zKhYPrxjHltb9PXRCZUvJFQ/xg1tNwA2EPyyJ44bPruFrNwgnM8RFwc//FB44YtHHrETyoJ4\nZrEGAeVzv/wC11zjTrUSH2/TRdes6YOTGQPPPAM33ogj084DyKhch0nDk9jV+FIfnFD5Wrs6B+gW\n/6arPH97c4Z+OTj4ViQDO7rh+++hVy/3tpdftjlQ0oMoMZ4HDQLKp379Fa6+Go471wupVw8WLIAm\nTXxwsqwsGD7czjhz2hdTj/f/tJi99Tv64ITKX1rXmseTvRe6ytM3tuX6T2/hVE4QpvquXt1+yxk0\nyL1t5ky45BK7eH2Q0SCgfGbhQrtc62Gbop8aNeziTB7p2b1n+3bo0QMmT3ZtOnXxxTx93RgO1fLF\nCZW//avnD/yj+xJXec6WVlz7ya2cyA7CPoLoaNve6TnqYe1auOgid46UIKFBQPnEtGnQr597fe4a\nNeyXo/PP98HJZsyAzp3tZUeBP/2JtEmTyIjWVBAVhQi8cOU8nuj5vWvbwh3NuPqj2zl4MghHDYWH\n26ag99+HghXJDh2Cq66Cf/0LcnICWz8nDQLKq4yB116zI+QKOoHj4uDHH+2XIK/KyLDJvK6/Ho4d\ns9siImxSr3ffdf/jqQpDBJ7ss4in+7i/TS/Z3Ziu79zD8ewgTDEBdpRQUpJtCwX7T/L00zZPSnJy\nIGsGaBBQXnTqFNx5p/1cLhgM0aqVnRncvr2XT7Z4MXTsaCNOgYQEu33UKN+NO1VB4bGei5nQ9xv3\netBHa/BjyufM3hykgaB7d1i5svCsyKVL7UiiN98M6OghDQLKK3butE3yniuDde1qP5MTErx4ouPH\nbTtrr16wdat7+4AB8Ntv9qQqJDxw8XJmDJlKFeeEslxTles+uZWxixLJzQ/Cj7b69WHuXHj2WZuE\nDuyIob/8xU6a2bLlzI/3kSB8pVR5Yoz94O/QofD623/6kx0pV8dba4MYY3NNtG5tm3sKUo/GxNgF\nCGbM8NGYUxXMBrTexJK73qNx7FHALlH5fz8k0mviCHYcqR7YyhXH4YDRo+23o9Yey7Z//73tMHvi\nCXdHmr+q5NezqQpl/36bVn3YsMJN8v/7n22Sj/bWmiDLlkHPnnDHHTb/dIGrr4Z16+ywUG3+CVkX\n1NvHinveoXb0Mte2pSmN6PjWvbz/eyfyTRC+N7p1s81Djz3mvirIyrIrLLVqBR984LcmIg0C6qzl\n5cE779h2fs88Wc2a2S80997rpc/klSvh2mtte2pBylGwE3I++shmcIyP98KJVHlXt0oGlzQcxlN9\nFrjSTBzPiuauWQO5bNJw1uwLwvUioqNtB/GKFdDFI5ttWpr9YtOhA3zxhc+DgQYBdVZ++MG+X//8\nZzh40L39L3+BVavs53WZGGMvlQcNsrnaZ8923xcRYafhb9oEt92m3/5VISL5PN7zR5bc9R7Naxx2\nbV+8K4FOb93LQ99eHZxDSTt1guXLbcI5z3zqa9faYXYdO8LUqT4bUqpBQJXKsmU29cNll9kv6AUa\nN7bj/994A6qWZUh+VhZ8+qmNIj172hmWBUTsEmTr1tn1XKtVK8OJVEXXNS6VVff+j9E9FhPuyAMg\nzzj4z7LuNJ0wijGLenM0M8jWL3Y47NC6zZttv0CVKu771qyBoUPtpfYzz9i5Bt48tVePpiqU/Hw7\nw/eKK+xn85w57vsqVYLx42HDBjv35ZytX28TbMXH25WZli8vfP+gQfYSY8oUd6pepUpQJTKHZ6+Y\nz6p73+SyhGTX9hPZUTz5w2U0eeVBHvnuyuBbo6BqVXjySTt/4NFHCweDlBSbEiU+vvBi92WkQUD9\nwcGD8NJL0KaN/YBfsMB9n4j9UrJxI4wZA5XP5ep60yb4979tm2f79nZWpWfbUmQk3HWX/eY/fbqP\nphmrUNCuzgEW3TmJL276lHZ19ru2H8uK5oUlPWj26ihu/Oxm5m5tEVzDSmvXtt/6k5Nh7Fi7/mqB\nzEyvjiAKwuxLKhCOHLEtMJ99ZlObFG1+DAuzLTKPPQZt257lwTMzbWfCN9/Yn02bit+vcWM7tnTk\nSDumWikvEIEb221gUJuNfLruPMYlJbLlcC3Arlo2bUM7pm1oR90qJxjSfi1DzltL17hUwhwmwDXH\nBoNx4+w/3qefwoQJdj7MqFFeO4UGgRCVl2fH9X/3HcybZwffFNfvFBtrh4COGgXNS7se+8GDdsTD\njz/an59/dueQKCo6Gq67zn7zv+IK93A5pbwszGG49fw13NJ+Ld9sbcmE5Rczf7v7Tb0/oyqv/tyN\nV3/uRu3KGfRtsZX+LbZwWZOdNKwW4DTQUVH2H/GOO2D1ansV7SWlCgIi0hd4Bdt89J4x5rli9nkV\n6AdkAMONMStL+1jlW8bY5sTVq22T+9Kl9veZ0pt362ZHAN1yyxmafHJz7azddevsz6pVNonbzp1n\nrlB0tM0ud/PNtrdZO3qVH4U5DNe22sy1rTazbn8d3v+9E5+sPZ+0E+734cGTVfhodQc+Wm0/bJvV\nOMyljXdxIHoNS5bYz2DP5nq/EfFqAIBSBAERcQCvA5cDe4AVIjLTGLPRY59+QHNjTEsRuRh4E+hW\nmseqP0pKSiIxMfGsHpOfDwcOwO7dthlx61b7s2mTHWl29GjJx+jSxX4u33STM99/RgbsSYM9e+yB\nd++GXbts2uatW+2HfW5u6SrYujX07Ws//Hv1sj3LJTiX16GiSj66kibVdU0EgKTkZBK9tCBF+7oH\neOnqeTx/5XcsSm7K1LXnMXtLS/aeKPzFZPuRmmw/UhPoyDc97Gdx8+b2bd26tZ3f1aSJbdFMSDjH\nvrIAKc2VQFdgizFmJ4CITAUGAp4f5AOByQDGmOUiEisi9YCmpXiswn6WZmban5kzk6hfP5GMDNv/\nk55uf44ds7n5jxyxv/fvtz/79tn5JadrcTmd+tVOcEXCVq6qv4orYlbQ4OQ2+PwQvHnIHvhcO5+i\nouy3lW7d7HDPnj3dGRTPggYBNw0Cbt4MAgXCHIYrmm3nimbbyTfC72n1mb2lFYuSm7AsJZ7M3MLr\noBrj/qLlOZWlQPXqtlurfn3bp1urlv2pWdM2scbE2N9Vq9qAUaWK/V2pkr1Qjoqyo0b9oTRBIA7Y\n7VFOwQaGkvaJK+VjXa6r93MpqmMZSj9RyJymf6foMYzHNs/HGCMYV75Cu09+wTYj5Bf8dt7ONw7y\nEfKMgzzjIDc/jDzs71zjINeEkW0iyMm3v7Pzw8kr8qd45ZVSP71Sqc4RzmcNHVlJd5bSnaUkpO9E\n1gJry3DguDg7wqfg58IL7W+fLR6slG85xHBhwzQubJjGmMu+JzsvjF/3NGDJ7kZ8sK0aedGXsHHj\nmSfyHj1qfzaW4etueLgNBpGR9t8pIsJui4iwTbUPP3zuxy50Hu8c5g/OaSrn1/s1A2RZ1OAw8aTQ\nmF20YKvrpz3riCfl7P8oUVHurzONGrl/mjWz18LNmgX1da/D4SA3dy+7d3/s9WNnZh5CwsLYnJFB\n+KlTXj9+ek4O69PT/fd1UJ1WZFge3Rul0L1RCukNkxk36RJOnbLzujZtsj/bttnW0Z07batpaVtJ\nzyQ31/5kZPzxvsOH/7jtXIk53dfkgh1EugHjjDF9neVHAePZwSsibwKLjDGfOssbgcuwzUFnfKzH\nMYJgPJZSSpUvxpQtQ15prgRWAC1EJAFIA4YAQ4vsMwv4G/CpM2gcNcbsE5GDpXgsUPYnopRS6uyV\nGASMMXkich8wD/cwzw0iMtLebd42xswRkf4ishU7RHTEmR7rs2ejlFLqrJTYHKSUUqriCnivk4j0\nFZGNIrJZREYHuj7+JCLxIrJQRNaJyBoRecC5vYaIzBORTSLyrYgEWZYr3xERh4j8JiKznOWQfC2c\nw6w/F5ENzvfHxSH8WvxdRNaKyGoRmSIikaHyWojIeyKyT0RWe2w77XMXkcdEZIvzfVOq1I4BDQIe\nk8muBtoDQ0WkTSDr5Ge5wEPGmPZAd+Bvzuf/KDDfGNMaWAg8FsA6+tsoYL1HOVRfiwnAHGNMW6AD\ndm5NyL0WItIQuB/obIy5ANuEPZTQeS0mYj8fPRX73EWkHXAz0BabveENkZIX3Qj0lYBrIpoxJgco\nmEwWEowxewvSaxhjTgAbgHjsa/CBc7cPgEGBqaF/iUg80B9412NzyL0WIhID9DTGTAQwxuQaY44R\ngq+FUxhQRUTCgUpAKiHyWhhjFgNHimw+3XMfAEx1vl+SgS2cYV5WgUAHgdNNMgs5ItIE6AgsA+oZ\nY/aBDRRAEK6N5xP/AR4GPDuqQvG1aAocFJGJzqaxt0WkMiH4Whhj9gAvAbuwH/7HjDHzCcHXwkPd\n0zz3op+nqZTi8zTQQUABIlIV+AIY5bwiKNpbX+F770XkGmCf88roTJewFf61wDZ5dAb+a4zpjB1x\n9yih+b6ojv3mmwA0xF4R3EYIvhZnUKbnHuggkAo09ijHO7eFDOcl7hfAh8aYgjUV9zlzLyEi9YH9\np3t8BdIDGCAi24FPgD4i8iGwNwRfixRgtzHmF2f5S2xQCMX3xRXAdmPMYWNMHjAduITQfC0KnO65\npwKNPPYr1edpoIOAayKaiERiJ5PNCnCd/O19YL0xZoLHtlnAcOftO4GZRR9U0RhjHjfGNDbGNMO+\nDxYaY+4AviL0Xot9wG4RaeXcdDmwjhB8X2CbgbqJSLSzk/Ny7MCBUHothMJXx6d77rOAIc7RU02B\nFkCJCdkCPk/Aud7ABNyTyZ4NaIX8SER6AD8AayjIXwePY/9wn2Gj+k7gZmNMKZJBVwwichnwD2PM\nABGpSQi+FiLSAdtBHgFsx07ADCM0X4ux2C8GOcDvwN1ANULgtRCRj4FEoBawDxgLzAA+p5jnLiKP\nAXdhX6tRxph5JZ4j0EFAKaVU4AS6OUgppVQAaRBQSqkQpkFAKaVCmAYBpZQKYRoElFIqhGkQUEqp\nEKZBQCmlQpgGAaWUCmH/H7yXj0E2Lg/SAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10bcd77b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = arange( 0, 100, 1 )\n",
    "\n",
    "# Plot the histograms of each class' actual scores\n",
    "hist( bio508_scores, 5, normed = True, color = \"red\", alpha = 0.5 )\n",
    "hist( bio200_scores, 5, normed = True, color = \"blue\", alpha = 0.5 )\n",
    "\n",
    "# Plot the normal distributions that might underly each class, based on their mean and stdev\n",
    "bio508_mean, bio508_std = mean( bio508_scores ), std( bio508_scores )\n",
    "plot( x, scipy.stats.norm( bio508_mean, bio508_std ).pdf( x ), color = \"red\", lw = 3 )\n",
    "bio200_mean, bio200_std = mean( bio200_scores ), std( bio200_scores )\n",
    "plot( x, scipy.stats.norm( bio200_mean, bio200_std ).pdf( x ), color = \"blue\", lw = 3 )\n",
    "\n",
    "legend( [\"BIO508\", \"BIO200\"], loc = \"upper left\" )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Depending on the data that's been generated here, those two normal distributions may or may not look very similar by eye.  But we can put numbers on that - what's the probability we'd get two samples of these sizes so different by chance, if the normal distributions were actually supposed to be the same?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2.7059998122789688, 0.010140879855558357]\n"
     ]
    }
   ],
   "source": [
    "# \"ttest_ind\" stands for \"t-test, independent\" since these are unpaired samples\n",
    "t_statistic, p_value = scipy.stats.ttest_ind( bio508_scores, bio200_scores )\n",
    "print( [t_statistic, p_value] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PolyCollection at 0x105ca5e10>"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEACAYAAAC9Gb03AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUVPWZ//H30w0IAoIiQWVzIQEkshhFFsF2C6gJOGNO\nApkso9GQGU1yfs78YmacjI1jzsRxTiYaszkuJyb+gnESt8QYUKwIyirgqAFBCYSlZV8EBbq7nt8f\n32ooO73c7q6qW3Xr8zqnD7Xcb92nuumnv/V8l2vujoiIJFdF3AGIiEh+KdGLiCScEr2ISMIp0YuI\nJJwSvYhIwinRi4gkXKREb2ZTzWyNma01s1taOO58M6s1s79ua1sREckPa20evZlVAGuBS4GtwDJg\nhruvaeK4ecD7wIPu/uuobUVEJH+i9OjHAuvcfaO71wJzgOlNHPdV4H+A7e1oKyIieRIl0fcHNmXd\n35x57CgzOw242t1/BFhb2oqISH7lajD2e4Dq7yIiRahThGO2AIOy7g/IPJbtPGCOmRlwMnCFmdVF\nbAuAmWnTHRGRNnJ3i3JQi19AJfAWMBjoAqwChrdw/EPAX7e1bQglmW677ba4Q8irJL6/+nr3J590\nv/xy95Ejb/Pf/969tjbuqPIjiT+/bEl+f5m82Woeb7V04+71wE3AXOANYI67rzazWWb25aaatNa2\n1b8+IjGaPx/OPhtuvx2++EU47TT4l3+BwYPh1Vfjjk6k7aKUbnD3Z4GhjR77STPHXtdaW5Fi9eab\nMGMGPPggXHUVmMG6dVBdDXPmwCc/CYsWQX9NKZASEinRS8dUVVXFHUJeJeX97d8PV18N3/42fOIT\nxx5veH8zZsCGDeG5F1+Enj1jCTPnkvLza07S318UrS6YKhQz82KJRcpPOg3XXAP9+sGPf9z8ce4w\naxa88w48+WTo8YvExcwiDcZqrxsR4OGHYcsWuPvulo8zgx/8IJRz5s0rTGwiHaUevZS9ujoYNgwe\neAAuuiham8ceg//4D1i6VL16iY969CIR/fznMHBg9CQPocxTVwdPPJG/uERyRT16KWsNvfn774e2\njtk98wx84xthymVlZV7CE2mRevQiETzyCAwY0PYkD3DFFdCrF/ziFzkPSySn1KOXslVfD0OHtq83\n32DePPjHf4RVq1Srl8JTj16kFfPmQe/e7U/yAJdeGubfr1yZs7BEck6JXsrWww/D3/5tx16jogKu\nvRYeeignIYnkhUo3Upb27Qt717z9NvTp07HX2rgRPvYx2LwZunbNTXwiUah0I9KCxx6DSy7peJKH\n8AdjzJiwUlakGCnRS1n66U/DzpS5ovKNFDOVbqTsvP02jB8fSi1duuTmNd9/P0zTXLUqLL4SKQSV\nbkSa8bOfwcyZuUvyAN26wac/HVbZihQbJXopK+4hGX/+87l/7Wuu0ZYIUpyU6KWs/PGPcORImCWT\naxddFHa13Lo1968t0hFK9FJWnn4apk3LzyrWzp1h6lT4zW9y/9oiHaFEL2Xl6afD5QDzZfp0TbOU\n4hNp1o2ZTQW+R/jD8IC739no+WnAvwFpoBb4P+7+Uua5DcC+hufcfWwz59CsG8mrHTtgyBDYvh2O\nOy4/59i/P8y+2boVevTIzzlEGuRs1o2ZVQD3AlOAEcBMMxvW6LDn3H2Uu48BvgTcn/VcGqhy9zHN\nJXmRQvjtb+Gyy/KX5AFOOCFM3fz97/N3DpG2ilK6GQusc/eN7l4LzAGmZx/g7u9l3e1BSO4NLOJ5\nRPIq32WbBtOmqXwjxSVKAu4PbMq6vznz2AeY2dVmthp4Grgu6ykH5pnZMjO7oSPBirTXoUPw3HNw\n1VX5P9e0aeGiJHV1+T+XSBQ562m7+xPuPhy4Grgj66mJ7n4ucCVwo5ldmKtzikSVSsFHPwp9++b/\nXAMHhv1vXnop/+cSiaJThGO2AIOy7g/IPNYkd19oZmea2UnuvtvdazKP7zCzxwmloIVNta2urj56\nu6qqiqqObBQukuU3vylM2abBlCnhE0RbrkMr0ppUKkUqlWpzu1Zn3ZhZJfAmcClQAywFZrr76qxj\nznL3tzO3zwWedPeBZnY8UOHuB8ysOzAXmO3uc5s4j2bdSN4MHQpz5oRdJgth/ny49VZYtKgw55Py\nFHXWTas9enevN7ObCEm6YXrlajObFZ72+4BrzOwLwBHgfeDTmeb9gMfNzDPneqSpJC+ST5s3w65d\nMGpU4c45YQK8/nrY975Xr8KdV6Qp2r1SEu+nPw2lm8ceK+x5P/5xuOmmMDgrkg/avVIk4/nnw7Vd\nC+2yy0KdXiRuSvSSaO5K9CJK9JJob74JlZVh64NCGz0atm2DLc3OURMpDCV6SbSG3nw+dqtsTUVF\nuC7t888X/twi2ZToJdHmzw/JNi4q30gx0KwbSaz6evjQh+C11+C00+KJ4e23YdKkUL6J41OFJJtm\n3UjZW7UqJPq4kjzAmWeGC5KsXRtfDCJK9JJYcZdtIPTiJ02CBQvijUPKmxK9JNbChTB5ctxRKNFL\n/JToJZHS6ZDoLyyCvVKV6CVuSvSSSGvWhD1m+v/FlRMKb/jwcIlBzaeXuCjRSyItWBB60sXALHyy\nUK9e4qJEL4m0YEFxlG0aTJ6sRC/xUaKXRFq4sHh69KA6vcRLiV4SZ9MmOHgwXGykWIwZAxs2wO7d\ncUci5UiJXhKnYbZNMa1E7dQJLrhA15GVeCjRS+IU00BsNpVvJC5K9JI4xTJ/vjEleomLNjWTRNmz\nBwYNCrXwzp3jjuaDDh4Me+/s2gVdu8YdjSRBTjc1M7OpZrbGzNaa2S1NPD/NzF41s5VmttTMJkZt\nK5JLixbB+ecXX5IH6N4dhg2DFSvijkTKTauJ3swqgHuBKcAIYKaZDWt02HPuPsrdxwBfAu5vQ1uR\nnFm8GMaPjzuK5o0fH/4YiRRSlB79WGCdu29091pgDjA9+wB3fy/rbg8gHbWtSC4tWqREL9JYlETf\nH9iUdX9z5rEPMLOrzWw18DRwXVvaiuRCfT0sXRqmMRarhkSv4SgppE65eiF3fwJ4wswuBO4ALm/r\na1RXVx+9XVVVRVVVVa7CkzKwenUY7OzbN+5ImnfGGVBXFxZ1DRoUdzRSalKpFKlUqs3toiT6LUD2\nf8kBmcea5O4LzexMMzuprW2zE71IWxV7fR7CIq6GXr0SvbRV4w7w7NmzI7WLUrpZBgwxs8Fm1gWY\nATyVfYCZnZV1+1ygi7vvjtJWJFcWLYJx4+KOonWq00uhtZro3b0euAmYC7wBzHH31WY2y8y+nDns\nGjN73cxWAN8HPt1S2zy8D5GiH4htoEQvhaYFU5IIe/fCwIFhwVSnnI085cd774VxBC2cko7K6YIp\nkWK3ZAl87GPFn+QBjj8+XHXqlVfijkTKhRK9JEIpDMRmU/lGCkmJXhKhVAZiG4wbF/44iRSCEr2U\nvHQ6lG5KKdFfcEFY3CVSCEr0UvLWrYPevaFfv7gjie6ss8JuljU1cUci5UCJXkrekiXFve1BU8xg\n7NgQu0i+KdFLyVu6NCTNUqPyjRSKEr2UvFLs0UOIWT16KQQtmJKSdugQnHQS7NwZ5qeXkp07Q61+\nzx6oUJdL2kELpqQsrFoFQ4eWXpIHOPnksEJ2zZq4I5GkU6KXklbs+8+3ZuxY1ekl/5TopaQtWVKa\nA7ENVKeXQlCil5JW6j16JXopBA3GSsnatStcsWnPHqisjDua9jl0CPr0CQOz3brFHY2UGg3GSuIt\nWwbnnVe6SR7CNsVnnw0rVsQdiSSZEr2UrFKvzzfQClnJNyV6KVmluiK2sbFjw6cTkXxRopeS5K5E\nLxKVEr2UpA0boHNnGDAg7kg6buhQ2LEjDC6L5EOkRG9mU81sjZmtNbNbmnj+s2b2auZroZmNzHpu\nQ+bxlWampSGSE8uWJaM3D2H7g499TL16yZ9WE72ZVQD3AlOAEcBMMxvW6LD1wGR3HwXcAdyX9Vwa\nqHL3Me6ekF9NidvSpXD++XFHkTsq30g+RenRjwXWuftGd68F5gDTsw9w98Xuvi9zdzHQP+tpi3ge\nkciSUp9voK0QJJ+iJOD+wKas+5v5YCJv7Hrgd1n3HZhnZsvM7Ia2hyjyQXV1sHJlmEOfFOefHxK9\n1gxKPnTK5YuZ2cXAtcCFWQ9PdPcaM+tLSPir3X1hU+2rq6uP3q6qqqKqqiqX4UlCrF4Np54KJ54Y\ndyS5M2BAqNVv2gSDBsUdjRSrVCpFKpVqc7tWt0Aws3FAtbtPzdz/JuDufmej40YCvwKmuvvbzbzW\nbcC77v7dJp7TFggSyYMPwvz58POfxx1Jbk2bBl/4AnzqU3FHIqUil1sgLAOGmNlgM+sCzACeanSy\nQYQk//nsJG9mx5tZj8zt7sDHgdejvw2Rv5S0gdgGGpCVfGk10bt7PXATMBd4A5jj7qvNbJaZfTlz\n2LeAk4AfNppG2Q9YaGYrCYO0T7v73Jy/CykrSRuIbaABWckX7V4pJeX998Nuj7t2JW+3x9274fTT\nS3s3Tiks7V4pibRqFQwblrwkD+Hat/366dKCkntK9FJSkrQitimq00s+KNFLSUnqQGwD1eklH5To\npaSU+qUDW6NEL/mgwVgpGbt3w+DBsHdvcgcrGwabd+8OV58SaYkGYyVxli8PuzwmNclDGGQeNiwM\nOovkihK9lIykzp9vTOUbyTUleikZSvQi7aNELyUhSZcObI0SveSaEr2UhE2ZjbIHDow3jkIYPhxq\nasIKWZFcUKKXktCwUMpanV9Q+iorw6Dz8uVxRyJJoUQvJaFcyjYNGi5EIpILSvRSEsot0atOL7mk\nBVNS9Orrw9WkNmwIG3+Vg40bwwrgmpryKFdJ+2jBlCTGmjVwyinlk+QhXE7Q/dggtEhHKNFL0Vuy\nJNkbmTXFLPTolyyJOxJJAiV6KXpLlsC4cXFHUXgXXACLF8cdhSSBEr0UvSVLkr1jZXPUo5dc0WCs\nFLUDB8JVl3bvhuOOizuawtq3D/r3DwunOneOOxopRjkdjDWzqWa2xszWmtktTTz/WTN7NfO10MxG\nRm0r0pJXXoFzzim/JA/Qq1fYlvm11+KOREpdq4nezCqAe4EpwAhgppkNa3TYemCyu48C7gDua0Nb\nkWaVa32+gco3kgtRevRjgXXuvtHda4E5wPTsA9x9sbvvy9xdDPSP2lakJeVan2+gRC+5ECXR9wey\nZ/Nu5lgib8r1wO/a2VbkAxYvVqJXopeO6pTLFzOzi4FrgQvb0766uvro7aqqKqqqqnISl5SmzZuh\nthbOOCPuSOLz0Y+GRVN790Lv3nFHI3FLpVKkUqk2t4uS6LcAg7LuD8g89gGZAdj7gKnuvqctbRtk\nJ3qRhrJNOW8B0KlT2Mly2TK4/PK4o5G4Ne4Az549O1K7KKWbZcAQMxtsZl2AGcBT2QeY2SDgV8Dn\n3f3ttrQVaU65l20aqHwjHdVqonf3euAmYC7wBjDH3Veb2Swz+3LmsG8BJwE/NLOVZra0pbZ5eB+S\nQOU+46aBVshKR2nBlBSlurpQk96yJcwnL2dbtsDo0bB9e3mXseQvafdKKWmvvRZ2cCz3JA9hdWy3\nbvDWW3FHIqVKiV6K0ssvw4QJcUdRPCZMCN8TkfZQopeitGiREn228ePD90SkPZTopSi9/HJIbhKM\nH68evbSfBmOl6LzzDpx9NuzcCRXqigBw5Ei4wtbWrXDCCXFHI8VCg7FSshYtCj1YJfljunSBMWN0\nwXBpH/0qSdHRQGzTVL6R9lKil6Kj+nzTJkzQgKy0j2r0UlQOHw616G3boEePuKMpLtu2wbBhsGuX\nyloSqEYvJWnlShg6VEm+Kf36wYknwpo1cUcipUaJXoqK6vMtU/lG2kOJXoqK6vMtmzABXnop7iik\n1CjRS9FwPza1Upp24YWwcGHcUUipUaKXovGnP4VkX85XlGrNiBGwY0cYmBWJSoleisaCBTBpkrbi\nbUllZSjfqFcvbaFEL0VjwQKYPDnuKIrfpEnheyUSlRK9FI0XXwxJTFo2aZJ69NI2WjAlReGdd2D4\ncC0GiuLwYejTB2pqoGfPuKOROGnBlJSUhQth4kQl+SiOOw7OPVfXkZXoIv1amdlUM1tjZmvN7JYm\nnh9qZi+b2SEzu7nRcxvM7NXsi4aLNNYwECvRqE4vbdFqojezCuBeYAowAphpZsMaHbYL+CpwVxMv\nkQaq3H2Mu4/tYLySUEr0bXPhhUr0El2UHv1YYJ27b3T3WmAOMD37AHff6e6vAHVNtLeI55EytW8f\nrF0L550XdySlY8IEWL48XJBEpDVREnB/YFPW/c2Zx6JyYJ6ZLTOzG9oSnJSHl18OSb5Ll7gjKR29\nesGQIbBiRdyRSCnoVIBzTHT3GjPrS0j4q929yclh1dXVR29XVVVRVVVVgPAkbirbtM/kyfCHP8C4\ncXFHIoWSSqVIpVJtbtfq9EozGwdUu/vUzP1vAu7udzZx7G3Au+7+3WZeq9nnNb2yfF14Ifzrv8LH\nPx53JKXliSfgxz+GZ5+NOxKJSy6nVy4DhpjZYDPrAswAnmrp3FlBHG9mPTK3uwMfB16PcE4pEwcP\nwqpVYWqltM1FF4WyV21t3JFIsWu1dOPu9WZ2EzCX8IfhAXdfbWazwtN+n5n1A5YDPYG0mX0dOBvo\nCzxuZp451yPuPjdfb0ZKz8KFYU549+5xR1J6Tjwx1OmXLdMe/tKySDV6d38WGNrosZ9k3d4GDGyi\n6QFgdEcClGR74QW4+OK4oyhdF18M8+cr0UvLNO1RYjV/PlxySdxRlK6LLw5/LEVaor1uJDZ798LA\ngbBzZ1jWL223bx8MGKDvYbnSXjdS9F58MUwNVIJqv1694Oyzte+NtEyJXmLzwgsq2+SCyjfSGiV6\nic38+RqIzYWGAVmR5qhGL7HYsSNMDdy1CzoVYn12gh04AKecAtu3w/HHxx2NFJJq9FLUUqmw7YGS\nfMf16AFjxmg3S2meEr3E4vnnVZ/PpSlT4Pe/jzsKKVZK9FJw7mF/lilT4o4kOZTopSVK9FJwa9dC\nfX2YFii5ce65sG0bbNrU+rFSfpTopeCefRamTgVrdQhJoqqshMsug7naSUqaoEQvBdeQ6CW3pkxR\nopemaXqlFNT778OHPhRKDL17xx1NsmzZAiNHhmmWlZVxRyOFoOmVUpQWLIBRo5Tk86F/fzjttHAt\nWZFsSvRSUCrb5Jdm30hTlOiloJTo80uJXpqiGr0UzMaNcN55YRpghboYeXHoEPTrB+vXQ58+cUcj\n+aYavRSd3/0uXABcST5/unaFSy+FZ56JOxIpJvqVk4J58kmYPj3uKJJv2rTwvRZpECnRm9lUM1tj\nZmvN7JYmnh9qZi+b2SEzu7ktbaU87N8PL72k+nwhXHUVPPdcKOOIQIREb2YVwL3AFGAEMNPMhjU6\nbBfwVeCudrSVMvDsszBxIpxwQtyRJF/fvnDOOboYiRwTpUc/Fljn7hvdvRaYA3zgA7i773T3V4C6\ntraV8qCyTWFNmwZPPRV3FFIsoiT6/kD2VkmbM49F0ZG2khC1tWEgdtq0uCMpHw2JXhPZBKCoLvtQ\nXV199HZVVRVVVVWxxSK584c/wIc/HFZtSmEMHRouSPLKK2FKqyRDKpUilUq1uV2URL8FGJR1f0Dm\nsSja1DY70UtyPPkkXH113FGUn+nTQ69eiT45GneAZ8+eHaldlNLNMmCImQ02sy7ADKCl6l/25P22\ntpWEcVd9Pi7Tp8Ovfx13FFIMWu3Ru3u9md0EzCX8YXjA3Veb2azwtN9nZv2A5UBPIG1mXwfOdvcD\nTbXN27uRorN8eVjEM3x43JGUn/HjYe9eeOMNGDEi7mgkTtoCQfLqH/4BuneH22+PO5LypO9/skXd\nAkGJXvImnYZBg8LFMHTZwHgsXQqf+xy8+aau6JVE2utGYrdgQdhYS0k+PuefD3V1sGpV3JFInJTo\nJW/mzIGZM+OOoryZwWc+A48+GnckEieVbiQvamvDvPmlS+GMM+KOpry9+mqY3rp+vco3SaPSjcTq\nuedgyBAl+WIwciQcd1z4oyvlSYle8uIXv4AZM+KOQiD04mfMgEceiTsSiYtKN5JzBw7AwIHwxz/C\nqafGHY0A/OlPMHYsbN4ceveSDCrdSGx++UuYNElJvpiccQaMGgVPPBF3JBIHJXrJufvvh+uvjzsK\naey66+CBB+KOQuKg0o3k1BtvwOWXw5//DJ2Kam9Uef99GDAAVqyAwYPjjkZyQaUbicX998O11yrJ\nF6Nu3cK6hoceijsSKTT16CVnDh8OPcYlS+DMM+OORpqycuWxOfWVlXFHIx2lHr0U3OOPhwE/Jfni\nNWZM2JZi3ry4I5FCUqKXnPnJT+CGG+KOQlpz441wzz1xRyGFpNKN5MSKFeFCF+vXQ+fOcUcjLTl0\nKAzGplK6TkCpU+lGCuq734WvfU1JvhR07Qpf+QrcfXfckUihqEcvHbZ5c9hPZf166N077mgkim3b\nYNgweOutULOX0qQevRTMPffAF7+oJF9K+vULs29+8pO4I5FCiNSjN7OpwPc4dt3XO5s45h7gCuAg\ncK27r8w8vgHYB6SBWncf28w51KMvQe++G5bXL18Op58edzTSFq++CldeGfbB6dIl7mikPXLWozez\nCuBeYAowAphpZsMaHXMFcJa7fxiYBfwo6+k0UOXuY5pL8lK6/vu/4dJLleRL0ahRcM458OCDcUci\n+RaldDMWWOfuG929FpgDTG90zHTgYQB3XwL0MrN+mecs4nmkxBw8CHfdBbfeGnck0l633w7f/naY\niSPJFSUB9wc2Zd3fnHmspWO2ZB3jwDwzW2ZmmmWdIPfcA1VVYSBWStPYsTB6dPhkJslViB1JJrp7\njZn1JST81e6+sADnlTzauzdMqXzppbgjkY66/Xa46qqw42i3bnFHI/kQJdFvAQZl3R+QeazxMQOb\nOsbdazL/7jCzxwmloCYTfXV19dHbVVVVVFVVRQhP4vCf/wnTpsFHPhJ3JNJRY8bAuHHwox/BzTfH\nHY20JJVKkUql2tyu1Vk3ZlYJvAlcCtQAS4GZ7r4665grgRvd/SozGwd8z93HmdnxQIW7HzCz7sBc\nYLa7z23iPJp1UyK2bw8rKrXdbXK89lrYXnrNGk2TLSVRZ920ZXrl3RybXvkdM5sFuLvflznmXmAq\nx6ZXrjCzM4DHCXX6TsAj7v6dZs6hRF8irr8eevaE//qvuCORXPrKV8I0S+2DUzpymugLQYm+NCxa\nBJ/6FKxeDSecEHc0kku7doVPas89pwH2UqGVsZJzdXXw938f6vNK8snTp08YmL3xRlCfK1mU6CWy\nH/4QTjwRZsyIOxLJlxtugPfeg0ceiTsSySWVbiSSTZvg3HPhxRe1tW3SLVkStpxeuRJOPTXuaKQl\nKt1IztTXw+c/H6beKckn3wUXwJe/DNddpxJOUijRS6vuvBPM4BvfiDsSKZRvfSsMzv7wh3FHIrmg\n0o20aMmSsDDqlVfChb+lfKxdCxMnqlxXzFS6kQ7buRNmzgy9OiX58vORj4RPc3/1V2HLCyld6tFL\nkw4fDtsPT5oE//7vcUcjcfr618O6iWeegU6F2B1LItOCKWk3d/jc5+DIEXj0UajQ576yVlcHn/gE\nfPjD8P3vxx2NZFPpRtrtW98K1xJ9+GEleQm9+Ecfheefh+80uYGJFDt9EJOj3EOSf+KJ8EutLWul\nQa9eYWuEhg1lv/nNWMORNlKiFyAk+Vtugblz4YUXoG/fuCOSYnPaaZBKhWTvDv/0T3FHJFGpRi8c\nPgw33RS2HZ43D046Ke6IpJht3QqXXQaXXBJ2MO3cOe6Iypdq9BJJTQ1cfHGYSplKKclL6047Lexi\nun49TJ0Ku3fHHZG0Rom+jM2fH64ZesUV8KtfhT3mRaLo1Quefjrsf3TeeWFRlRQvlW7K0P79YTuD\nZ56B++4LvTKR9nrqKfi7v4PPfAa+/W0N4heSSjfyF+rr4Wc/g3POgXQ6XD5OSV46ato0+N//DbX7\n4cPr+MEPdpJOxx2VZFOPvgyk0/Cb34Spk927h2XtkybFHZUk0YwZD/Loo6M4+eS+fP/7J/CZz/TG\nWu1vSnupRy/s3w/33gtnnnmEL33pz3TqdBcvvaQkL/kzcOA64BH27Hmez352K336vMMdd+zg4MG4\nIytvkRK9mU01szVmttbMbmnmmHvMbJ2ZrTKz0W1pK7nz3nvw+ONwzTV1nHLKEaqrF1BT89fs3j2b\nLVvuU+9KCqAv9fXX4t6d/fuf5LbbltG79wEmTfozjz12mEOH4o6v/LSa6M2sArgXmAKMAGaa2bBG\nx1wBnOXuHwZmAT+O2rYcpFKpvL32oUNhqttdd8HkyYfp06eWG254g6eeupnKys+ya9efOHLkMdLp\n/C1lzOf7KwZ6f+01mPr6WaTTF1FX9yTLlv2aGTOW07Pn+4wcuZXbb3+XpUvDnkr5lPSfXxRRVsaO\nBda5+0YAM5sDTAfWZB0zHXgYwN2XmFkvM+sHnBGhbeKlUimqGtaOt1NtLWzYAG+/HXYSXLWqjuXL\nD/PWW13o3n0rR468xOHDv+W44yrZtWsycCsHDvTLRfitysX7K2Z6fx3VHfgbDh8GqCGd/gXr1u1g\n9uy+3HHHRNLp0xk8eB+jR1cwYUIvRo7szFlnwcCBuVmMlfSfXxRREn1/YFPW/c2E5N/aMf0jti0b\n7qH38v77ocRy8CAcOBC+9u0Le37v2QPbtzs1NXVs3nyETZvSbNtWyd69XejWbR+VlX/myJHXOXx4\nKV27vktlZSf27DkXGA98mro67WohxexU4LpM+aaWdHol8BBbt+5gyxZ4/PEP0bnzaMxOp7a2D716\nvcfJJ9fSvz8MGNCZAQO60r9/F3r3ht69w3z+nj2hR48w0aBbt/DVpQsqU2bJV1Yo2m9xbS1ceWUt\nNTU1uDvu0DBoHW4fO7bhuYbHs++n08f+bfxVX19BOl2R+beSI0dq+bd/O0I63QmzeioqDmN2GLOD\nwAHS6XdJp3fhvhvYC+wEdgDbgW106lSPeycOHBgEnAUMAf6G997L7u6kgVdbeOd/zuF3UaQlW4BX\nIh5bCZy3CM6GAAAEE0lEQVSfVbc/wpEja4DfAhvZt+8A+/fDunVdgD7AyUBfoDfQG7OTqKjoiVlP\n3I8HupJOd8W9koqKOior60inD3PXXXuorExTUZGmosKprExj5lRUQEWFH/0y4+i/4evYbWj6XzPn\n5pvruP76szr8ncuXVqdXmtk4oNrdp2bufxNwd78z65gfAy+4+6OZ+2uAiwilmxbbZr2G5laKiLRR\nlOmVUXr0y4AhZjYYqAFmADMbHfMUcCPwaOYPw15332ZmOyO0jRysiIi0XauJ3t3rzewmYC5hls4D\n7r7azGaFp/0+d3/GzK40s7eAg8C1LbXN27sREZG/UDQrY0VEJD+KamWsmX3VzFab2WtmlsiLlpnZ\nP5hZ2swStSGwmf1H5me3ysx+ZWYnxB1TRyV5sZ+ZDTCz+Wb2Rub37Wtxx5QPZlZhZivM7Km4Y8m1\nzDT2xzK/d2+Y2QXNHVs0id7MqoBPAue4+znAf8YbUe6Z2QDgcmBj3LHkwVxghLuPBtYBJX39oTJY\n7FcH3OzuIwhzc29M2Ptr8HXgj3EHkSd3A8+4+3BgFNBsWbxoEj3wd8B33L0OwN13xhxPPvwX8H/j\nDiIf3P05d2/Ys3AxMCDOeHLg6EJBd68FGhb7JYK7v+PuqzK3DxCSRP94o8qtTMfqSuD+uGPJtcwn\n5knu/hCAu9e5+/7mji+mRP8RYLKZLTazF8zsvLgDyiUzmwZscvfX4o6lAK4Dfhd3EB3U3CLAxDGz\n04HRwJJ4I8m5ho5VEgcizwB2mtlDmdLUfWbW7JUACrqM0szmAdnr8o3wQ/iXTCwnuvs4Mzsf+CVw\nZiHj66hW3t8/E8o22c+VlBbe363u/nTmmFuBWnf/fzGEKG1kZj2A/wG+nunZJ4KZXQVsc/dVmbJw\nyf2+taITcC5wo7svN7PvAd8Ebmvu4IJx98ube87MvgL8OnPcssyAZR9331WwADuoufdnZh8FTgde\nNTMjlDVeMbOx7r69gCF2SEs/PwAz+1vCR+VLChJQfm0BBmXdH5B5LDHMrBMhyf/M3Z+MO54cmwhM\nM7MrgW5ATzN72N2/EHNcubKZUCFYnrn/P0CzEwaKqXTzBJkEYWYfATqXUpJvibu/7u6nuPuZ7n4G\n4Yc0ppSSfGvMbCrhY/I0dz8cdzw5cHShoJl1ISz2S9rMjQeBP7r73XEHkmvu/s/uPsjdzyT87OYn\nKMnj7tuATZlcCXApLQw6F9MOWA8BD5rZa8BhIDE/lCY4yfso+X2gCzAvfGhhsbv/fbwhtV/SF/uZ\n2UTgb4DXzGwl4f/kP7v7s/FGJm3wNeARM+sMrCezULUpWjAlIpJwxVS6ERGRPFCiFxFJOCV6EZGE\nU6IXEUk4JXoRkYRTohcRSTglehGRhFOiFxFJuP8P+NayzKhVRdUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10be9d518>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Separate the range from -5 to 5 into values less than and greater than our t-statistic\n",
    "x_below_t = arange( -5, -abs( t_statistic ), 0.1 )\n",
    "x_between_t = arange( -abs( t_statistic ), abs( t_statistic ), 0.1 )\n",
    "x_above_t = arange( abs( t_statistic ), 5, 0.1 )\n",
    "x = concatenate( [x_below_t, x_between_t, x_above_t] )\n",
    "\n",
    "# Calculate the probability density function for a t-distribution over this range\n",
    "# Note that the total degrees of freedom is the size of both samples minus two\n",
    "degrees_of_freedom = len( bio508_scores ) + len( bio200_scores ) - 2\n",
    "t_distribution = scipy.stats.t( df = degrees_of_freedom )\n",
    "\n",
    "y = t_distribution.pdf( x )\n",
    "plot( x, y )\n",
    "fill_between( x_below_t, t_distribution.pdf( x_below_t ) )\n",
    "fill_between( x_above_t, t_distribution.pdf( x_above_t ) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.010140879855558293"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Integrate the two shaded areas using the cumulative distribution function\n",
    "left_region = t_distribution.cdf( -abs( t_statistic ) )\n",
    "right_region = 1 - t_distribution.cdf( abs( t_statistic ) )\n",
    "p_value = left_region + right_region\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parametric two-sample comparison exercises\n",
    "\n",
    "**1.** Create two samples that will have a low probability of being drawn from the same population when t-tested."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:82: RuntimeWarning: Degrees of freedom <= 0 for slice\n",
      "  warnings.warn(\"Degrees of freedom <= 0 for slice\", RuntimeWarning)\n",
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:116: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  ret = ret.dtype.type(ret / rcount)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "nan"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# A reminder of what one sample should look like:\n",
    "# sample_one = [1, 2, 3, 4, 5]\n",
    "sample_one = [0]\n",
    "sample_two = [1]\n",
    "\n",
    "t_statistic, p_value = scipy.stats.ttest_ind( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**2.** Create two samples that will have a high p-value when t-tested."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:82: RuntimeWarning: Degrees of freedom <= 0 for slice\n",
      "  warnings.warn(\"Degrees of freedom <= 0 for slice\", RuntimeWarning)\n",
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:116: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  ret = ret.dtype.type(ret / rcount)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "nan"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_one = [0]\n",
    "sample_two = [1]\n",
    "\n",
    "t_statistic, p_value = scipy.stats.ttest_ind( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**3.** Create two samples with very different means that will nevertheless have a high p-value (>0.05) when t-tested."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:82: RuntimeWarning: Degrees of freedom <= 0 for slice\n",
      "  warnings.warn(\"Degrees of freedom <= 0 for slice\", RuntimeWarning)\n",
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:116: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  ret = ret.dtype.type(ret / rcount)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "nan"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_one = [0]\n",
    "sample_two = [1]\n",
    "\n",
    "t_statistic, p_value = scipy.stats.ttest_ind( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**4.** Create two samples with the same *medians* that will have a *low* p-value when t-tested."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:82: RuntimeWarning: Degrees of freedom <= 0 for slice\n",
      "  warnings.warn(\"Degrees of freedom <= 0 for slice\", RuntimeWarning)\n",
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:116: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  ret = ret.dtype.type(ret / rcount)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "nan"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_one = [0]\n",
    "sample_two = [1]\n",
    "\n",
    "t_statistic, p_value = scipy.stats.ttest_ind( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Comparing two samples nonparametrically\n",
    "\n",
    "Determining whether two samples are likely to have come from the same population or not is tremendously useful, so much so that sometimes we'd like to do it even when we can't make the kinds of guarantees required by parametric tests like a t-test.  For example, the eagle-eyed among you may have noticed that when comparing test scores betwen our BIO508 and BIO200 classes, the t-test required us to assume our measurements came from a normal distribution, when in fact we generated our fake data using a uniform distribution.  Our t-test was wrong!  In this case, violating the parametric assumptions of the t-test had relatively little effect, but this is by no means assured to be the case for more complex real-world data (an excellent in-depth guide to when this will or won't matter [is provided here](http://www.basic.northwestern.edu/statguidefiles/ttest_unpaired_ass_viol.html)).\n",
    "\n",
    "Instead, a variety of *nonparametric* tests are available that will assess whether two samples are likely to have been drawn from the same distribution, *without* requiring that distribution to be a normal.  There's no such thing as a free lunch, though, so the cost of nonparametric tests tends to be reduced sensitivity or power.  That is:\n",
    "\n",
    "* *Parametric tests* tend to be very good at detecting true differences between samples when the appropriate assumptions are met.  However, they tend to produce false positives when their assumptions are violated.  That is, if your data aren't well-behaved, many parametric tests (including t-tests) will tell you something's interesting when it isn't!\n",
    "\n",
    "* *Nonparametric tests* tend to be very good at detecting true similarities between samples, even when the data aren't well-behaved.  However, they tend to produce false negatives when samples are truly different, but only slightly so.  That is, if your signal isn't especially strong, many nonparametric tests will tell you nothing's interesting, when it is!\n",
    "\n",
    "We'll discuss this tradeoff between **sensitivity** (the ability to detect true differences) and **specificity** (the ability to ignore false differences) more in the future, but for now, realize that it is a drawback of nonparametric tests.  They're robust to assumption violations, but that robustness comes at a cost."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Rank-based tests\n",
    "\n",
    "All that being said, there exist a family of nonparametric tests that parallel the family of t-tests.  Each uses slightly different test statistic (just like the z- and t-statistics differed for different applications), and each is a **rank-based test**.  That is, like Spearman correlation, they ignore the actual values of measurements and instead test whether the *relative ranks* of measurements differ from each other.\n",
    "\n",
    "To recall what a **rank transformation** is, consider a data sample:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.92, 0.98, 0.66, 0.72, 0.56, 0.6, 0.21, 0.72, 0.61, 0.17]\n"
     ]
    }
   ],
   "source": [
    "# Generate 10 random numbers between 0 and 1, rounded to two decimal places, so they're easy to read\n",
    "sample_one = [round( random.random( ), 2 ) for i in range( 10 )]\n",
    "print( sample_one )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[  9.   10.    6.    7.5   3.    4.    2.    7.5   5.    1. ]\n"
     ]
    }
   ],
   "source": [
    "# Now print their rank transform\n",
    "print( scipy.stats.rankdata( sample_one ) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could generate many different datasets, all of which looked the same after a rank transform.  This is why nonparametric tests are less sensitive - they don't \"see\" as many differences between samples, which lets them ignore false differences but miss some real ones.  With this caveat, there are analogs to each of the t-test family members mentioned above:\n",
    "\n",
    "* A [one-sample Wilcoxon test](http://www.stats.gla.ac.uk/steps/glossary/nonparametric.html#wsrt) when you're comparing a set of measurements to a fixed reference point.\n",
    "\n",
    "* An equivalent [paired Wilcoxon test](http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) when you're comparing a sample to itself at a different point in time (and thus lose some independence).\n",
    "\n",
    "In general, just as these tests substitute ranks for quantitative values, they substitute medians for means (e.g. the single-sample Wilcoxon test compares a sample median to a fixed value, while the single-sample t-test compares a sample mean to a fixed value).  The analog to our general t-test detailed above is the **[Mann-Whitney-Wilcoxon test](http://en.wikipedia.org/wiki/Mann-Whitney-Wilcoxon_test)**.  It is not assumption-free - it still requires that all measurements be independent, and (like Spearman correlation) it has some funny behavior in the case of ties.  But it will operate correctly even for data that would be poorly behaved in a t-test:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[21, 25, 21, 18, 7, 59, 45, 45, 40, 56]\n"
     ]
    }
   ],
   "source": [
    "# Generate bimodal data from two different uniform distributions\n",
    "sample_one = [random.randint( 0, 30 ) for i in range( 5 )] + \\\n",
    "    [random.randint( 40, 60 ) for i in range( 5 )]\n",
    "print( sample_one )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEACAYAAABI5zaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEAhJREFUeJzt3H+sZGV9x/H3B5df1kgQ49KwslgKUkwVbFyWkIax1srS\nAP+YWmtqS9KEGIkkNI2WmOz9p4n9q5GqwU2RiKkVf6SwVGihgQlrQxcEtqywq1AVkbq3GFwMIHbF\nb/+Ys5vr5e7O3NmZuXsf3q9kwjlnnp3n++XM/dyzz8zZVBWSpLYctdIFSJImz3CXpAYZ7pLUIMNd\nkhpkuEtSgwx3SWrQ0HBPcmyS7UkeSrIzyeaDjLs2yWNJdiQ5Z/KlSpJGtWbYgKr6eZJ3VtULSV4F\n/EeS26vqvv1jkmwCTq+qM5KcB1wHbJxe2ZKkQxlpWaaqXug2j2XwC2HxnU+XATd2Y7cDJyRZO6ki\nJUnLM1K4JzkqyUPAHuDOqrp/0ZBTgCcX7D/VHZMkrYBRr9x/WVXnAuuA85KcPd2yJEmHY+ia+0JV\n9dMkdwMXAY8ueOop4I0L9td1x35FEv8hG0kaQ1VlOeNH+bbM65Oc0G0fD7wb2L1o2Fbgg92YjcDe\nqpo/SIHNPjZv3rziNQx7dGdhzMfmMf/c6jjvq+H82dsrs79xjHLl/uvA55McxeCXwU1VdVuSKwY/\ns7Wl2784yePA88DlY1UjSZqIUb4KuRN4+xLHP7to/8oJ1iVJOgzeoTpBvV5vpUuYst5KFzBVLZ+/\nlnuD9vsbR8ZdzxlrsqRmOZ9eLgkvv01h6rOOvW4oafBzW5P+QFWStPoY7pLUIMNdkhpkuEtSgwx3\nSWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJek\nBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUoKHhnmRdkruSPJJkZ5KP\nLDHmwiR7kzzYPT4+nXIlSaNYM8KYXwBXV9WOJK8BHkhyR1XtXjTunqq6dPIlSpKWa+iVe1Xtqaod\n3fZzwC7glCWGZsK1SZLGtKw19ySnAecA25d4+vwkO5J8PcnZE6hNkjSmUZZlAOiWZL4KXNVdwS/0\nAHBqVb2QZBNwM3Dm5MqUJC3HSOGeZA2DYP9CVd2y+PmFYV9Vtyf5TJLXVdUzi8fOzc0d2O71evR6\nvTHKlqR29ft9+v3+Yb1Gqmr4oORG4MdVdfVBnl9bVfPd9gbgy1V12hLjapT5ND1JgFmfg+B5l8aX\nhKpa1ueaQ6/ck1wAfADYmeQhBslwDbAeqKraArw3yYeAfcDPgPctt3hJ0uSMdOU+scm8cl9xXrlL\nq884V+7eoSpJDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXI\ncJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3\nSWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1KCh4Z5kXZK7kjySZGeSjxxk3LVJHkuyI8k5ky9VkjSq\nNSOM+QVwdVXtSPIa4IEkd1TV7v0DkmwCTq+qM5KcB1wHbJxOyZKkYYZeuVfVnqra0W0/B+wCTlk0\n7DLgxm7MduCEJGsnXKskaUTLWnNPchpwDrB90VOnAE8u2H+Kl/8CkCTNyCjLMgB0SzJfBa7qruDH\nMjc3d2C71+vR6/XGfSlJK+Dkk09jfv6Jmc65du169uz5/kznXMk++/0+/X7/sF4rVTV8ULIG+Bfg\n9qr65BLPXwfcXVU3dfu7gQuran7RuBplPk1PEmDW5yB43tvxSnkPHUl9JqGqspxXGnVZ5nPAo0sF\ne2cr8MGuiI3A3sXBLkmanaHLMkkuAD4A7EzyEINfZdcA64Gqqi1VdVuSi5M8DjwPXD7NoiVJhzbS\nsszEJnNZZsUdSX/V1Or0SnkPHUl9TnNZRpK0ihjuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGG\nuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhL\nUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDhoZ7kuuTzCd5+CDPX5hkb5IH\nu8fHJ1+mJGk51oww5gbg74EbDzHmnqq6dDIlSZIO19Ar96r6BvCTIcMymXIkSZMwqTX385PsSPL1\nJGdP6DUlSWMaZVlmmAeAU6vqhSSbgJuBMw82eG5u7sB2r9ej1+tNoARJake/36ff7x/Wa6Sqhg9K\n1gO3VtVbRxj7PeB3quqZJZ6rUebT9CQBZn0Ogue9Ha+U99CR1GcSqmpZy9+jLsuEg6yrJ1m7YHsD\ng18YLwt2SdLsDF2WSfJFoAeclOQHwGbgGKCqagvw3iQfAvYBPwPeN71yJUmjGGlZZmKTuSyz4o6k\nv2pqdXqlvIeOpD6nuSwjSVpFDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3\nSWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJek\nBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUFDwz3J9Unmkzx8iDHXJnksyY4k50y2REnSco1y\n5X4D8J6DPZlkE3B6VZ0BXAFcN6HaJEljGhruVfUN4CeHGHIZcGM3djtwQpK1kylPkjSONRN4jVOA\nJxfsP9Udm5/Aa8/M008/zb59+2Y654knnsjxxx8/0zklvTJMItyXZW5u7sB2r9ej1+vNuoSX+da3\nvsW5576DY445cWZzvvTS/7Fx4/n0+7fObM6VcyxJZjrj2rXr2bPn+zOdcyWcfPJpzM8/sdJlaML6\n/T79fv+wXiNVNXxQsh64tareusRz1wF3V9VN3f5u4MKqetmVe5IaZb5Z27ZtG5dccg3PPrtthrPe\ny1lnXc2uXffOcE66kJ31OViZOY/E99qkeT6nOOMK/b9dqs8kVNWyrpBG/SpkusdStgIf7ArYCOxd\nKtglSbMzdFkmyReBHnBSkh8Am4FjgKqqLVV1W5KLkzwOPA9cPs2CJUnDDQ33qvqTEcZcOZlyJEmT\n4B2qktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ\n4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnu\nktQgw12SGmS4S1KDDHdJatBI4Z7koiS7k3wnyUeXeP7CJHuTPNg9Pj75UiVJo1ozbECSo4BPAe8C\n/ge4P8ktVbV70dB7qurSKdQoSVqmUa7cNwCPVdUTVbUP+BJw2RLjMtHKJEljGyXcTwGeXLD/w+7Y\nYucn2ZHk60nOnkh1kqSxDF2WGdEDwKlV9UKSTcDNwJkTem1J0jKNEu5PAacu2F/XHTugqp5bsH17\nks8keV1VPbP4xebm5g5s93o9er3eMkuWpLb1+336/f5hvUaq6tADklcB32bwgeqPgPuA91fVrgVj\n1lbVfLe9AfhyVZ22xGvVsPlWwrZt27jkkmt49tltM5z1Xs4662p27bp3hnNCEmDW52Bl5jwS32uT\n5vmc4owr9P92qT6TUFXL+lxz6JV7Vb2U5ErgDgZr9NdX1a4kVwyeri3Ae5N8CNgH/Ax433KKkCRN\n1khr7lX1r8CbFx377ILtTwOfnmxpkqRxeYeqJDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkN\nMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDD\nXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGjRSuCe5KMnuJN9J8tGDjLk2\nyWNJdiQ5Z7JlSpKWY2i4JzkK+BTwHuAtwPuTnLVozCbg9Ko6A7gCuG4Kta4C/ZUuYMr6K13AVPX7\n/ZUuYYr6K13AVLV97sYzypX7BuCxqnqiqvYBXwIuWzTmMuBGgKraDpyQZO1EK10V+itdwJT1V7qA\nqWo7IPorXcBUtX3uxjNKuJ8CPLlg/4fdsUONeWqJMZKkGVmz0gUcCY4++mhefPERXvvaSw7rdV58\n8dscd9wDI4196aVnOO64ow9rPkk6mFTVoQckG4G5qrqo2/8YUFX1twvGXAfcXVU3dfu7gQuran7R\nax16MknSkqoqyxk/ypX7/cBvJlkP/Aj4Y+D9i8ZsBT4M3NT9Mti7ONjHKU6SNJ6h4V5VLyW5EriD\nwRr99VW1K8kVg6drS1XdluTiJI8DzwOXT7dsSdKhDF2WkSStPjO7Q3WUG6FWkyTXJ5lP8vCCYycm\nuSPJt5P8W5ITVrLGcSVZl+SuJI8k2ZnkI93xVvo7Nsn2JA91/W3ujjfRHwzuT0nyYJKt3X4zvQEk\n+X6S/+rO4X3dsSZ6THJCkq8k2dX9DJ43Tm8zCfdRboRahW5g0M9CHwP+vareDNwF/PXMq5qMXwBX\nV9VbgPOBD3fnq4n+qurnwDur6lzgHGBTkg000l/nKuDRBfst9QbwS6BXVedW1YbuWCs9fhK4rap+\nC3gbsJtxequqqT+AjcDtC/Y/Bnx0FnNPua/1wMML9ncDa7vtk4HdK13jhPq8Gfj9FvsDXg18E3hH\nK/0B64A7gR6wtTvWRG8LevwecNKiY6u+R+C1wH8vcXzZvc1qWWaUG6Fa8IbqviVUVXuAN6xwPYct\nyWkMrm7/k8Gbq4n+umWLh4A9wJ1VdT/t9Pd3wF8BCz9Qa6W3/Qq4M8n9Sf6iO9ZCj28Cfpzkhm5Z\nbUuSVzNGb/6rkNO1qj+tTvIa4KvAVVX1HC/vZ9X2V1W/rMGyzDpgQ5K30EB/Sf4QmK+qHcChvnq8\n6npb5IKqejtwMYNlw9+lgfPH4BuMbwc+3fX3PIOVjmX3Nqtwfwo4dcH+uu5Ya+b3/5s6SU4G/neF\n6xlbkjUMgv0LVXVLd7iZ/varqp8y+IdXLqKN/i4ALk3yXeCfgN9L8gVgTwO9HVBVP+r++zSDZcMN\ntHH+fgg8WVXf7Pa/xiDsl93brML9wI1QSY5hcCPU1hnNPU3hV6+OtgJ/3m3/GXDL4j+winwOeLSq\nPrngWBP9JXn9/m8bJDkeeDewiwb6q6prqurUqvoNBj9nd1XVnwK3ssp72y/Jq7u/VZLk14A/AHbS\nxvmbB55McmZ36F3AI4zR28y+557kIgafAu+/EeoTM5l4SpJ8kcEHVicB88BmBlcQXwHeCDwB/FFV\n7V2pGseV5ALgHgY/MNU9rgHuA77M6u/vt4HPM3gvHgXcVFV/k+R1NNDffkkuBP6yqi5tqbckbwL+\nmcH7cg3wj1X1iVZ6TPI24B+Ao4HvMrgp9FUsszdvYpKkBvmBqiQ1yHCXpAYZ7pLUIMNdkhpkuEtS\ngwx3SWqQ4S5JDTLcJalB/w8DTWFwVW7nNAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10bcb5ac8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dummy = hist( sample_one )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[48, 48, 48, 64, 62, 62, 82, 81, 81]\n"
     ]
    }
   ],
   "source": [
    "# Generate more weird data\n",
    "sample_two = [random.randint( 40, 50 ) for i in range( 3 )] + \\\n",
    "    [random.randint( 60, 70 ) for i in range( 3 )] + \\\n",
    "    [random.randint( 80, 90 ) for i in range( 3 )]\n",
    "print( sample_two )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEACAYAAABI5zaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD/pJREFUeJzt3X2spGV5x/HvD3Z9QcpWbbqtLKAi1NCkXTUFlBqmVSpL\nU2gaU0QTWoyGGIlWm1ZDTPbsf7VJ24hicCulYnyh2BZoxHZtdGr0D6T7UpEXBa2yrLLV4voC0qJe\n/WOexWH2nJ05Z2f2zLn7/SSTfV7uPM+118785pn7zHM2VYUkqS3HrHYBkqTpM9wlqUGGuyQ1yHCX\npAYZ7pLUIMNdkho0NtyTPDnJbUl2J7kjydYlxl2V5N4ke5Jsnn6pkqRJrRs3oKr+J8lvVNUjSY4F\nPpfkE1X1+YNjkmwBTq2q05KcBVwDnD27siVJhzPRtExVPdItPpnBG8LonU8XAdd3Y28DNiTZOK0i\nJUnLM1G4JzkmyW7gQeCTVXX7yJATgb1D6/u6bZKkVTDplftPquoFwCbgrCRnzLYsSdKRGDvnPqyq\nvpfk08D5wF1Du/YBJw2tb+q2PUESf5GNJK1AVWU54yf5tszPJdnQLT8VOA+4Z2TYLcCl3ZizgQNV\ntX+JAmf22LVrFyecsJnBjwSO5LF14rEbNrycHTt2zPTvtdRj69atq3Leo1Fn92w5yo/ZPj/9N5/9\nY/nPm8lf66v5vFmJSa7cfxH4QJJjGLwZ3FBVtya5fPB3qu3d+gVJ7gMeBi5bUTWSpKmY5KuQdwAv\nXGT7+0bWr5hiXZKkI+AdqovqrXYBE+n1eqtdwkTWSp1rwVrp5Vqpc6281lfCcF9Ub7ULmMhaeQGt\nlTrXgrXSy7VS51p5ra+E4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y\n3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNd\nkhpkuEtSgwx3SWqQ4S5JDTLcJalBY8M9yaYkn0pyZ5I7krxpkTHnJjmQZFf3eMdsypUkTWLdBGN+\nBLy1qvYkOR7YmWRHVd0zMu4zVXXh9EuUJC3X2Cv3qnqwqvZ0yz8A7gZOXGRoplybJGmFljXnnuTZ\nwGbgtkV2vzjJniQfT3LGFGqTJK3QJNMyAHRTMh8D3txdwQ/bCZxcVY8k2QLcBJw+vTIlScsxUbgn\nWccg2D9YVTeP7h8O+6r6RJL3JnlGVT00OnZhYeHx5V6vR6/XW0HZktSufr9Pv98/omNMeuX+N8Bd\nVfWuxXYm2VhV+7vlM4EsFuzwxHCXJB1q9MJ327Ztyz7G2HBPcg7wGuCOJLuBAq4ETgGqqrYDr0zy\nBuAx4IfAxcuuRJI0NWPDvao+Bxw7ZszVwNXTKkqSdGS8Q1WSGmS4S1KDDHdJapDhLkkNMtwlqUGG\nuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhL\nUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUFjwz3J\npiSfSnJnkjuSvGmJcVcluTfJniSbp1+qJGlS6yYY8yPgrVW1J8nxwM4kO6rqnoMDkmwBTq2q05Kc\nBVwDnD2bkiVJ44y9cq+qB6tqT7f8A+Bu4MSRYRcB13djbgM2JNk45VolSRNa1px7kmcDm4HbRnad\nCOwdWt/HoW8AkqSjZJJpGQC6KZmPAW/uruBXZGFh4fHlXq9Hr9db6aEkqUn9fp9+v39Ex5go3JOs\nYxDsH6yqmxcZsg84aWh9U7ftEMPhLkk61OiF77Zt25Z9jEmnZf4GuKuq3rXE/luASwGSnA0cqKr9\ny65GkjQVY6/ck5wDvAa4I8luoIArgVOAqqrtVXVrkguS3Ac8DFw2y6IlSYc3Ntyr6nPAsROMu2Iq\nFUmSjph3qEpSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y\n3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNd\nkhpkuEtSgwx3SWqQ4S5JDTLcJalBY8M9ybVJ9if5whL7z01yIMmu7vGO6ZcpSVqOdROMuQ54N3D9\nYcZ8pqounE5JkqQjNfbKvao+C3xnzLBMpxxJ0jRMa879xUn2JPl4kjOmdExJ0gpNMi0zzk7g5Kp6\nJMkW4Cbg9KUGLywsPL7c6/Xo9XpTKEGS2tHv9+n3+0d0jFTV+EHJKcA/VdWvTDD2P4EXVdVDi+yr\nSc63Urt376bXey3f+97umZ1j1IYN53HjjX/Keeedd9TO+f9BEmB2z5Ulzsosn5+avVafN0moqmVN\nf086LROWmFdPsnFo+UwGbxiHBLsk6egZOy2T5MNAD3hmkvuBrcCTgKqq7cArk7wBeAz4IXDx7MqV\nJE1ibLhX1avH7L8auHpqFUmSjph3qEpSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIa\nZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGG\nuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBY8M9ybVJ9if5wmHGXJXk3iR7kmye\nbomSpOWa5Mr9OuAVS+1MsgU4tapOAy4HrplSbZKkFRob7lX1WeA7hxlyEXB9N/Y2YEOSjdMpT5K0\nEtOYcz8R2Du0vq/bJklaJeuO9gkXFhYeX+71evR6vaNdgiTNtX6/T7/fP6JjTCPc9wEnDa1v6rYt\najjcJUmHGr3w3bZt27KPMem0TLrHYm4BLgVIcjZwoKr2L7sSSdLUjL1yT/JhoAc8M8n9wFbgSUBV\n1faqujXJBUnuAx4GLptlwZKk8caGe1W9eoIxV0ynHEnSNHiHqiQ1yHCXpAYZ7pLUIMNdkhpkuEtS\ngwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXI\ncJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkho0UbgnOT/J\nPUm+nORti+w/N8mBJLu6xzumX6okaVLrxg1IcgzwHuBlwDeA25PcXFX3jAz9TFVdOIMaJUnLNMmV\n+5nAvVX19ap6DPgocNEi4zLVyiRJKzZJuJ8I7B1af6DbNurFSfYk+XiSM6ZSnSRpRcZOy0xoJ3By\nVT2SZAtwE3D6lI4tSVqmScJ9H3Dy0PqmbtvjquoHQ8ufSPLeJM+oqodGD7awsPD4cq/Xo9frLbNk\nSWpbv9+n3+8f0TEmCffbgeclOQX4JvAq4JLhAUk2VtX+bvlMIIsFOzwx3CVJhxq98N22bduyjzE2\n3Kvqx0muAHYwmKO/tqruTnL5YHdtB16Z5A3AY8APgYuXXYkkaWommnOvqn8Gfmlk2/uGlq8Grp5u\naZKklfIOVUlqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGG\nuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhL\nUoMMd0lqkOEuSQ0y3CWpQYa7JDVoonBPcn6Se5J8OcnblhhzVZJ7k+xJsnm6ZUqSlmNsuCc5BngP\n8Argl4FLkjx/ZMwW4NSqOg24HLhmBrUeRf3VLmAi/X5/tUuYyFqpcy1YK71cK3Wuldf6Skxy5X4m\ncG9Vfb2qHgM+Clw0MuYi4HqAqroN2JBk41QrPar6q13ARNbKC2it1LkWrJVerpU618prfSUmCfcT\ngb1D6w902w43Zt8iYyRJR8m61S5gmtavX8+jj36FE074nSM6zqOPfomnPGXnhGN3s379+iM6nyRN\nW6rq8AOSs4GFqjq/W387UFX1zqEx1wCfrqobuvV7gHOrav/IsQ5/MknSoqoqyxk/yZX77cDzkpwC\nfBN4FXDJyJhbgDcCN3RvBgdGg30lxUmSVmZsuFfVj5NcAexgMEd/bVXdneTywe7aXlW3JrkgyX3A\nw8Blsy1bknQ4Y6dlJElrz0zvUE1yTJLdSW7p1rcmeSDJru5x/izPP2GNX0vyH12dn++2PT3JjiRf\nSvIvSTbMaZ3z2M8NSW5McneSO5OcNaf9XKzOuepnktO7f+9d3Z/fTfKmeevnYeqct36+JckXk3wh\nyYeSPGneerlEnU9eSS9neuWe5C3Ai4ATqurCJFuB71fVX87spMuU5KvAi6rqO0Pb3gn8d1X9eXdH\n7tOr6u2rViRL1jmP/fxb4N+q6rok64CnAVcyf/1crM4/Ys76eVB3M+EDwFnAFcxZPw8aqfO1zEk/\nkzwL+Czw/Kr63yQ3ALcCZzBHvTxMnc9mmb2c2ZV7kk3ABcD7R3fN6pwrFA7tw0XAB7rlDwC/e1Qr\nWtxidR7cPheSnAC8tKquA6iqH1XVd5mzfh6mTpijfo54OfCVqtrLnPVzxHCdMF/9PBZ4Wvdm/lQG\n9+PMYy+H6zyOQZ2wzF7Oclrmr4A/AUY/GlyRwe+fef88fARiUN8nk9ye5HXdto0Hv+1TVQ8CP79q\n1f3UcJ2vH9o+T/18DvDtJNd1Hx23JzmO+evnUnXCfPVz2MXAh7vleevnsIuBjwytz0U/q+obwF8A\n9zMIy+9W1b8yZ71cpM4DXZ2wzF7OJNyT/Dawv6r28MR3m/cCz62qzcCDwKp/XAPOqaoXMviU8cYk\nL+XQN6R5+KnzaJ2/zvz1cx3wQuDqrtaHgbczf/0crfMRBnXOWz8BSLIeuBC4sds0b/0EFq1zbvqZ\n5GcZXKWfAjyLwZXxa5izXi5S5/FJXs0KejmrK/dzgAu7eeKPAL+Z5Pqq+lb9dJL/r4Ffm9H5J1ZV\n3+z+/BZwE4PfpbM/3e/GSfILwH+tXoUDI3X+I3DmHPbzAWBvVf17t/73DEJ03vo5WufHgBfMYT8P\n2gLsrKpvd+vz1s+DDtb5LRg8V+eony8HvlpVD1XVjxm8hl7C/PVytM5/AF6ykl7OJNyr6sqqOrmq\nnsvgpqdPVdWlXfMO+j3gi7M4/6SSHJfk+G75acBvAXcwuCnrD7thfwDcvCoFdpao84vz1s/u4+3e\nJKd3m14G3Mmc9XOJOu+at34OuYQnTnXMVT+HPKHOOevn/cDZSZ6SJHT/5sxfLxer8+6V9HLm33NP\nci7wx923Za4HNgM/Ab4GXL7YnaxHS5LnMHgHLwYf1T9UVX+W5BnA3wEnAV8Hfr+qDsxhnXPVT4Ak\nv8rgh+jrga8yuKHtWOaon7Bkne9m/vp5HIOePbeqvt9tm6vnZ1fTYnXO1fOz+3bZq4DHgN3A64Cf\nYf56OVznLuD1wLUss5fexCRJDfK/2ZOkBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1\n6P8Apjo9o59t1c8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c0e0dd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dummy = hist( sample_two )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Mann-Whitney test uses a summary called the u-statistic (hence sometimes called the **Mann-Whitney U-test**), which is a function not of the sample means or standard deviations, but of the two samples' relative ranks.  The u-statistic is a function of how often the global ranks of values from one sample are lower than those from the second sample.  In particular:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[21, 25, 21, 18, 7, 59, 45, 45, 40, 56, 48, 48, 48, 64, 62, 62, 82, 81, 81]"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Merge the values from both samples\n",
    "sample_total = sample_one + sample_two\n",
    "sample_total"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[  3.5   5.    3.5   2.    1.   13.    7.5   7.5   6.   12.   10.   10.\n",
      "  10.   16.   14.5  14.5  19.   17.5  17.5]\n"
     ]
    }
   ],
   "source": [
    "# Rank transform all values jointly; note that ties split the difference\n",
    "ranks_total = scipy.stats.rankdata( sample_total )\n",
    "print( ranks_total )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[  3.5   5.    3.5   2.    1.   13.    7.5   7.5   6.   12. ]\n"
     ]
    }
   ],
   "source": [
    "# Add up the ranks that correspond to the first sample's values\n",
    "size_of_sample_one = len( sample_one )\n",
    "ranks_from_sample_one = ranks_total[:size_of_sample_one]\n",
    "print( ranks_from_sample_one )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "61.0"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum_of_ranks_from_sample_one = sum( ranks_from_sample_one )\n",
    "sum_of_ranks_from_sample_one"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we calculate the u-statistic using the formula:\n",
    "\n",
    "> $$U = R - \\frac{|x|(|x|+1)}{2}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.0"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "u_statistic = sum_of_ranks_from_sample_one - ( len( sample_one ) * ( len( sample_one ) + 1 ) / 2.0 )\n",
    "u_statistic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The distribution of the u-statistic is a bit complicated - it's not normal, it's not a t-distribution, it's nothing pretty (although for large samples it does, unsurprisingly, become approximately normal).  However, the underlying process of converting this statistic into a p-value is otherwise identical: plug it into a function and integrate.  This results in a probability that the two samples' relative ranks would be this dissimilar by chance if they were derived from the same distribution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[6.0, 0.001607182360354916]\n"
     ]
    }
   ],
   "source": [
    "u_statistic, p_value = scipy.stats.mannwhitneyu( sample_one, sample_two )\n",
    "print( [u_statistic, p_value] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that under most circumstances, because these data are pretty strange, a t-test will be *more* confident that they're different - greater sensitivity, lower p-value.  Good sometimes (like now, since we know the two samples really are from different populations), but bad other times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-4.0677111061070939, 0.00080074915934357513]\n"
     ]
    }
   ],
   "source": [
    "t_statistic, p_value = scipy.stats.ttest_ind( sample_one, sample_two )\n",
    "print( [t_statistic, p_value] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Nonparametric two-sample comparison exercises\n",
    "\n",
    "**1.** Create two samples that are surprisingly different (low p-value) both in a Mann-Whitney test and a t-test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# A reminder of what one sample should look like:\n",
    "# sample_one = [1, 2, 3, 4, 5]\n",
    "sample_one = [0]\n",
    "sample_two = [1]\n",
    "\n",
    "u_statistic, p_value = scipy.stats.mannwhitneyu( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:82: RuntimeWarning: Degrees of freedom <= 0 for slice\n",
      "  warnings.warn(\"Degrees of freedom <= 0 for slice\", RuntimeWarning)\n",
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:116: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  ret = ret.dtype.type(ret / rcount)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "nan"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t_statistic, p_value = scipy.stats.ttest_ind( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**2.** Create two samples that are similar (high p-value) both in a Mann-Whitney test and a t-test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_one = [0]\n",
    "sample_two = [1]\n",
    "\n",
    "u_statistic, p_value = scipy.stats.mannwhitneyu( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:82: RuntimeWarning: Degrees of freedom <= 0 for slice\n",
      "  warnings.warn(\"Degrees of freedom <= 0 for slice\", RuntimeWarning)\n",
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:116: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  ret = ret.dtype.type(ret / rcount)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "nan"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t_statistic, p_value = scipy.stats.ttest_ind( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**3.** Create two samples with very different means that have a high probability of coming from the same rank distribution.  That is, even with different means, their rank distributions should overlap to produce a high Mann-Whitney p-value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_one = [0]\n",
    "sample_two = [1]\n",
    "\n",
    "u_statistic, p_value = scipy.stats.mannwhitneyu( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**4.** Create two samples with (nearly) identical means that have a low Mann-Whitney p-value.  Think about what kind of variation \"matters\" to a t-test (e.g. outliers) versus what kind matters to a nonparametric test.  This is very doable, but trickier than it might look at first!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_one = [0]\n",
    "sample_two = [1]\n",
    "\n",
    "u_statistic, p_value = scipy.stats.mannwhitneyu( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:82: RuntimeWarning: Degrees of freedom <= 0 for slice\n",
      "  warnings.warn(\"Degrees of freedom <= 0 for slice\", RuntimeWarning)\n",
      "/Users/chuttenh/anaconda/lib/python3.4/site-packages/numpy/core/_methods.py:116: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  ret = ret.dtype.type(ret / rcount)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "nan"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t_statistic, p_value = scipy.stats.ttest_ind( sample_one, sample_two )\n",
    "p_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Comparing more than two samples\n",
    "\n",
    "Another equivalent way of thinking about what a t-test does is that it \"correlates\" a continuous sample with a binary one.  To see what I mean, consider what a normal correlation would look like.  We take two *paired* samples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sample_one = [0.1, 0.2, 0.3, 0.5, 0.6, 0.65, 0.8, 0.9]\n",
    "sample_two = [0.25, 0.3, 0.29, 0.44, 0.75, 0.9, 0.8, 0.95]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Line them up with each other:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE8FJREFUeJzt3W9sXNd55/HvI0vCMt3IRWrCaOV4po3XYB1ECdRa1WKD\n7ci2aqboVkawQKUAyTar3co1XKQvispZbGEuECDxuyYw1FYt0aIvSqmog0b9h7jparbwrtOwjm0l\nrWjJSYaV7CCdJI2ApCzEqM++mJFEM6RmSN7hkIffDzDA/XN4+eCA/PHy3DlnIjORJJVp27ALkCQN\njiEvSQUz5CWpYIa8JBXMkJekghnyklSwniEfEZMR8bWIOHeLNp+IiIsR8VJEvKvaEiVJq9XPnfzv\nAg8vdzIi3gO8LTP/HXAM+M2KapMkrVHPkM/M54B/ukWTQ8Dvd9v+DXB7RNxZTXmSpLWoYkx+N3Bp\nwf5r3WOSpCHzwaskFWx7Bdd4DXjrgv27use+R0S4UI4krUJmxmq+rt87+ei+lnIG+ABAROwHvpWZ\nX1vuQpnpK5Mnn3xy6DVslJd9YV/YF7d+rUXPO/mI+AOgAfxARPwD8CSws5PXeTIz/zwifjoiXgW+\nA3xwTRVJkirTM+Qz8319tHm8mnIkSVXyweuQNBqNYZewYdgXN9kXN9kX1Yi1jves6JtF5Hp+P0kq\nQUSQA37wKknahAx5SSqYIS9JBTPkJalghrwkFcyQl6RltNttpqenabfbwy5l1Qx5SVrC1NRparUx\nDh58lFptjKmp08MuaVV8n7wkLdJut6nVxpibOwvsAc4xMnKA2dkZRkdH170e3ycvSRVqtVrs3Fmn\nE/AAe9ixo0ar1RpeUatkyEvSIvV6natXW8D1j7Y+x/z8LPV6fXhFrZIhL0mLjI6OMjl5gpGRA+za\ntZeRkQNMTp4YylDNWjkmL0nLaLfbtFot6vX6UAN+LWPyhrwkbXA+eJUkLcmQl6SCGfKSVDBDXpIK\n1lfIR8R4RMxExIWIOL7E+e+PiE9GxMsR8dmIuK/6UiVJK9Uz5CNiG/A08DDwduBIRIwtavY/gBcz\n853AfwE+UXWhkqSV6+dOfh9wMTNnM3MeOAUcWtTmPuB/A2TmK0A9IjbfrAFpkyth1URVq5+Q3w1c\nWrB/uXtsoZeB9wJExD7gbuCuKgqU1J9SVk1UtbZXdJ2PAR+PiM8DXwBeBK4t1XBiYuLGdqPRoNFo\nVFSCtHW1222OHn2MubmzzM11Vk08evQADz30wKacir/VNZtNms1mJdfqOeM1IvYDE5k53t1/AsjM\nfOoWX/MV4B2Z+e1Fx53xKg3A9PQ0Bw8+ypUrL9w4tmvXXj7zmd/i/vvvH2JlqsKgZ7xOA/dERC0i\ndgKHgTOLCrg9InZ0t/878H8WB7ykwSlp1URVq2fIZ+Y14HHgWeDvgFOZeT4ijkXEL3Sb/SjwxYg4\nT+ddOB8aVMGSvldJqyaqWi5QJhVko6yaqGq5CqUkFcxVKCVJSzLkJalghrwkFcyQl6SCGfKSVDBD\nXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+Ql\nqWB9hXxEjEfETERciIjjS5zfFRFnIuKliPhCRPx85ZVKklas52e8RsQ24ALwIPA6MA0czsyZBW0+\nDOzKzA9HxB3AK8CdmfndRdfyM14l9eQHkr/RoD/jdR9wMTNnM3MeOAUcWtQmgTd3t98MfGNxwEtS\nP6amTlOrjXHw4KPUamNMTZ0edkmbWj8hvxu4tGD/cvfYQk8D90XE68DLwIeqKU/SVtJutzl69DHm\n5s5y5coLzM2d5ejRx2i328MubdPaXtF1HgZezMwHIuJtwF9GxJ7M/PbihhMTEze2G40GjUajohIk\nbXatVoudO+vMze3pHtnDjh01Wq3Wlhq2aTabNJvNSq7Vz5j8fmAiM8e7+08AmZlPLWjzp8BHM/P/\ndvf/CjiemX+76FqOyUtaVrvdplYbY27uLLAHOMfIyAFmZ2e2VMgvNugx+WngnoioRcRO4DBwZlGb\nWeChbjF3AvcCX15NQZK2rtHRUSYnTzAycoBdu/YyMnKAyckTWzrg16rnnTx03kIJfJzOH4XJzPxY\nRByjc0d/MiJ+EPg94Ae7X/LRzJxa4jreyUvqyXfXvNFa7uT7CvmqGPKStHKDHq6RJG1ShrwkFcyQ\nl6SCGfKSVDBDXpIKZshLUsEMeWmDaLfbTE9Pu06LKmXISxuAKy9qUJwMJQ2Z67WoFydDSZvY9ZUX\nOwEPC1delNbKkJeGrF6vc/VqCzjXPXKO+flZ6vX68IpSMQx5achceVGD5Ji8tEG48qKW4yqUklQw\nH7xKkpZkyEtSwQx5SSqYIS9JBTPkJalgfYV8RIxHxExEXIiI40uc/5WIeDEiPh8RX4iI70bE91df\nriRpJXq+hTIitgEXgAeB14Fp4HBmzizT/meAX87Mh5Y451soJWmFBv0Wyn3Axcyczcx54BRw6Bbt\njwBTqylGklStfkJ+N3Bpwf7l7rHvEREjwDjwzNpLkySt1faKr/efgOcy81vLNZiYmLix3Wg0aDQa\nFZcgSZtbs9mk2WxWcq1+xuT3AxOZOd7dfwLIzHxqibafBP4wM08tcy3H5CVphQa6dk1E3Aa8QufB\n61eBzwFHMvP8ona3A18G7srMuWWuZchL0gqtJeR7Dtdk5rWIeBx4ls4Y/mRmno+IY53TebLb9BHg\n08sFvCRp/bkKpSRtcK5CKUlakiEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpgh\nL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBTPkJalgfYV8RIxHxExEXIiI\n48u0aUTEixHxxYg4W22ZkqTV6PlB3hGxDbgAPAi8DkwDhzNzZkGb24H/B/xUZr4WEXdk5teXuJYf\n5C1JKzToD/LeB1zMzNnMnAdOAYcWtXkf8ExmvgawVMBLktZfPyG/G7i0YP9y99hC9wJviYizETEd\nEe+vqkBJ0uptr/A6e4EHgO8Dno+I5zPz1cUNJyYmbmw3Gg0ajUZFJUhSGZrNJs1ms5Jr9TMmvx+Y\nyMzx7v4TQGbmUwvaHAf+TWb+r+7+7wB/kZnPLLqWY/KStEKDHpOfBu6JiFpE7AQOA2cWtfkU8O6I\nuC0i3gT8BHB+NQVJkqrTc7gmM69FxOPAs3T+KExm5vmIONY5nSczcyYiPg2cA64BJzPz7wdauSSp\np57DNZV+M4drJGnFBj1cI0napAx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEM\neUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kF6yvkI2I8ImYi4kJE\nHF/i/E9GxLci4vPd1/+svlRJ0kpt79UgIrYBTwMPAq8D0xHxqcycWdT0rzPzZwdQoyRplfq5k98H\nXMzM2cycB04Bh5Zot6pPEpckDU4/Ib8buLRg/3L32GL/PiJeiog/i4j7KqlOkrQmPYdr+vQCcHdm\n/nNEvAf4Y+DepRpOTEzc2G40GjQajYpKkKQyNJtNms1mJdeKzLx1g4j9wERmjnf3nwAyM5+6xdd8\nBfixzPzmouPZ6/tJkt4oIsjMVQ2J9zNcMw3cExG1iNgJHAbOLCrgzgXb++j88fgmkqSh6jlck5nX\nIuJx4Fk6fxQmM/N8RBzrnM6TwH+OiF8E5oE54OcGWbQkqT89h2sq/WYO10jSig16uEaStEkZ8pJU\nMENekgpmyEtSwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfIS0G63mZ6ept1uD7sUqVKGvLa8qanT\n1GpjHDz4KLXaGFNTp4ddklQZ167RltZut6nVxpibOwvsAc4xMnKA2dkZRkdHh12eBLh2jbRqrVaL\nnTvrdAIeYA87dtRotVrDK0qqkCGvLa1er3P1ags41z1yjvn5Wer1+vCKkipkyGtLGx0dZXLyBCMj\nB9i1ay8jIweYnDzhUI2K4Zi8RGdsvtVqUa/XDXhtOGsZkzfkJWmD88GrJGlJhrwkFayvkI+I8YiY\niYgLEXH8Fu3uj4j5iHhvdSVKklarZ8hHxDbgaeBh4O3AkYgYW6bdx4BPV12kJGl1+rmT3wdczMzZ\nzJwHTgGHlmj3S8AfAf9YYX2SpDXoJ+R3A5cW7F/uHrshIn4IeCQzfwNY1RNgSVL1tld0nV8HFo7V\nLxv0ExMTN7YbjQaNRqOiEiSpDM1mk2azWcm1er5PPiL2AxOZOd7dfwLIzHxqQZsvX98E7gC+A/xC\nZp5ZdC3fJy9JKzTQyVARcRvwCvAg8FXgc8CRzDy/TPvfBf4kMz+5xDlDXpJWaC0h33O4JjOvRcTj\nwLN0xvAnM/N8RBzrnM6Ti79kNYVIkqrnsgaStMG5rIEkaUmGvCQVzJCXpIIZ8pJUMENekgpmyEtS\nwQx5SSqYIS9JBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBWs\nr5CPiPGImImICxFxfInzPxsRL0fEixHxuYj4D9WXKklaqZ6f8RoR24ALwIPA68A0cDgzZxa0eVNm\n/nN3+x3AH2bmjy5xLT/jVZJWaNCf8boPuJiZs5k5D5wCDi1scD3gu/4t8K+rKWaraLfbTE9P0263\nh12KpML1E/K7gUsL9i93j71BRDwSEeeBPwH+azXllWdq6jS12hgHDz5KrTbG1NTpYZckqWCVPXjN\nzD/uDtE8AnykquuWpN1uc/ToY8zNneXKlReYmzvL0aOPeUcvaWC299HmNeDuBft3dY8tKTOfi4gf\niYi3ZOY3F5+fmJi4sd1oNGg0Gn0Xu9m1Wi127qwzN7ene2QPO3bUaLVajI6Orns97XabVqtFvV4f\nyveXtLRms0mz2azkWv08eL0NeIXOg9evAp8DjmTm+QVt3paZX+pu7wU+lZlvXeJaW/rBa7vdplYb\nY27uLLAHOMfIyAFmZ2fWPWSnpk5z9Ohj7NxZ5+rVFpOTJzhy5OfWtQZJ/VnLg9eeId/9BuPAx+kM\n70xm5sci4hiQmXkyIn4V+ABwFZgDfiUzn1/iOls65OFmuO7YUWN+fnYo4bqR/thI6m3gIV8VQ75j\n2MMk09PTHDz4KFeuvHDj2K5de/nMZ36L+++/f93rkXRrawn5fsbkVbHR0dGh3jHX650hGjjH9Tv5\n+flZ6vX60GqSNBgua7AFjY6OMjl5gpGRA+zatZeRkQNMTp5wqEYqkMM1W9iwh40k9ccxeUkq2KCX\nNZAkbVKGvCQVzJCXpIJtuZB3BUhJW8mWCnlXgJS01WyZd9c4lV/SZuW7a/pwfQXITsDDwhUgJalU\nWybk3ziVH5zKL2kr2DIh71R+SVvRlhmTv86p/JI2G5c1kKSC+eBVkrQkQ16SCmbIS1LBDHlJKlhf\nIR8R4xExExEXIuL4EuffFxEvd1/PRcQ7qi9VkrRSPUM+IrYBTwMPA28HjkTE2KJmXwb+Y2a+E/gI\n8NtVF1qaZrM57BI2DPviJvviJvuiGv3cye8DLmbmbGbOA6eAQwsbZOZnM/NKd/ezwO5qyyyPP8A3\n2Rc32Rc32RfV6CfkdwOXFuxf5tYh/t+Av1hLUZKkamyv8mIRcQD4IPDuKq8rSVqdnjNeI2I/MJGZ\n4939J4DMzKcWtdsDPAOMZ+aXlrmW010laRVWO+O1nzv5aeCeiKgBXwUOA0cWNoiIu+kE/PuXC/i1\nFClJWp2eIZ+Z1yLiceBZOmP4k5l5PiKOdU7nSeDXgLcAJyIigPnM3DfIwiVJva3rAmWSpPU1kBmv\nvSZPddt8IiIuRsRLEfGuQdSxETiR7KZ+fi667e6PiPmIeO961ree+vwdaUTEixHxxYg4u941rpc+\nfkd2RcSZblZ8ISJ+fghlDlxETEbE1yLi3C3arDw3M7PSF50/HK8CNWAH8BIwtqjNe4A/627/BPDZ\nquvYCK8++2I/cHt3e3wr98WCdn8F/Cnw3mHXPcSfi9uBvwN2d/fvGHbdQ+yLDwMfvd4PwDeA7cOu\nfQB98W7gXcC5Zc6vKjcHcSffc/JUd//3ATLzb4DbI+LOAdQybE4ku6mfnwuAXwL+CPjH9SxunfXT\nF+8DnsnM1wAy8+vrXON66acvEnhzd/vNwDcy87vrWOO6yMzngH+6RZNV5eYgQr6fyVOL27y2RJsS\nOJHspp59ERE/BDySmb8BlPxOrH5+Lu4F3hIRZyNiOiLev27Vra9++uJp4L6IeB14GfjQOtW20awq\nNyudDKXVcyIZAL8OLByTLTnoe9kO7AUeAL4PeD4ins/MV4db1lA8DLyYmQ9ExNuAv4yIPZn57WEX\nthkMIuRfA+5esH9X99jiNm/t0aYE/fTF9YlkJ+lMJLvVv2ubWT998ePAqe7bcO8A3hMR85l5Zp1q\nXC/99MVl4OuZ+S/Av0TEXwPvpDN+XZJ++uKDwEcBMvNLEfEVYAz423WpcONYVW4OYrjmxuSpiNhJ\nZ/LU4l/SM8AH4MaM2m9l5tcGUMuw9eyLfieSFaBnX2Tmj3RfP0xnXP6xAgMe+vsd+RTw7oi4LSLe\nROdB2/l1rnM99NMXs8BDAN0x6HvprHxbomD5/2BXlZuV38lnH5OnMvPPI+KnI+JV4Dt0/lIXp5++\nYItMJOuzL97wJete5Drp83dkJiI+DZwDrgEnM/Pvh1j2QPT5c/ER4PcWvLXwVzPzm0MqeWAi4g+A\nBvADEfEPwJPATtaYm06GkqSC+fF/klQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIL9\nf0CiVzxROxJsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x106016a58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dummy = scatter( sample_one, sample_two )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And measure how much they \"agree\" with each other:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.93760987875256918, 0.0005790830442965703]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_value, p_value = scipy.stats.pearsonr( sample_one, sample_two )\n",
    "[r_value, p_value]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A t-test does the exact same thing, except that one of the two samples is binary (that is, categorical with exactly two values).  Take a look; we replace one sample:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Note that the fact that 1 and 2 are ordered numbers doesn't matter -\n",
    "# we're just using them as convenient symbols meaning \"control\" and \"case\"\n",
    "sample_two = [1, 1, 1, 1, 2, 2, 2, 2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Line them up with each other (different visualization, same process):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXEAAAEACAYAAABF+UbAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD5ZJREFUeJzt3W9sXXd9x/H3t6SprjZS1cMpLB33SpTKGlpA1ag6DW3X\nLVkNTzLxZG0npFWZlqp042HKo/jBJMqzwaqOFXlMnYQTCSatG0wwsVxNDChe1yYM6iYduiYNrD6M\nUqnIqBn67oFvwq3jP9fJsa9/1++XdCOf45/P+SZyPjr6nd8538hMJEllum7YBUiSrp4hLkkFM8Ql\nqWCGuCQVzBCXpIIZ4pJUsA1DPCJmIuLliDizzphPRcS5iHguIt5Tb4mSpLUMciX+WeCetb4ZER8A\n3pGZ7wSOAp+uqTZJ0gY2DPHM/BrwyjpDDgNP9sY+DdwYETfXU54kaT11zIkfAM73bV/o7ZMkbTFv\nbEpSwfbUcIwLwK/1bd/S23eFiPBFLZJ0FTIzVts/aIhH77Oap4CPACcj4k7gJ5n58jqFDHhKbWR6\neprp6elhlyFdwd/NekWsFb8DhHhEfA5oA78SEd8HjgN7gczMJzLzSxHxwYh4Efgp8EAtVUuSNrRh\niGfm/QOMebieciRJm+GNzYK12+1hlyCtyt/N7RPbOUcdEemcuCRtTkSseWPTK3FJKpghLkkFM8Ql\nqWCGuCQVzBCXpIIZ4pJUMENckgpmiEtSwQxxSSqYIS5JBTPEJalghrgkFcwQl6SCGeKSalVVFXNz\nc1RVNexSdgVDXFJtZmdP0mxOcOjQgzSbE8zOnhx2SSPP94lLqkVVVTSbEywtnQIOAmdoNCZZWJhn\nfHx82OUVbb33iRvikq7aeg1812MObI5NISRticy8/FlcXKTRGANOAwmcptEYY3Fx8Q3jDPB6GeKS\najE+Ps7MzOM0GpMANBqTzMw87lTKFnM6RVKtqqpi//5xFhcrA7wmzolL2lYR4H/1+jgnLmlbHT8+\n7Ap2D6/EJWmH80pckkaUIS5JBTPEJalghrgkFcwQl1S76elhV7B7uDpFUu1cJ14vV6dI0ogyxCWp\nYIa4JBXMEJekghnikjY0NrZ8s3LQD2xufMTyObR5A4V4RExFxHxEnI2IY6t8f19EPBURz0XEtyPi\nj2qvVNLQvPLK8mqTrfy88sqw/5Zl2nCJYURcB5wF7gZ+AMwB92bmfN+YjwH7MvNjEfEW4AXg5sz8\nvxXHcomhVKDNLBmsqoput0ur1drU+8Rdlri2a11ieAdwLjMXMvMicAI4vGJMAm/uff1m4H9XBrik\n0We3++03SIgfAM73bb/U29fvMeDXI+IHLDfY+2g95UkqRVVVHDnyEEtLp3j11WdYWjrFkSMPUVXV\nsEsbaXtqOs49wLOZeVdEvAP4l4g4mJmvrRw43fc8brvdpt1u11SCpGHqdrvs3dtiaelgb89Brr++\nSbfbtU3bJnU6HTqdzkBjB5kTvxOYzsyp3vYjQGbmJ/rG/BPw8cz89972V4FjmfkfK47lnLhUoEHm\nq6uqotmcYGnpFHAQOEOjMcnCwvxAIe6c+NqudU58Drg1IpoRsRe4F3hqxZgF4P29k90M3AZ87+pL\nllSa/m73+/bdbrf7bTLQC7AiYgr4JMuhP5OZj0bEUZavyJ+IiLcBfwu8rfcjH8/M2VWO45W4VCBX\npwyX3e4lXZPtCFhDfG2+xVCSRpQhLkkFM8QlqWCGuCQVzBCXpIIZ4pJUMENckgpmiEtSwQxxSSqY\nIS5JBTPEJalghrgkFayuphCSRlgSsOrrl+o8xy/+1OAMcUkbCnJ73mK4tacYSU6nSFLBDHFJKpgh\nLkkFM8QlqWCGuCQVzBCXpIIZ4pJUMENckgpmiEtSwQxxSSqYIS5JBTPEJalghrikWlVVxdzcHFVV\nDbuUXcEQl1Sb2dmTNJsTHDr0IM3mBLOzJ4dd0siL3Or3S/afLCK383yS6hHBhq+iraqKZnOCpaVT\nwEHgDI3GJAsL84yPj9dyjt0qIsjMVd/o7pW4pFp0u1327m2xHOAAB7n++ibdbnd4Re0ChrikWrRa\nLV5/vQuc6e05w8WLC7RareEVtQsY4pJqMT4+zszM4zQak+zbdzuNxiQzM48PNJWiq+ecuKQNxRb3\n1wS46Sb48Y+3/jwlWm9O3B6bkja02Wsvb1JuH6dTJKlgA4V4RExFxHxEnI2IY2uMaUfEsxHxXxFx\nqt4yJUmr2XBOPCKuA84CdwM/AOaAezNzvm/MjcDXgd/LzAsR8ZbM/NEqx3JOXNoFnE6p17WuE78D\nOJeZC5l5ETgBHF4x5n7gC5l5AWC1AJe0O1x63N7H7rfHICF+ADjft/1Sb1+/24CxiDgVEXMR8eG6\nCpRUjkuP3d9ww1/72P02qWt1yh7gduAu4JeAb0TENzLzxZqOL2mHq6qKI0ce6nvs/rc4cmSS97//\nLteKb6FBQvwC8Pa+7Vt6+/q9BPwoM38G/Cwi/g14N3BFiE9PT1/+ut1u0263N1expB0jVl1A/u7L\nXy0twf79+68Y4b2x9XU6HTqdzkBjB7mx+SbgBZZvbP4Q+BZwX2Y+3zdmAvhLYAq4AXga+IPM/O6K\nY3ljUxpR1/oCLK3tmm5sZubPgYeBrwDfAU5k5vMRcTQi/qQ3Zh74MssvTfgm8MTKAJc02nzsfjh8\n7F5Sraqqotvt0mq1DPCarHclbohL0g7n+8QlaUQZ4pJUMENckgpmiEtSwQxxSSqYIS5JBTPEJalg\nhrgkFcwQl6SCGeKSVDBDXJIKZohLUsEMcUkqmCEuSQUzxAtVVRVzc3N2FJd2OUO8QJc6ih869KAd\nxaVdzqYQhbGPobT72NmncKt3FF+f/87S6LCzT+Ey8/JncXGRRmMMOA0kcJpGY4zFxcU3jJO0Oxji\nhbGjuKR+TqcUyo7i0u7hdMoIGh8f54tffK8BLu1yXokXLAL855RGn1fikjSiDHFJKpghLkkFM8Ql\nqWCGeMGOHx92BZKGzdUpkrTDuTpFkkaUIS5JBTPEJalghrgkFcwQL9j09LArkDRsrk4pmO9OkXaH\na16dEhFTETEfEWcj4tg6494bERcj4kNXW6wkaXAbhnhEXAc8BtwDvAu4LyIm1hj3KPDluouUJK1u\nkCvxO4BzmbmQmReBE8DhVcb9KfB5YLHG+iRJ6xgkxA8A5/u2X+rtuywifhX4/cz8K2DzXX0lSVel\nrtUpfwH0z5Ub5NvAd6dI2jPAmAvA2/u2b+nt6/ebwImICOAtwAci4mJmPrXyYNN96+La7TbtdnuT\nJesSlxhKo6nT6dDpdAYau+ESw4h4E/ACcDfwQ+BbwH2Z+fwa4z8L/GNm/v0q33OJoSRt0npLDDe8\nEs/Mn0fEw8BXWJ5+mcnM5yPi6PK384mVP3LNFUuSBuLDPpK0w/kqWkkaUYZ4wbyxKcnplIL57hRp\nd3A6RZJGlCEuSQUzxCWpYIa4JBXMEN8hxsaWb1Ru5gObGz82Nty/o6T6uTplh9iOlSauZpHK5OoU\nSRpRhrgkFcwQl6SCGeKSVDBDXJIKZohLUsEM8UJVVcXc3BxVVQ27FElDZIgXaHb2JM3mBIcOPUiz\nOcHs7MlhlyRpSHzYZ4cY9EGcqqpoNidYWjoFHATO0GhMsrAwz/j4eC3nkLSz+LDPCOl2u+zd22I5\nwAEOcv31Tbrd7vCKkjQ0hnhhWq0Wr7/eBc709pzh4sUFWq3W8IqSNDSGeGHGx8eZmXmcRmOSfftu\np9GYZGbm8Q2nUiSNJufEd4jNzldXVUW326XVag0c4M6JS2Vab07cEN8hfIuhpLV4Y1OSRpQhLkkF\nM8QlqWCGuCQVzBCXpIIZ4pJUsD3DLkDLkoBVFxDVeY5f/ClpNBjiO0SQ27NOfGtPIWmbOZ0iSQUz\nxCWpYIa4JBXMEJekghniklSwgUI8IqYiYj4izkbEsVW+f39EnO59vhYRv1F/qZKklTYM8Yi4DngM\nuAd4F3BfREysGPY94Hcy893AnwOfqbtQvZHd7iXBYFfidwDnMnMhMy8CJ4DD/QMy85uZ+Wpv85vA\ngXrLVD+73Uu6ZJAQPwCc79t+ifVD+o+Bf76WorS2qqo4cuQhlpZO8eqrz7C0dIojRx7yilzapWp9\nYjMiJoEHgPetNWZ6evry1+12m3a7XWcJI+9St/ulpSu73dtnUxoNnU6HTqcz0NgN27NFxJ3AdGZO\n9bYfATIzP7Fi3EHgC8BUZv73GseyPdsaBm2dVlUVzeYES0ungIPAGRqNSRYW5jcMcduzSWW61vZs\nc8CtEdGMiL3AvcBTK07wdpYD/MNrBbjqYbd7Sf0GapQcEVPAJ1kO/ZnMfDQijrJ8Rf5ERHwG+BCw\nwPK7+C5m5h2rHMcr8TXY7V7SWux2XwC73Utai93uJWlE+T7xHSS2uCnETTdt7fElbT9DfIe4mmkO\np0ckOZ0iSQUzxCWpYIa4JBXMEJekghniBTt+fNgVSBo2H/aRpB3Oh30kaUQZ4pJUMENckgpmiEtS\nwQzxgvU1SZK0S7k6pVBVVbF//ziLi5UNIaQR5+qUEXOp2z1gt3tpl/NKvDDX0mNTUpns7FO4uIoX\njfvvLI0Op1MKl5mXP4uLizQaY8BpIIHTNBpjLC4uvmGcpN3BEC+M3e4l9XM6pVBX0+1eUpmcE5ek\ngjknLkkjyhCXpIIZ4pJUMENckgpmiEtSwQxxSSqYIS5JBTPEJalghrgkFcwQl6SCGeKSVDBDXJIK\nZohLUsEGCvGImIqI+Yg4GxHH1hjzqYg4FxHPRcR76i1TkrSaDUM8Iq4DHgPuAd4F3BcREyvGfAB4\nR2a+EzgKfHoLatUKnU5n2CVIq/J3c/sMciV+B3AuMxcy8yJwAji8Ysxh4EmAzHwauDEibq61Ul3B\n/yjaqfzd3D6DhPgB4Hzf9ku9feuNubDKGElSzbyxKUkF27A9W0TcCUxn5lRv+xEgM/MTfWM+DZzK\nzJO97XngdzPz5RXHsjebJF2Ftdqz7RngZ+eAWyOiCfwQuBe4b8WYp4CPACd7of+TlQG+XhGSpKuz\nYYhn5s8j4mHgKyxPv8xk5vMRcXT52/lEZn4pIj4YES8CPwUe2NqyJUmwzd3uJUn18sZmgSJiJiJe\njogzw65F6hcRt0TEv0bEdyLi2xHxZ8OuadR5JV6giHgf8BrwZGYeHHY90iUR8VbgrZn5XET8MvAM\ncDgz54dc2sjySrxAmfk14JVh1yGtlJn/k5nP9b5+DXgenxnZUoa4pC0RES3gPcDTw61ktBnikmrX\nm0r5PPDR3hW5toghLqlWEbGH5QD/u8z8h2HXM+oM8XJF7yPtNH8DfDczPznsQnYDQ7xAEfE54OvA\nbRHx/Yjw4SrtCBHx28AfAndFxLMR8Z8RMTXsukaZSwwlqWBeiUtSwQxxSSqYIS5JBTPEJalghrgk\nFcwQl6SCGeKSVDBDXJIK9v9nUx6MeaQ1GAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10603eda0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "scatter( sample_two, sample_one )\n",
    "# The first four values come from bin 1, the last four values from bin 2\n",
    "dummy = boxplot( [sample_one[:4], sample_one[4:]] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And measure how much they \"agree\" with each other:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[-4.6368157981485947, 0.00038458758557440367]"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t_statistic, p_value = scipy.stats.ttest_ind( sample_one, sample_two )\n",
    "[t_statistic, p_value]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also phrase this as measuring how much one variable \"affects\" another - a correlation measures how much a continuous change along one axis is expected to affect the second axis, and a t-test measures how much a binary change along one axis is expected to affect the second.\n",
    "\n",
    "This should beg the question of how one might measure the association between *more* than two categories and a continous outcome.  We introduced the t-test above to compare \"case\" and \"control\" conditions - what if we'd like to include a placebo as well, or two different treatments rather than only one?  This gives us an idea of what such data would look like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXEAAAEACAYAAABF+UbAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAElJJREFUeJzt3W+MXFd5x/Hv4ySOJoVE3rKLaNLsqkC0aiQnpcVNRdWu\noamdVGqq+AUJBVTeJFSkAVWtgvrGa6kS8KIqf6I0hG6pqIRjCSPVLVSAilcVpYStSGwKMUmKdklS\nyA7YtAWWxoSnL3ZMJsPs7nh31veeO9+PNNbO7Nm7TyZnfj4+9557IjORJJVpR9UFSJI2zxCXpIIZ\n4pJUMENckgpmiEtSwQxxSSrYhiEeEXMR8UxEnFynzfsj4vGIeCQirh9uiZKktQwyEv8wsG+tb0bE\nTcDLM/OVwJ3A/UOqTZK0gQ1DPDM/B5xZp8ktwEc6bR8CroiIlw6nPEnSeoYxJ34l8GTX86c7r0mS\ntpknNiWpYBcP4RhPAz/f9fyqzms/JSK8UYskbUJmRr/XBx2JR+fRzzHgzQARcQPw3cx8Zp1Cav84\nePBg5TU06eH76XtZ10cp7+d6NhyJR8RHgRngZyPiG8BBYOdqHucDmfnJiLg5Ip4Avg+8ZaNjSpKG\nY8MQz8w3DNDmruGUI0k6H57Y7GNmZqbqEhrF93N4fC+HqwnvZ2w03zLUXxaRF/L3SVITRAS5xROb\nkqQaMsQlqWCGuCQVzBCXpIIZ4pJUMENckgpmiEtSwQxxSSqYIS5JBTPEJalghrgkFcwQl6SCGeKS\nVDBDXNuq3W6zsLBAu92uuhTpBZrSNw1xbZvDh48wOTnNjTe+lcnJaQ4fPlJ1SRLQrL7p/cS1Ldrt\nNpOT06ysHAd2AydptfaytHSK8fHxqsvTCCuxb653P3FDXJsWsdbe2Ztj39AwNal/uimEtsV6u3Mv\nLy/Tao0BJ4AETtBqjbG8vLypHb2l8zXMvlnn/mmIa1uMj48zN3cfrdZeAFqtvczN3Vfbf65qdHT3\nzcsvf1XxfdPpFG2rdrvNxMQ4y8vtYj8kaqZ2u83i4iJTU1O175tOp6gy5z4cdf+QaPSMj4/ziU+8\nuvi+aYhr2x08WHUFUn+HDlVdwdY5nSJpZEVACZHkdIokNZQhLkkFM8QlqWCGuKSR1YST7oa4tt3s\nbNUVSP01oW96dYq2XSlXAEh15dUpktRQhrgkFcwQl6SCGeKSRpYnNs/3l3lisxhjY3DmTNVV/LRd\nu+D06aqrUFOUctJ9yyc2I2J/RJyKiMci4p4+3788Io5FxCMR8eWI+IMt1qyKnTmz2rnr9qjjXyxS\nlTYM8YjYAdwL7AOuBW6PiOmeZm8DvpKZ1wN7gb+IiIuHXazK05QdxdU85/pk6X1zkJH4HuDxzFzK\nzLPAg8AtPW0SeHHn6xcD38nMHw2vTJWoSTuKq1nO9U2g+L654Zx4RBwA9mXmHZ3nbwT2ZObdXW1e\nBBwDpoEXAa/PzH/qcyznxAux1bnC7dpRvJQ5TNVX03a7H9aUxz7g4cx8bUS8HPhMROzOzO/1Npzt\nOh08MzPDzMzMkEpQnSwuLrJz5xQrK7s7r+zmkksmWVxcrO0HRWXY+kn3ceA7Xc93s7LyHSYmtlbX\nME+6z8/PMz8/P1DbQUbiNwCzmbm/8/ydQGbme7ra/CPwrsz8187zfwbuycx/7zmWI/FCOBJXXY1i\n39zq1SkLwCsiYjIidgK3sTp10m0J+K3OL3spcA3w9c2XrNI1bUdxNUfT+uZA14lHxH7gfayG/lxm\nvjsi7mR1RP5ARLwM+FvgZZ0feVdmHu5zHEfihRjWqGLYO4o7Etco9s31RuIu9lFfdQ3LutalC6eu\nfaDO0ymSpJoyxCWpYIa4JBXMEJekghniklQwQ1ySCmaIS1LBDHFJKpghLkkFM8QlqWCGuCQVzBCX\npIK5D6b6SgL63m6nWtn1pyRDXGsIsr53iqu6CKlGnE6RpIIZ4pJUMENckgpmiEtSwQxxSSqYIS5J\nBTPEJalghrgkFcwQl6SCGeKSVDBDXJIKZohLUsEMcW2rdrvNwsIC7Xa76lKkF2hK3zTEtW0OHz7C\n5OQ0N974ViYnpzl8+EjVJUlAs/pm5AW832hE5IX8fdq8CLZ0K9p2u83k5DQrK8eB3cBJWq29LC2d\nYnx8vLK6VL5R7JsRQWb2vcO/9xPXtlhcXGTnzilWVnZ3XtnNJZdMsri4uKUPirTVDUvGgR8AcN3z\nL64AExNbrOv5Py8kp1O0Laampnj22UXgZOeVk5w9u8TU1FR1RakRglwd8m7y0V5e5rLWGMEJgiQ4\nwWWtMdrLy1s6blS0XYkhrm0xPj7O3Nx9tFp7ufzyV9Fq7WVu7j5H4apc0/qmc+LqK2q4vybArl1w\n+nTVVahKw5p7brfbLC4uMjU1NZQAr2pO3BDXtvNkpIaprv2pqhB3OkWSCjZQiEfE/og4FRGPRcQ9\na7SZiYiHI+I/IuL4cMuUJPWz4XRKROwAHgNeB/wXsADclpmnutpcAXwe+O3MfDoiXpKZ3+5zLKdT\nRlBd//mrMtW1P9V5OmUP8HhmLmXmWeBB4JaeNm8Ajmbm0wD9Alyj6dyS5tKXNqt5RmnZ/ZXAk13P\nn+q81u0aYCwijkfEQkS8aVgFqlznljZfeukHi1/arGYZqWX3EXEA2JeZd3SevxHYk5l3d7X5APDL\nwGuBnwH+Dbg5M5/oOZbTKSNiu5Y2Sy67f6FBlt0/DVzd9fyqzmvdngK+nZk/BH4YEf/C6prWJ3ra\nMTs7+5OvZ2ZmmJmZGaAE1VEMdDH580ubV1ZgYp2lzf4Fr0ENto5ho0aD981V6/fPXbsGqWkw8/Pz\nzM/PD9R2kJH4RcDXWD2x+U3gi8DtmfloV5tp4APAfuBS4CHg9Zn51Z5jORIfEY7EVVcl9s0tndjM\nzOeAu4BPA18BHszMRyPizoi4o9PmFPApVm+U8QXggd4A12hp2tJmNUfT+qYrNrWthr20WRqWkvqm\ny+4lqWAuu5ekhjLEJalghrgkFcwQl6SCGeKSVDBDXJIKZohLUsEMcUkqmCEuSQUzxCWpYIa4JBXM\nEJekghniklQwQ1ySCmaI92jKDthqHvum+jHEuzRpB2w1i31Ta3FTiI4S993TaLBvyp19XljD0I5V\n9X+LmmWYfRPsn03izj5dMrPvY3l5mVZrDDgBJHCCVmuM5eXlNX9GGqZh9k375+gYuRBfS9N2wFZz\n2De1npGbTtlISTtga7TYN0eXc+LnaXZ29SFJdeCc+Hk6dKjqCqT+HFyolyPxPiKggDI1guybo8mR\nuCQ1lCEuSQUzxCWpYIZ4HwcPVl2BJA3GEO/DKwBUVw4w1MurUySp5rw6RZIayhCXpIIZ4pJUMEO8\nD09sSiqFId6H905RXTnAUK+BQjwi9kfEqYh4LCLuWafdqyPibETcOrwSJZ3jAEO9NgzxiNgB3Avs\nA64Fbo+I6TXavRv41LCLlCT1N8hIfA/weGYuZeZZ4EHglj7t/gj4GLA8xPokSesYJMSvBJ7sev5U\n57WfiIifA34vM/8KGO5ur5KkNQ3rxOZ7ge658qKD3KXNkkpx8QBtngau7np+Vee1br8CPBgRAbwE\nuCkizmbmsd6DzXadXp+ZmWFmZuY8S95+XgGgYRsbgzNnhnOsGNIQadcuOH16OMfScM3PzzM/Pz9Q\n2w3vnRIRFwFfA14HfBP4InB7Zj66RvsPA/+QmR/v8z3vnaKRVMcdeepYk/pb794pG47EM/O5iLgL\n+DSr0y9zmfloRNy5+u18oPdHtlyxJGkg3sVQugDqOOqtY03qz7sYSlJDGeJ9eGJTUimcTunDf2Zq\n2OrYp+pYk/pzOkWSGsoQl6SCGeKSVDBDXJIKNsiy+yIMc1kzuLRZUhkac3VKXc+017UuXVh17Ad1\nrEn9eXWKJDWUIS5JBTPEJalghrgkFcwQl6SCGeKSVDBDvEe73WZhYYF2u111KdIL2DfVjyHe5fDh\nI0xOTnPjjW9lcnKaw4ePVF2SBNg3tTYX+3S0220mJ6dZWTkO7AZO0mrtZWnpFOPj45XVpWbYSj+w\nb2pLe2yOisXFRXbunGJlZXfnld1ccskki4uLW/qgSABJwCZv5TAO/ACA655/cQWYmNhiTc//qXI5\nndIxNTXFs88uAic7r5zk7NklpqamqitKjRHk6rB3E4/28jKXtcYIThAkwQkua43RXl7e9DHJXK1J\nxTPEO8bHx5mbu49Way+XX/4qWq29zM3d5yhclbNvaj3Oifdot9ssLi4yNTU1lA+J846C4fQD++bo\nWm9O3BDfZnWtSxdWHftBHWtSf97FUJIayhCXpIIZ4pJUMENckgpmiEtSwQxxSSpYY5bdb2VZ83Zy\nabPOiZr1z127qq5Aw9CYEA+ylte8RhjhGt712F7brV5Op0hSwQxxSSqYIS5JBTPEJalghrhUkIMH\nq65AdTPQXQwjYj/wXlZDfy4z39Pz/TcA93Se/i/wh5n55T7H8S6GknSetnQXw4jYAdwL7AOuBW6P\niOmeZl8HfiMzrwP+HPjQ1kqujjuKq67sm+pnkOmUPcDjmbmUmWeBB4Fbuhtk5hcy8787T78AXDnc\nMi8MdxRXXdk3tZYNp1Mi4gCwLzPv6Dx/I7AnM+9eo/2fANeca9/zvdpOp7ijuOpqu/qmynHBdruP\niL3AW4BfX6vN7OzsT76emZlhZmZmmCVsmrvdq2qx4br853e7X1mBiQ12u7+Qu3ZpuObn55mfnx+o\n7SAj8RuA2czc33n+TiD7nNzcDRwF9mfmf65xLEfi0nlyJK6tbs+2ALwiIiYjYidwG3Cs5xdczWqA\nv2mtAK87dxRXXdk3tZ7zucTwfTx/ieG7I+JOVkfkD0TEh4BbgSVW7yV4NjP39DlObUfi57ijuOpq\n2H1T5XC3+wrVtS5J5XC3e0lqqMbcTxzqd9N98Mb7krZXY0J8mFMWToFIKoXTKZJUMENckgpmiEtS\nwQxxSSqYId6HN96XVIrGLPaRpKZysY8kNZQhLkkFM8QlqWCGuCQVzBDvo2vzIUmqNUO8R7vd5tAh\n3FFcarh2u839999f/GfdEO9ybkdxwB3FpQY791l/xzsOFf9Z9zrxDvcxlEbDCz/rHwdurf1n3evE\nu0RE38fExAQrK6dZ3VE8gOtYWTnNxMTEmj8jqTxHjx7lxz/eyWqAHwI+znPP7eTo0aMVV7Y5Ixfi\nmdn3sby8TKs1BpwAEjhBqzXG8vLymj8jqTwHDhxgx45nWd0W+CBwKxdd9CwHDhyouLLNGbkQX4s7\nikujofuzvnPnB4v/rDsn3sMdxaXR0G63OXr0KAcOHKj9Z30kdruXpKbyxKYkNZQhLkkFM8QlqWCG\nuCQVzBCXpIIZ4pJUMENckgpmiEtSwQxxSSqYIS5JBTPEJalghrgkFcwQl6SCDRTiEbE/Ik5FxGMR\ncc8abd4fEY9HxCMRcf1wy5Qk9bNhiEfEDuBeYB9wLXB7REz3tLkJeHlmvhK4E7h/G2q9YObn56su\noVF8P4fH93K4mvB+DjIS3wM8nplLmXkWeBC4pafNLcBHADLzIeCKiHjpUCu9gJrwP7ZOfD+Hx/dy\nuJrwfg4S4lcCT3Y9f6rz2nptnu7TRpI0ZJ7YlKSCbbg9W0TcAMxm5v7O83cCmZnv6WpzP3A8M490\nnp8CfjMzn+k5lnuzSdImrLU928UD/OwC8IqImAS+CdwG3N7T5hjwNuBIJ/S/2xvg6xUhSdqcDUM8\nM5+LiLuAT7M6/TKXmY9GxJ2r384HMvOTEXFzRDwBfB94y/aWLUmCC7zbvSRpuDyx2SUi5iLimYg4\nWXUtpYuIqyLisxHxlYj4ckTcXXVNJYuISyPioYh4uPN+Hqy6ptJFxI6I+FJEHKu6lq0wxF/ow6wu\natLW/Qj448y8Fvg14G29i8Q0uMz8P2BvZv4ScD1wU0Tsqbis0r0d+GrVRWyVId4lMz8HnKm6jibI\nzG9l5iOdr78HPIprB7YkM3/Q+fJSVs9nORe6SRFxFXAz8NdV17JVhri2XURMsTp6fKjaSsrW+ef/\nw8C3gM9k5kLVNRXsL4E/pQF/ERri2lYR8SLgY8DbOyNybVJm/rgznXIV8KsR8YtV11SiiPgd4JnO\nvxSj8yiWIa5tExEXsxrgf5eZf191PU2Rmf8DHAf2V11LoV4D/G5EfB04DOyNiI9UXNOmGeI/rfi/\nmWvkb4CvZub7qi6kdBHxkoi4ovN1C7gROFVtVWXKzD/LzKsz8xdYXbz42cx8c9V1bZYh3iUiPgp8\nHrgmIr4RES5a2qSIeA3w+8BrO5fFfSkiHDlu3suA4xHxCKvnFj6VmZ+suCbVgIt9JKlgjsQlqWCG\nuCQVzBCXpIIZ4pJUMENckgpmiEtSwQxxSSqYIS5JBft/EBpgOz/s1HgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c71b748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sample_one = [0.1, 0.2, 0.3, 0.5, 0.6, 0.65, 0.8, 0.9, 0.2, 0.3, 0.5, 0.6, 0.65, 0.8, 0.9, 0.1]\n",
    "sample_two = [1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 4, 4, 4, 4, 4]\n",
    "\n",
    "scatter( sample_two, sample_one )\n",
    "# Some gimmickry to split sample_one into its four conditions\n",
    "dummy = boxplot( [sample_one[:4], sample_one[4:8], sample_one[8:11], sample_one[11:]] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But we don't yet know how to ask the numerical question:\n",
    "\n",
    "> Suppose that all of these many samples had really been drawn from the same underlying population.  What's the probability that they would be this different simply due to chance?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T-testing more than two groups: ANOVA\n",
    "\n",
    "Unsurprisingly, the most common way to quantify this test is *parametrically*, by making exactly the same assumptions that go into a t-test.  That is, if we can safely assume that all of our values are independently drawn from a normal distribution with evenly distributed error, then a procedure called an **ANOVA** will tell us whether any of our samples are surprisingly different.\n",
    "\n",
    "ANOVA stands for \"ANalaysis Of VAriance,\" and is [extensively described on Wikipedia](http://en.wikipedia.org/wiki/Anova) with some excellent examples if you're looking for the full details.  It completely generalizes a t-test: an ANOVA between just two levels, a case and a control, will provide exactly the same statistics and p-values as a t-test.  A t-test assesses the *normalized distance between two samples*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.LineCollection at 0x10cb4fa90>"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEACAYAAABVtcpZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHZtJREFUeJzt3XtsVved5/H313dssLna5mIbAgFCAjiFEBgKuNs2l7a7\ndHeqXma0lRppN8oqSqWtqqbTdiHbrabVqpUm00zSrpJROzvddjTTadKm7SRtcCGQEMIlXA0k5eqA\nDfiGbXz/7h9+oMb48jx+Ds/x4/N5SVae55zfc8735ODn43P5/Y65OyIiEk0ZYRcgIiLhUQiIiESY\nQkBEJMIUAiIiEaYQEBGJMIWAiEiEJR0CZjbPzF4zsyNmdsjMnhim3dNmdtLMDphZZbLrFRGR5GUF\nsIwe4L+7+wEzmwzsNbNX3L3megMzexhY6O53mtn9wHPA2gDWLSIiSUj6SMDdL7r7gdjrVuAYMHdQ\ns83Aj2NtdgNFZlaS7LpFRCQ5gV4TMLP5QCWwe9CsucC5Ae9ruTUoREQkxQILgdipoH8Gvhg7IhAR\nkXEuiGsCmFkW/QHwD+7+4hBNaoGyAe/nxaYNtSwNZiQikiB3t7F8LqgjgReAo+7+N8PMfwn4PICZ\nrQWa3L1uuIW5+4T82bJlS+g1aPu0fdq+ifeTjKSPBMxsPfCXwCEz2w848FdARf/3uf/Q3X9tZh8z\ns3eBNuALya5XRESSl3QIuPtOIDOOdo8nuy4REQmWegynUFVVVdgl3FbavvSm7YsmS/Z8UtDMzMdb\nTSIi45mZ4SFfGBYRkTSkEBARiTCFgIhIhCkEREQiTCEgIhJhCgERkQhTCIiIRJhCQEQkwhQCIiIR\nphAQEYkwhYCIcPjwYb7xjW+EXYaEQCEgItTX14ddgoREISAiEmEKARGRCFMIiIhEmEJARCTCFAIi\nIhGmEBARiTCFgIhIhCkEREQiTCEgIhJhCgERkQhTCIiIRJhCQEQkwhQCIiIRphAQEYmwQELAzJ43\nszozOzjM/E1m1mRm+2I/Xw9ivSIikpysgJbz98DfAj8eoc12d/8PAa1PREQCEMiRgLu/DjSO0syC\nWJeIiAQnldcE1pnZATN72cyWpXC9IiIyjKBOB41mL1Du7u1m9jDwC2BxitYtIiLDSEkIuHvrgNe/\nMbO/M7Pp7t4wVPutW7feeF1VVUVVVdVtr1FEJF1UV1dTXV0dyLLM3YNZkNl84JfuvnyIeSXuXhd7\nvQb4J3efP8xyPKiaRCQ+r732Gtu2beOb3/xm2KXIGJgZ7j6m666BHAmY2U+AKmCGmZ0FtgA5gLv7\nD4FPmdljQDdwDfhMEOsVEZHkBBIC7v4Xo8x/BngmiHWJiEhw1GNYRCTCFAIiIhGmEBARiTCFgIhI\nhCkEREQiTCEgIhJhCgERkQhTCIiIRJhCQEQkwhQCIiIRphAQEYkwhYCISIQpBEREIkwhICISYQoB\nEZEIUwiIiESYQkBEJMIUAiKCu6Nne0eTQkBEMDNAIRBFCgERkQhTCIhI7FSQjgSiSCEgIhJhCgER\nwcx0YTiiFAIiEtMXdgESAoWAiMT0hl2AhEAhICIxOhKIIoWAiMSuB/SEXYaEQCEgIvT09ODeTU+P\ngiBqFAIiQkdHK+49dHR0hF2KpFggIWBmz5tZnZkdHKHN02Z20swOmFllEOsVkWC0tjYC3bS3t4dd\niqRYUEcCfw88ONxMM3sYWOjudwKPAs8FtF4RCUBvbzuZmU5zc3PYpUiKBRIC7v460DhCk83Aj2Nt\ndwNFZlYSxLpFJDltbW309bWRnZ3B5cu1YZcjKZaqawJzgXMD3tfGpolIyOrr68nN7SQvL5v6+lP0\n9elW0SjJCruAoWzduvXG66qqKqqqqkKrRWSiO3u2hlmzCqitzSY39xr19fWUlpaGXZaMoLq6murq\n6kCWZUGNF2JmFcAv3X3FEPOeA7a5+89i72uATe5eN0Rb1xgm6el73/sen/70p5k3b17YpUicmpub\nqa7+R8rLM/n97w/ywAOraW4uY8OGj4RdmiQgNvaTjeWzQZ4OstjPUF4CPg9gZmuBpqECQNJbY2Mj\nFy9eDLsMScDhw/uYOzcn9lAZKC2dTmPjcS5fvhxyZZIqQd0i+hNgF7DYzM6a2RfM7FEz+68A7v5r\n4JSZvQv8APhvQaxXRMaurq6OhoZjlJfPujEtIyODO+4oYP/+nbo2EBGBXBNw97+Io83jQaxLRJLX\n2dnJ22//jqVLp5KRkcHVq1e5dKmelpZmSktncPHiWY4ePcw999xydlcmmHF5YVhEbp++vj7efPMP\nzJjRzrRppWzfvoPvfvcZrlyZxO7dv+WJJx5j7dq17Nmzg6lTp+sazwSnYSNEIsTdefvtN+npOcnC\nhSW0tDTz9NPP0tX1Jbq6PkNX17d4+uln6ehoZ8WKmezd+2tdH5jgFAIiEeHu7Nv3Fk1N+1m+fB5m\nRl1dPdnZxcD1v/YXkJVVTF1dPVOm5HPXXQXs3PkiDQ0NYZYut5FCQCQC+vr62LPnDS5f3sPKlXPJ\nyOj/1S8pKaa7ux44H2t5ip6eekpKigGYPr2QJUty2bHj59TX14dTvNxWCgGRCa6np4ddu6ppadnP\nvfeWkZmZeWNeYWERTzzxGDk53yUn52fk5HyNJ554jMLCohttZs4sYtmyfN544xecP3/+1hVIWtOF\nYQmMe58eVj7OdHR0sHPnq+TknGflyrIb/QEG2rhxA7NmlfCrX73Jo49uvikArps2bQorV2ayd++L\nXLv2Ee68c0kqypcU0JGABGqoLxkJx9WrV3nttRcpLLzIXXfNG3HfTJkyhVmziocMgOsmT85n1apS\nTpx4lXfe2afAnyAUAiITUENDA9u2/Zyysg4WLBh9HKCB/QRGkpeXw6pV86ir28Xu3a+rQ9kEoBAQ\nmWAuXbrEjh0/Z/HiTGbPnj5q++3bd/Dkk/+DF1/cxiOPPMb27TtGbJ+VlUllZRnt7QfZuXObHkmZ\n5hQCEiDXKYKQNTQ0sGvXiyxbls/MmcOf2rluuH4Cox0RZGRksHx5GX19x3njjT/oiCCNKQREJoi2\ntjZ27vwVS5bkMW3alLg+M1I/gdGYGcuWzaOzs4Z9+94ae+ESKoWABEZHAeFxd/bs2cGcOV1xHQFc\nN1o/gdGYGcuXz+XChb1cuHAh8cIldAoBCZBOCYSltraWjo73KC+P78v7unj6CYwmMzOTpUunsW9f\ntf4QSEPqJyAB0hdAWE6dOsa8eZPHdItuPP0ERjNt2hQyM89z6dIliosTCyIJl44EJEA6EghLQ0Mt\n06fHdx1gKPH0ExhNUZHR3DzyBWUZfxQCEqA+dRYL0MBnbY/GfTz8v3fdJZSGFAISiP5f/h59CQTo\nqaeeirvtjBlzaWi4ehurGV1LC0ybNi3UGiRxCgEJRH+HoR66u7vDLiWSysuXcv58eCHQ3NxKd3ch\nM2bMCK0GGRuFgASiq6sL9x46OzvDLiWSysvLycoqp7b20pg+H++wEUPp6+ujpuYKK1ZsuGmEUkkP\nCgEJRGdnJ2Y9dHa2hV1KJJkZ9923idOnjcbGxI4IEh02YrAjR84zc2YlZWVlCX1OxgeFgATi2rVr\nQA/t7S1hlxJZhYWFrFv3cY4evUpLS3xhPNZhI66rqanFbCGrV68bBxemZSwUAhKI1tZWzHq5dk0h\nEKbi4mLuv38zhw+30NTUOmr7sQ4b4e4cPXqenp75rF//YZ0GSmMKAQlEU1MdOTlZ9PZeo7e3N+xy\nIq20tJS1az/JkSPt1NU1jth2LMNG9Pb28s475zBbzAc/+BGys7ODK15STiEggWhoqCUjIwP3a+ow\nNA4UFxezadOfc/p0JmfPDv9XfWFhEQ88UAX8NfB/gS/zwANVw3Ya6+rqZv/+8xQVfYD16z9EVpYG\nHUh3CgFJWltbG93dzeTlZVNYmMHly2O7Q0WCNXXqVKqqPsmlS9M4efL9Icf1aWlp5pVXqoEngAeA\nLbzySvWQ1wTa2jp4++33KS+vYvXqtTceVi/pTXtRklZfX8/Uqf0XBadNy6e29mTIFcl1BQUFfOhD\nn6Czcz5Hj9be0pnvT9cElgDFwPIhrwm0tLRx4MBlli//GMuW3aOLwBOIQkCSdubMMWbNygdg+vQp\nNDefo729PeSq5LqcnBw2bPgI2dl3cejQ+ZuCIJ5rAk1NrRw61MJ9921mwYI7Ulu83HYKAUnK1atX\naW4+e2MMezNj5kzjzJlTIVcmA2VmZrJ27UYKClbcFASjDSXd1NTKkSNXWbfuk8yZMyfMTZDbJJAQ\nMLOHzKzGzE6Y2VeGmL/JzJrMbF/s5+tBrFfCd/z4EWbPzrjp/HBZ2QxOnnxbz54dZzIyMlizZj35\n+cs5duz9G9M3btzAt7/9P9m8+UO88MKzbNy4Aei/BnDkSDNr127W8NATWNIhYGYZwPeBB4G7gc+Z\n2dIhmm539w/Efv5XsuuV8LW2tnL+/DuUlc2ipaWZ+vpLXL16lYKCPKZMaeO993RtYLy5HgS9veWc\nOnXxxvTBQ0l3d/dw8GA9lZUPU1JSEla5kgJBHAmsAU66+xl37wZ+Cmweop2uJE0wBw68RVlZFm+8\n8QaPPPIYL7+8gy996ats376DhQuLOXZsZ6wnsYwnmZmZrFv377hwIWfYDmU1NRcoK1tLRUVFiquT\nVAsiBOYC5wa8Px+bNtg6MztgZi+b2bIA1ishOn/+PM3NRykqyo4NO/Aturr+E93dX+Hpp5+lp6eD\nOXP62LfvzbBLlSFMmjSJ1as/Sk1Nwy23jtbXN9LVVcLy5ZUhVSeplKqeHnuBcndvN7OHgV8Ai4dr\nPPBhGlVVVVRVVd3u+lJi69atCY0Rn16+eNO7hQv/c0h1TCypvhXzy1/+u5SuL5W2bNmS0IN6xrPq\n6mqqq6sDWZYl+2BoM1sLbHX3h2LvnwTc3b8zwmdOAavcvWGIea6HVY9fvb29/OEPv6Wo6CIVFcW0\ntDTzyCOP0dX1LS5dOkxh4TymTPnfvPDCsxQWFtHW1sGBA41s2vRppk6dGmgtZqYHmyepsbGRHTv+\nH+XlWfzud+/w0ENruHKlmE2bHg67NElA7HdhTH8xBHE6aA+wyMwqzCwH+Czw0qACSwa8XkN/+NwS\nADL+7dv3FmZnqKjov1vkT7cYfo2cnH8hO/s7N91iWFCQx6JFeezc+Rs6OjrCLF2GMG3aNKZMKefK\nlf6B/95/v40771wZclWSSkmfDnL3XjN7HHiF/lB53t2Pmdmj/bP9h8CnzOwxoBu4Bnwm2fVK6tXU\nHOXSpb2sWjXvpukbN26gsnIFzz33L2ze/CGWLFlw0/ySkmm0tV1g167fs3HjgxpvZpwpL7+Lt97a\nS29vD21tubobKGIC+W1099/S3+984LQfDHj9DPBMEOuScJw5c4YTJ7axatWcIYcNLiwsorh4FlOm\nTBny83fcMZujR8+ye/cO1q3bpHFnxpGSkhI6O7Pp6OiipOQODQsdMfpNlFFdvHiRAwd+w8qVxeTm\njn3Y4Lvumsu1a0fZu3e3zuWPI5MnTyYjo4CeHmfmzKFu7JOJTCEgI7p8+TK7d7/E8uVTKSjIS2pZ\nZsby5XO5fPltDh06EFCFEoSMjEl0d0NR0dBDSMvEpRCQYTU3N7Nr1y9ZurSAwsKCQJaZmZlJZeU8\nzp17nRMnagJZpiRv8uRpmGVTUBDMfpb0oRCQIbW3t/P66y9zxx0ZzJhRGOiys7OzWLlyDjU1v+fs\n2bOBLlvGJi9vCmZZ5OUld7Qn6UchILfo6elh165XKS29Rmnp9Nuyjry8HFasmMX+/b/lypUrt2Ud\nEr/s7GzMsnTnVgQpBOQWe/e+SU7O+zf6AtwukydPYsmSfN54Q30IwtbfM1nPCo4ihYDc5PTp01y+\nfIClS1MzdvzMmUUUF7ezZ8/rumModPo6iCLtdbmhs7OTgwe3cffdsxK+j3/gUNKJWrCglNbW47o+\nEDr1D4gihYDccPz4UWbM6GTy5PyEPrd9+45bhpJOhJmxePF0Dh/eecszcCWV9HUQRdrrAvRfDP7j\nH/dTUTEroc+1tDQPOZR0S0tzQsspKppMTk4TFy5cSOhzEhw9Oz6aFAIC9HcKKyjoIi8vJ6HP1dXV\nk51dDFwfL6iMrKxi6urqE66huDiP2lo9mzgM/ddjlAJRpBAQAJqamhhm2J8RlZQU091dD1z/8j5H\nT089JSWJ31lUVFRAQ8P7ozeUwPXfHaQQiCKFgADQ3d1JZmbi/xxGG0o6EVlZWfT0dMbdfsuWLQmv\nQ0Rupp4hAkBeXj6trWO7KDvaUNLx6u7uIScn/ovSE+UpUeOBTgdFl44EBOgfOKylZez36Y82lHQ8\nGhuvMn16avonyM3MLOWPspTxQSEgAMycOZPOznza2sbWczeZfgLX1dd3MW/e2I4iRGRsFAICQEZG\nBnfeuZrTpy8n/Nlk+wkAXL7cjFmJnmolkmIKAblh0aLFtLYW0tgY/1/zQfQT6Ovr4+TJZlauXK9T\nEiIpphCQG7Kzs1m16sMcO9ZId3dPXJ8Jop/AiRMXKCmppLS0NPGiRSQpCgG5SWlpKRUV6zh69P24\nBnRLtp/AhQtXaG2dyb33rhl70SIyZgoBucXy5ZVkZS3m3XdHH8IhmX4CTU2t/PGPffzZnz1AdraG\nMRYJg/oJyC0yMjJYt24T27Zd5ezZesrLR/6rfiz9BFpbr3HkSBPr1v05hYXBPrlMROKnIwEZUk5O\nDhs2PMSFC/lcvBjsk786Orp4551L3Hvvxykuvr0PrhGRkSkEZFj5+fls2PBxTp3K4PLl4e/2SeQW\n0a6ubg4cuMDddz9AeXn57ShbRBKgEJARFRYWsn79Jzh+vIOmptZb5idyi2hvby/vvFPL/PkbWLTo\nzlSULyKjUAjIqKZPn87atf+eI0eabulRHO8tou7OoUO1lJTczz33rExN4SIyKoWAxKWkpITKyo9x\n8GA9XV3dA6bHd4toTU0teXl3UVm5OnVFi8ioFAISt4qKChYs2MChQ3/qQxDPLaLnzl3i2rXZ3H//\nhoSfXSwit1cgv5Fm9pCZ1ZjZCTP7yjBtnjazk2Z2wMwqg1ivpN7ddy+noGAZ77138ca0jRs38MIL\nz/Lxj2/ku9/9azZu3HBjXnNzK+fOGevXf1R9AUTGoaRDwMwygO8DDwJ3A58zs6WD2jwMLHT3O4FH\ngeeSXa+Ew8y47771XLkymYaGlhvThxpKure3l6NHG1i9+kEKCgrCKFdERhHEkcAa4KS7n3H3buCn\nwOZBbTYDPwZw991AkZlpuMg0lZuby+rVH6ampone3t5h2737bh2zZ69izhw9I0BkvAoiBOYC5wa8\nPx+bNlKb2iHaSBopKSmhtPReTp2qG3L+1avtXLmSx4oVH0hxZSKSiHE5bMTAxwZWVVVRVVUVWi0y\nvOXL7+Xf/u0IZWXd5ObefL7/vfeucM89D5GTkxNSdSITV3V1NdXV1YEsK4gQqAUGdv2cF5s2uE3Z\nKG1u0LNj08OkSZO4447VnDnzFosX/+mUT0tLG52dU5k/f354xYlMYIP/OH7qqafGvKwgTgftARaZ\nWYWZ5QCfBV4a1OYl4PMAZrYWaHL3oc8jSFpZtGgJdXV+07WBc+caWbx4tW4HFUkDSR8JuHuvmT0O\nvEJ/qDzv7sfM7NH+2f5Dd/+1mX3MzN4F2oAvJLteGR/y8/OZNWsRly6dB6C3t4/GxgzWrq0IuTIR\niUcg1wTc/bfAkkHTfjDo/eNBrEvGn/Lyxbz33kkAGhtbmDFjGbm5uSFXJSLx0PG6JK24uJimpv7X\nDQ3tzJ69MNyCRCRuCgFJWl5eHpMmzaC9vZOrV2HmzJlhlyQicVIISCCmT5+DO2Rm5utJYSJpRCEg\ngSgqmkVXVzcZGZN0V5BIGtFvqwRi8uTJQBYFBVPDLkVEEqAQkEBMmjQJyGLSpCmjthWR8UMhIIHI\nzc3FPYvc3PywSxGRBCgEJBA5OTmYZal/gEiaUQhIILKysoBsPThGJM0oBCQQZgZkxv4rIulCISAB\n0j8nkXSj31oJkP45iaQb/dZKgAx3D7sIEUmAQkACY5ahawIiaUYhIAHSkYBIulEISKB0JCCSXhQC\nEiAFgEi6UQiIiESYQkACpWsC6WnRokXk52vcpyhSCEhgzEzXBNJUeXk5X/3qV8MuQ0KgEJAAKQBE\n0o1CQEQkwhQCIiIRphAQEYkwhYCISIQpBCRQU6boGcMi6cTG233dZubjrSaJT3t7u+41FwmBmeHu\nY7o9TyEgIpLmkgmBrCRXPA34GVABnAY+7e7NQ7Q7DTQDfUC3u69JZr0iIhKMZK8JPAn8zt2XAK8B\nw3U57AOq3P1eBYCIyPiRbAhsBn4Ue/0j4JPDtLMA1iUiIgFL9ou52N3rANz9IlA8TDsHXjWzPWb2\nX5Jcp4iIBGTUawJm9ipQMnAS/V/qXx+i+XBXdNe7+wUzm0V/GBxz99cTrlZERAI1agi4+0eHm2dm\ndWZW4u51ZlYK1A+zjAux/14ys38F1gDDhsDWrVtvvK6qqqKqqmq0MkVEIqO6uprq6upAlpXULaJm\n9h2gwd2/Y2ZfAaa5+5OD2uQDGe7eamYFwCvAU+7+yjDL1C2iIiIJCK2fgJlNB/4JKAPO0H+LaJOZ\nzQb+j7t/wswWAP9K/6miLOAf3f3bIyxTISAikgB1FhMRibBkQkC3bYqIRJhCQEQkwhQCIiIRphAQ\nEYkwhYCISIQpBEREIkwhICISYQoBEZEIUwiIiESYQkBEJMIUAiIiEaYQEBGJMIWAiEiEKQRERCJM\nISAiEmEKARGRCFMIiIhEmEJARCTCFAIiIhGmEBARiTCFgIhIhCkEREQiTCEgIhJhCgERkQhTCIiI\nRJhCQEQkwhQCIiIRphAQEYmwpELAzD5lZofNrNfMPjBCu4fMrMbMTpjZV5JZp4iIBCfZI4FDwH8E\n/jBcAzPLAL4PPAjcDXzOzJYmud60VF1dHXYJt5W2L71p+6IpqRBw9+PufhKwEZqtAU66+xl37wZ+\nCmxOZr3paqL/I9T2pTdtXzSl4prAXODcgPfnY9NERCRkWaM1MLNXgZKBkwAHvubuv7xdhYmIyO1n\n7p78Qsy2AV9y931DzFsLbHX3h2LvnwTc3b8zzLKSL0hEJGLcfaTT8sMa9UggAcMVsAdYZGYVwAXg\ns8DnhlvIWDdEREQSl+wtop80s3PAWuBXZvab2PTZZvYrAHfvBR4HXgGOAD9192PJlS0iIkEI5HSQ\niIikp1B6DMfTeczMnjazk2Z2wMwqU11jMkbbPjPbZGZNZrYv9vP1MOocCzN73szqzOzgCG3Sed+N\nuH3pvO8AzGyemb1mZkfM7JCZPTFMu7Tbh/FsWzrvPzPLNbPdZrY/tn1bhmmX2L5z95T+0B887wIV\nQDZwAFg6qM3DwMux1/cDb6a6ztu8fZuAl8KudYzb90GgEjg4zPy03Xdxbl/a7rtY/aVAZez1ZOD4\nRPn9i3Pb0n3/5cf+mwm8CaxJdt+FcSQQT+exzcCPAdx9N1BkZiWkh3g7x6XlBXB3fx1oHKFJOu+7\neLYP0nTfAbj7RXc/EHvdChzj1n47abkP49w2SO/91x57mUv/jT2Dz+cnvO/CCIF4Oo8NblM7RJvx\nKt7Ocetih2svm9my1JSWEum87+I1Ifadmc2n/6hn96BZab8PR9g2SOP9Z2YZZrYfuAi86u57BjVJ\neN8FeYuoxG8vUO7u7Wb2MPALYHHINUl8JsS+M7PJwD8DX4z91TxhjLJtab3/3L0PuNfMCoFfmNky\ndz+azDLDOBKoBcoHvJ8Xmza4TdkobcarUbfP3VuvH9a5+2+AbDObnroSb6t03nejmgj7zsyy6P+S\n/Ad3f3GIJmm7D0fbtomw/wDcvQXYBjw0aFbC+y6MELjReczMcujvPPbSoDYvAZ+HGz2Om9y9LrVl\njtmo2zfwHJ2ZraH/Vt2G1JaZFGP486rpvO+uG3b7JsC+A3gBOOrufzPM/HTehyNuWzrvPzObaWZF\nsdeTgI8CNYOaJbzvUn46yN17zex657EM4Hl3P2Zmj/bP9h+6+6/N7GNm9i7QBnwh1XWOVTzbB3zK\nzB4DuoFrwGfCqzgxZvYToAqYYWZngS1ADhNg38Ho20ca7zsAM1sP/CVwKHZu2YG/ov9utrTeh/Fs\nG+m9/2YDP7L+4fkzgJ/F9lVS353qLCYiEmF6vKSISIQpBEREIkwhICISYQoBEZEIUwiIiESYQkBE\nJMIUAiIiEaYQEBGJsP8PAe5iYaVscmUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10caf8780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sample_one = [0.1, 0.2, 0.3, 0.5, 0.6, 0.65, 0.8, 0.9]\n",
    "sample_two = [1, 1, 1, 1, 2, 2, 2, 2]\n",
    "\n",
    "# Ignore this magic - it's providing a convenient visualization\n",
    "def violin_plot(ax,data,pos):\n",
    "    dist = max(pos)-min(pos)\n",
    "    w = min(0.15*max(dist,1.0),0.5)\n",
    "    for d,p in zip(data,pos):\n",
    "        k = scipy.stats.norm(mean(d), std(d))\n",
    "        m = min(d) - max(d)\n",
    "        M = 2 * max(d)\n",
    "        x = arange(m,M,(M-m)/100.)\n",
    "        v = k.pdf(x)\n",
    "        v = v/v.max()*w\n",
    "        ax.fill_betweenx(x,p,v+p,facecolor='y',alpha=0.3)\n",
    "        ax.fill_betweenx(x,p,-v+p,facecolor='y',alpha=0.3)\n",
    "\n",
    "scatter( sample_two, sample_one )\n",
    "\n",
    "# Split sample_one into the subsets from each condition for convenience\n",
    "sample_one_from_first_condition = sample_one[:4]\n",
    "sample_one_from_second_condition = sample_one[4:]\n",
    "violin_plot( gca( ), [sample_one_from_first_condition, sample_one_from_second_condition], [1, 2] )\n",
    "\n",
    "# Mark the mean of the total sample as a horizontal line\n",
    "hlines( mean( sample_one ), 0.5, 2.5 )\n",
    "# Mark the difference from the first condition mean as a vertical line on the left\n",
    "vlines( 1.25, mean( sample_one_from_first_condition ), mean( sample_one ) )\n",
    "# Mark the difference from the second condition mean as a vertical line on the left\n",
    "vlines( 1.75, mean( sample_one ), mean( sample_one_from_second_condition ) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is, the t-statistic is essentially a measure of how \"tall\" those two vertical lines are, normalized by how tall each sample is individually (i.e. how spread-out its distribution is).  This is reflected in the definition of the t-statistic above:\n",
    "\n",
    "> $$T = \\frac{\\bar{x}_1-\\bar{x}_2}{S_{1,2}}$$\n",
    "\n",
    "where $S_{1,2}$ is more or less equivalent to a standard deviation $\\sigma$.  An ANOVA uses a similarly-defined f-statistic:\n",
    "\n",
    "> $$F = \\frac{\\frac{1}{k-1}SS_{between}}{\\frac{1}{n-k}SS_{within}}$$\n",
    "\n",
    "where $n$ is the total sample size, $k$ is the number of different conditions (two for a t-test, for example), and the two $SS$ terms are the **sum of squared differences** we'd use just like calculating a standard deviation.  The first measures literally the standard deviations *within* each of the conditions' subsamples, or the spread of each distribution on the plot above:\n",
    "\n",
    "> $$SS_{within} = \\sum_{(condition\\ i)}\\sum_{(x\\ in\\ i)}(x - \\bar{x}_i)^2$$\n",
    "\n",
    "The second measures something like a standard devation *between* all of the conditions' subsamples, or the length of the vertical lines on the plot above:\n",
    "\n",
    "> $$SS_{between} = \\sum_{(condition\\ i)}|x_i|(\\bar{x}_i - \\bar{x})$$\n",
    "\n",
    "Notice that \\bar{x} is the overall mean (our horizontal line above), and each group's contribution to overall \"surprise\" is weighted by how big the group is.  A condition from which we have only a few samples might generate weird results just due to outliers or a fluke, so we don't want it to increase our statistic by very much.  Conversely, a condition from which we have many samples that are consistently unusual should increase the f-statistic by a lot.\n",
    "\n",
    "The downstream process of converting an f-statistic into a p-value again mirrors that of converting a t-statistic into a p-value.  An [f-distribution](http://en.wikipedia.org/wiki/F-distribution) has two parameters that are like the t-distribution's extra \"degrees of freedom\" component, which capture the sample size and number of conditions, respectively.  It's one-sided rather than two (note that sums-of-squares can only be positive), but otherwise plug in the f-statistic, integrate, and you've gotten yourself a p-value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I won't walk us through the arduous process of computing one of these by hand, since that's what computers are for!  Once you understand the intuition, what's important for this class is applying the test correctly.\n",
    "\n",
    "Let's generate some data.  Inspired by [the real-world data used on your problem set](http://growthrate.princeton.edu/), suppose we run 12 total different gene expression experiments in yeast, 4 replicates each under 3 different nutrient limitation environments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['carbon', 'carbon', 'carbon', 'carbon', 'nitrogen', 'nitrogen', 'nitrogen', 'nitrogen', 'phosphate', 'phosphate', 'phosphate', 'phosphate']\n"
     ]
    }
   ],
   "source": [
    "conditions = ([\"carbon\"] * 4) + ([\"nitrogen\"] * 4) + ([\"phosphate\"] * 4)\n",
    "print( conditions )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that there's no need for the number of replicates per condition to be the same, it's just for convenience.  We're using a special strain of yeast that only has 10 genes, so I'm going to generate a data matrix that has ten rows (one per gene) and 12 columns (one per sample):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[1, 2, 1, 2], [6, 9, 3, 1], [10, 9, 10, 9]], [[7, 3, 7, 4], [1, 4, 8, 3], [9, 3, 7, 6]], [[1, 9, 9, 2], [2, 1, 6, 5], [5, 6, 9, 3]], [[3, 8, 1, 7], [7, 1, 5, 9], [9, 1, 2, 1]], [[7, 2, 7, 2], [9, 2, 8, 7], [3, 6, 5, 3]], [[3, 5, 6, 2], [1, 1, 9, 4], [7, 6, 2, 4]], [[1, 6, 9, 2], [9, 1, 8, 5], [3, 6, 6, 5]], [[4, 6, 3, 6], [7, 8, 9, 1], [6, 4, 4, 2]], [[6, 6, 8, 8], [7, 2, 2, 5], [6, 3, 6, 1]], [[9, 7, 4, 6], [4, 8, 9, 9], [5, 6, 4, 4]]]\n"
     ]
    }
   ],
   "source": [
    "data_for_all_genes = []\n",
    "# For each of our 10 genes:\n",
    "for gene in range( 10 ):\n",
    "    data_for_this_gene = []\n",
    "# For each of our three conditions:\n",
    "    for condition in range( 3 ):\n",
    "# Generate four replicate measurements between 1 and 10:\n",
    "        data_for_this_gene.append( [random.randint( 1, 10 ) for i in range( 4 )] )\n",
    "    data_for_all_genes.append( data_for_this_gene )\n",
    "\n",
    "# Rig the game so that our first gene looks interesting\n",
    "# The first gene is repressed under carbon limitation:\n",
    "data_for_all_genes[0][0] = [1, 2, 1, 2]\n",
    "# And overexpressed under phosphate limitation:\n",
    "data_for_all_genes[0][2] = [10, 9, 10, 9]\n",
    "\n",
    "print( data_for_all_genes )"
   ]
  },
  {
   "cell_type": "markdown",