spacy_nlp_intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Natural Language Processing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Online materials:\n",
    "- https://www.youtube.com/watch?v=jB1-NukGZm0\n",
    "- https://course.spacy.io/\n",
    "- https://spacy.io/usage/spacy-101\n",
    "- https://github.com/explosion/spacy-notebooks\n",
    "\n",
    "In this notebook we are gonna learn the basics of NLP using spaCy(https://spacy.io) \n",
    "\n",
    "Batteries included\n",
    "\n",
    "- Index preserving tokenization (details about this later)\n",
    "- Models for Part Of Speech tagging, Named Entity Recognition and Dependency Parsing\n",
    "- Supports 8 languages out of the box\n",
    "- Easy and beautiful visualizations\n",
    "- Pretrained word vectors\n",
    "\n",
    "It plays nicely with all the other already existing tools that you know and love: Scikit-Learn, TensorFlow, ...\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Topics\n",
    "- Tokenization\n",
    "- Part Of Speech\n",
    "- Named Entity Recoginition\n",
    "- Sentence Detection\n",
    "- Text Normalization\n",
    "- Word Vectors\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So why we need a NLP library? We are going to explore the answer for that in a while, but before that lets see how a basic operation of splitting a paragraph can go wrong.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/envs/aie/bin/python\n"
     ]
    }
   ],
   "source": [
    "!which python\n",
    "# /opt/envs/aie/bin/python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The waiter was very rude, \n",
      "e\n",
      "g\n",
      " when I accidentally opened the wrong door\n",
      "he screamed \"Private!\"\n",
      "\n"
     ]
    }
   ],
   "source": [
    "paragraph = '''The waiter was very rude, \n",
    "e.g. when I accidentally opened the wrong door\n",
    "he screamed \"Private!\".'''\n",
    "\n",
    "sentences = paragraph.split('.')\n",
    "print('\\n'.join(sentences))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import spacy and English models\n",
    "import spacy\n",
    "nlp = spacy.load('en_core_web_sm')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Language Processing Pipelines\n",
    "\n",
    "- When you call `nlp` on a text, spaCy first `tokenizes` the text to produce a `Doc object`.\n",
    "- The `Doc` is then processed in several different steps – this is also referred to as the processing pipeline.\n",
    "- The pipeline used by the default models consists of a `tagger`, a `parser` and an `entity recognizer`.\n",
    "\n",
    "More info [here](https://spacy.io/usage/processing-pipelines)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tokenization\n",
    "\n",
    "https://spacy.io/usage/spacy-101#annotations-token\n",
    "\n",
    "Tokenization is the process of segmenting text into words, punctuation etc. \n",
    "spaCy tokenizes the text, processes it, and stores the data in the Doc object.\n",
    "The Token class exposes a lot of word-level attributes.\n",
    "\n",
    "![](https://spacy.io/tokenization-57e618bd79d933c4ccd308b5739062d6.svg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "username = getpass.getuser()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hi, Myself mageswarand. I am glad to learn NLP and use it wherver applicable!'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text = \"Hi, Myself {}. I am glad to learn NLP and use it wherver applicable!\".format(username)\n",
    "text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "doc = nlp(text)\n",
    "# Here’s something interesting—After processing the text, \n",
    "#spaCy keeps all the information about the original text intact within the Doc object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hi\t0\thi\tFalse\tFalse\tXx\tINTJ\tUH\n",
      ",\t2\t,\tTrue\tFalse\t,\tPUNCT\t,\n",
      "Myself\t4\tMyself\tFalse\tFalse\tXxxxx\tPROPN\tNNP\n",
      "mageswarand\t11\tmageswarand\tFalse\tFalse\txxxx\tNOUN\tNN\n",
      ".\t22\t.\tTrue\tFalse\t.\tPUNCT\t.\n",
      "I\t24\t-PRON-\tFalse\tFalse\tX\tPRON\tPRP\n",
      "am\t26\tbe\tFalse\tFalse\txx\tAUX\tVBP\n",
      "glad\t29\tglad\tFalse\tFalse\txxxx\tADJ\tJJ\n",
      "to\t34\tto\tFalse\tFalse\txx\tPART\tTO\n",
      "learn\t37\tlearn\tFalse\tFalse\txxxx\tVERB\tVB\n",
      "NLP\t43\tNLP\tFalse\tFalse\tXXX\tPROPN\tNNP\n",
      "and\t47\tand\tFalse\tFalse\txxx\tCCONJ\tCC\n",
      "use\t51\tuse\tFalse\tFalse\txxx\tVERB\tVB\n",
      "it\t55\t-PRON-\tFalse\tFalse\txx\tPRON\tPRP\n",
      "wherver\t58\twherver\tFalse\tFalse\txxxx\tADV\tRB\n",
      "applicable\t66\tapplicable\tFalse\tFalse\txxxx\tADJ\tJJ\n",
      "!\t76\t!\tTrue\tFalse\t!\tPUNCT\t.\n"
     ]
    }
   ],
   "source": [
    "for token in doc:\n",
    "    print(\"{0}\\t{1}\\t{2}\\t{3}\\t{4}\\t{5}\\t{6}\\t{7}\".format(\n",
    "        token.text,\n",
    "        token.idx,\n",
    "        token.lemma_,\n",
    "        token.is_punct,\n",
    "        token.is_space,\n",
    "        token.shape_,\n",
    "        token.pos_,\n",
    "        token.tag_\n",
    "    ))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[spaCy Doc](https://spacy.io/api/doc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I\n",
      "ca\n",
      "n't\n",
      "imagine\n",
      "spending\n",
      "$\n",
      "3000\n",
      "for\n",
      "a\n",
      "single\n",
      "bedroom\n",
      "apartment\n",
      "in\n",
      "N.Y.C.\n"
     ]
    }
   ],
   "source": [
    "text = u\"I can't imagine spending $3000 for a single bedroom apartment in N.Y.C.\"\n",
    "doc = nlp(text)\n",
    "# Print out tokens\n",
    "for token in doc:\n",
    "    print(token)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sentence Detection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_sentences(text):\n",
    "    doc = nlp(text)\n",
    "    for sent in doc.sents:\n",
    "        print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print_sentences(paragraph)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = \"The path of the righteous man is beset on all sides by the iniquities of the selfish and the tyranny of evil men. Blessed is he who, in the name of charity and good will, shepherds the weak through the valley of darkness, for he is truly his brother's keeper and the finder of lost children. And I will strike down upon thee with great vengeance and furious anger those who would attempt to poison and destroy My brothers. And you will know My name is the Lord when I lay My vengeance upon thee.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print_sentences(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stop Words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I\n",
      "ca\n",
      "n't\n",
      "for\n",
      "a\n",
      "in\n"
     ]
    }
   ],
   "source": [
    "doc = nlp(text)\n",
    "for word in doc:\n",
    "    if word.is_stop == True:\n",
    "        print(word)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Named Entity Recognition](https://en.wikipedia.org/wiki/Named-entity_recognition)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Named entity recognition refers to the identification of words in a sentence as an entity e.g. the name of a person, place, organization, etc.\n",
    "\n",
    "`spaCy` can recognise various types of named entities in a document, by asking the model for a prediction. Because models are statistical and strongly depend on the examples they were trained on, this doesn't always work perfectly and might need some tuning later, depending on your use case.\n",
    "\n",
    "https://spacy.io/usage/linguistic-features#section-named-entities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Apple 0 5 ORG\n",
      "U.K. 27 31 GPE\n",
      "$1 billion 44 54 MONEY\n"
     ]
    }
   ],
   "source": [
    "doc = nlp(\"Apple is looking at buying U.K. startup for $1 billion\")\n",
    "\n",
    "for ent in doc.ents:\n",
    "    print(ent.text, ent.start_char, ent.end_char, ent.label_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "**Entities Explained** : https://spacy.io/api/annotation#named-entities\n",
    "\n",
    "    PERSON: People, including fictional.\n",
    "    NORP: Nationalities or religious or political groups.\n",
    "    FAC: Buildings, airports, highways, bridges, etc.\n",
    "    ORG: Companies, agencies, institutions, etc.\n",
    "    GPE: Countries, cities, states.\n",
    "    LOC: Non-GPE locations, mountain ranges, bodies of water.\n",
    "    PRODUCT: Objects, vehicles, foods, etc. (Not services.)\n",
    "    EVENT: Named hurricanes, battles, wars, sports events, etc.\n",
    "    WORK_OF_ART: Titles of books, songs, etc.\n",
    "    LAW: Named documents made into laws.\n",
    "    LANGUAGE: Any named language.\n",
    "    DATE: Absolute or relative dates or periods.\n",
    "    TIME: Times smaller than a day.\n",
    "    PERCENT: Percentage, including \"%\".\n",
    "    MONEY: Monetary values, including unit.\n",
    "    QUANTITY: Measurements, as of weight or distance.\n",
    "    ORDINAL: \" , \"second\", etc.\n",
    "    CARDINAL: Numerals that do not fall under another type.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "def explain_text_entities(text):\n",
    "    doc = nlp(text)\n",
    "    for ent in doc.ents:\n",
    "        print(f'Entity: {ent}, Label: {ent.label_}, {spacy.explain(ent.label_)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Entity: Apple, Label: ORG, Companies, agencies, institutions, etc.\n",
      "Entity: U.K., Label: GPE, Countries, cities, states\n",
      "Entity: $1 billion, Label: MONEY, Monetary values, including unit\n"
     ]
    }
   ],
   "source": [
    "explain_text_entities(\"Apple is looking at buying U.K. startup for $1 billion\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from spacy import displacy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"entities\" style=\"line-height: 2.5; direction: ltr\">But \n",
       "<mark class=\"entity\" style=\"background: #7aecec; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Google\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       " is starting from behind. The company made a late push into hardware, and \n",
       "<mark class=\"entity\" style=\"background: #7aecec; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Apple\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       "’s \n",
       "<mark class=\"entity\" style=\"background: #bfeeb7; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Siri\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">PRODUCT</span>\n",
       "</mark>\n",
       ", available on iPhones, and \n",
       "<mark class=\"entity\" style=\"background: #7aecec; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Amazon\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       "’s \n",
       "<mark class=\"entity\" style=\"background: #7aecec; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Alexa\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       " software, which runs on its \n",
       "<mark class=\"entity\" style=\"background: #bfeeb7; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Echo\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">PRODUCT</span>\n",
       "</mark>\n",
       " and \n",
       "<mark class=\"entity\" style=\"background: #bfeeb7; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Dot\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">PRODUCT</span>\n",
       "</mark>\n",
       " devices, have clear leads in consumer adoption.</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "text = \"\"\"But Google is starting from behind. The company made a late push into hardware, and Apple’s Siri, available on iPhones, and Amazon’s Alexa software, which runs on its Echo and Dot devices, have clear leads in consumer adoption.\"\"\"\n",
    "doc = nlp(text)\n",
    "displacy.render(doc, style='ent', jupyter=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"entities\" style=\"line-height: 2.5; direction: ltr\">But \n",
       "<mark class=\"entity\" style=\"background: linear-gradient(90deg, #aa9cfc, #fc9ce7); padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Google\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       " is starting from behind. The company made a late push into hardware, and \n",
       "<mark class=\"entity\" style=\"background: linear-gradient(90deg, #aa9cfc, #fc9ce7); padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Apple\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       "’s Siri, available on iPhones, and \n",
       "<mark class=\"entity\" style=\"background: linear-gradient(90deg, #aa9cfc, #fc9ce7); padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Amazon\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       "’s \n",
       "<mark class=\"entity\" style=\"background: linear-gradient(90deg, #aa9cfc, #fc9ce7); padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Alexa\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       " software, which runs on its Echo and Dot devices, have clear leads in consumer adoption.</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "colors = {\"ORG\": \"linear-gradient(90deg, #aa9cfc, #fc9ce7)\"}\n",
    "options = {\"ents\": [\"ORG\"], \"colors\": colors}\n",
    "displacy.render(doc, style='ent', jupyter=True, options=options)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Excercise**\n",
    "Imagine you are a journalist who wants to publish a large set of documents while still hiding the identity of your sources. Can you write a function that masks all personal names, i.e. by replacing them with \"[MASKED]\"?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class=\"entities\" style=\"line-height: 2.5; direction: ltr\">\n",
       "<mark class=\"entity\" style=\"background: #aa9cfc; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Harry Potter\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">PERSON</span>\n",
       "</mark>\n",
       " is a series of fantasy novels written by \n",
       "<mark class=\"entity\" style=\"background: #c887fb; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    British\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">NORP</span>\n",
       "</mark>\n",
       " author \n",
       "<mark class=\"entity\" style=\"background: #aa9cfc; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    J. K. Rowling\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">PERSON</span>\n",
       "</mark>\n",
       ". The novels chronicle the lives of a young wizard, \n",
       "<mark class=\"entity\" style=\"background: #aa9cfc; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Harry Potter\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">PERSON</span>\n",
       "</mark>\n",
       ", and his friends \n",
       "<mark class=\"entity\" style=\"background: #aa9cfc; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Hermione Granger\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">PERSON</span>\n",
       "</mark>\n",
       " and \n",
       "<mark class=\"entity\" style=\"background: #aa9cfc; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Ron Weasley\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">PERSON</span>\n",
       "</mark>\n",
       ", all of whom are students at \n",
       "<mark class=\"entity\" style=\"background: #7aecec; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Hogwarts School of Witchcraft\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       " and \n",
       "<mark class=\"entity\" style=\"background: #feca74; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Wizardry\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">GPE</span>\n",
       "</mark>\n",
       ". The main story \n",
       "<mark class=\"entity\" style=\"background: #7aecec; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    arc\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       " concerns \n",
       "<mark class=\"entity\" style=\"background: #aa9cfc; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Harry\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">PERSON</span>\n",
       "</mark>\n",
       "'s struggle against Lord \n",
       "<mark class=\"entity\" style=\"background: #bfeeb7; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    Voldemort\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">PRODUCT</span>\n",
       "</mark>\n",
       ", a dark wizard who intends to become immortal, overthrow the wizard governing body known as \n",
       "<mark class=\"entity\" style=\"background: #7aecec; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
       "    the Ministry of Magic\n",
       "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
       "</mark>\n",
       ", and subjugate all wizards and Muggles (non-magical people).</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def mask_names(text):\n",
    "    # your code here\n",
    "    return text\n",
    "    \n",
    "original_text = \"Harry Potter is a series of fantasy novels written by British author J. K. Rowling. The novels chronicle the lives of a young wizard, Harry Potter, and his friends Hermione Granger and Ron Weasley, all of whom are students at Hogwarts School of Witchcraft and Wizardry. The main story arc concerns Harry's struggle against Lord Voldemort, a dark wizard who intends to become immortal, overthrow the wizard governing body known as the Ministry of Magic, and subjugate all wizards and Muggles (non-magical people).\"\n",
    "masked_text = mask_names(original_text)\n",
    "doc = nlp(masked_text)\n",
    "displacy.render(doc, style='ent', jupyter=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lemmatization\n",
    "\n",
    "Lemmatization is the assigning of the base forms of words. For example: “was” → “be” or “cats” → “cat”\n",
    "\n",
    "To perform lemmatization, the Doc object needs to be parsed. The processed Doc object contains the lemma of words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success success nsubj\n",
      "is be ROOT\n",
      "not not neg\n",
      "final final acomp\n",
      ". . punct\n"
     ]
    }
   ],
   "source": [
    "doc = nlp(u\"Success is not final.\")\n",
    "for token in doc:\n",
    "    print(token.text, token.lemma_, token.dep_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Men man nsubj\n",
      "are be aux\n",
      "climbing climb ROOT\n",
      "up up prep\n",
      "the the det\n",
      "trees tree pobj\n",
      ". . punct\n"
     ]
    }
   ],
   "source": [
    "doc = nlp(u\"Men are climbing up the trees.\")\n",
    "for token in doc:\n",
    "    print(token.text, token.lemma_, token.dep_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Part Of Speech Tagging\n",
    "\n",
    "Check the list [here](https://spacy.io/api/annotation#pos-tagging)!\n",
    "\n",
    "Parts of speech tagging simply refers to assigning parts of speech to individual words in a sentence.\n",
    "\n",
    "Sometimes, we want to quickly pull out keywords, or keyphrases from a larger body of text. This helps us mentally paint a picture of what this text is about. This is particularly helpful in analysis of texts like long emails or essays.\n",
    "\n",
    "After tokenization, the text goes through parsing and tagging. With the use of the statistical model, spaCy can predict the most likely tag/label for a token in a given context.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Harry\tHarry\tPROPN\tNNP\tcompound\tXxxxx\tTrue\tFalse\n",
      "Potter\tPotter\tPROPN\tNNP\tnsubj\tXxxxx\tTrue\tFalse\n",
      "is\tbe\tAUX\tVBZ\tROOT\txx\tTrue\tTrue\n",
      "a\ta\tDET\tDT\tdet\tx\tTrue\tTrue\n",
      "series\tseries\tNOUN\tNN\tattr\txxxx\tTrue\tFalse\n",
      "of\tof\tADP\tIN\tprep\txx\tTrue\tTrue\n",
      "fantasy\tfantasy\tNOUN\tNN\tcompound\txxxx\tTrue\tFalse\n",
      "novels\tnovel\tNOUN\tNNS\tpobj\txxxx\tTrue\tFalse\n",
      "written\twrite\tVERB\tVBN\tacl\txxxx\tTrue\tFalse\n",
      "by\tby\tADP\tIN\tagent\txx\tTrue\tTrue\n",
      "British\tbritish\tADJ\tJJ\tamod\tXxxxx\tTrue\tFalse\n",
      "author\tauthor\tNOUN\tNN\tcompound\txxxx\tTrue\tFalse\n",
      "J.\tJ.\tPROPN\tNNP\tcompound\tX.\tFalse\tFalse\n",
      "K.\tK.\tPROPN\tNNP\tcompound\tX.\tFalse\tFalse\n",
      "Rowling\tRowling\tPROPN\tNNP\tpobj\tXxxxx\tTrue\tFalse\n",
      ".\t.\tPUNCT\t.\tpunct\t.\tFalse\tFalse\n"
     ]
    }
   ],
   "source": [
    "doc = nlp(\"Harry Potter is a series of fantasy novels written by British author J. K. Rowling.\")\n",
    "# For each token, print corresponding part of speech tag\n",
    "for token in doc:\n",
    "    print('%s\\t%s\\t%s\\t%s\\t%s\\t%s\\t%s\\t%s' % (token.text, token.lemma_, token.pos_, token.tag_, token.dep_,\n",
    "          token.shape_, token.is_alpha, token.is_stop))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Harry PROPN\n",
      "Potter PROPN\n",
      "is AUX\n",
      "a DET\n",
      "series NOUN\n",
      "of ADP\n",
      "fantasy NOUN\n",
      "novels NOUN\n",
      "written VERB\n",
      "by ADP\n",
      "British ADJ\n",
      "author NOUN\n",
      "J. PROPN\n",
      "K. PROPN\n",
      "Rowling PROPN\n",
      ". PUNCT\n"
     ]
    }
   ],
   "source": [
    "for token in doc:\n",
    "    print(token, token.pos_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "def explain_pos(text):\n",
    "    doc = nlp(text)\n",
    "    for word in doc:\n",
    "        print(f'{word.text:{12}} {word.pos_:{10}} {word.tag_:{8}} {spacy.explain(word.tag_)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Harry        PROPN      NNP      noun, proper singular\n",
      "Potter       PROPN      NNP      noun, proper singular\n",
      "is           AUX        VBZ      verb, 3rd person singular present\n",
      "a            DET        DT       determiner\n",
      "series       NOUN       NN       noun, singular or mass\n",
      "of           ADP        IN       conjunction, subordinating or preposition\n",
      "fantasy      NOUN       NN       noun, singular or mass\n",
      "novels       NOUN       NNS      noun, plural\n",
      "written      VERB       VBN      verb, past participle\n",
      "by           ADP        IN       conjunction, subordinating or preposition\n",
      "British      ADJ        JJ       adjective\n",
      "author       NOUN       NN       noun, singular or mass\n",
      "J.           PROPN      NNP      noun, proper singular\n",
      "K.           PROPN      NNP      noun, proper singular\n",
      "Rowling      PROPN      NNP      noun, proper singular\n",
      ".            PUNCT      .        punctuation mark, sentence closer\n"
     ]
    }
   ],
   "source": [
    "explain_pos(\"Harry Potter is a series of fantasy novels written by British author J. K. Rowling.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Why POS Tagging is Useful?**\n",
    "\n",
    "POS tagging can be really useful, particularly if you have words or tokens that can have multiple POS tags. For instance, the word \"google\" can be used as both a noun and verb, depending upon the context. While processing natural language, it is important to identify this difference. Fortunately, the spaCy library comes pre-built with machine learning algorithms that, depending upon the context (surrounding words), it is capable of returning the correct POS tag for the word."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Can          VERB       MD       verb, modal auxiliary\n",
      "you          PRON       PRP      pronoun, personal\n",
      "google       VERB       VB       verb, base form\n",
      "it           PRON       PRP      pronoun, personal\n",
      "?            PUNCT      .        punctuation mark, sentence closer\n"
     ]
    }
   ],
   "source": [
    "explain_pos(u'Can you google it?')\n",
    "#From the output, you can see that the word \"google\" has been correctly identified as a verb."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Can          VERB       MD       verb, modal auxiliary\n",
      "you          PRON       PRP      pronoun, personal\n",
      "search       VERB       VB       verb, base form\n",
      "it           PRON       PRP      pronoun, personal\n",
      "on           ADP        IN       conjunction, subordinating or preposition\n",
      "google       PROPN      NNP      noun, proper singular\n",
      "?            PUNCT      .        punctuation mark, sentence closer\n"
     ]
    }
   ],
   "source": [
    "explain_pos(u'Can you search it on google?')\n",
    "#Here in the above script the word \"google\" is being used as a noun as shown by the output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xml:lang=\"en\" id=\"9e9f8a21c9924a7c910c06df311c1b32-0\" class=\"displacy\" width=\"1325\" height=\"307.0\" direction=\"ltr\" style=\"max-width: none; height: 307.0px; color: #000000; background: #ffffff; font-family: Arial; direction: ltr\">\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"50\">Harry</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"50\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"135\">Potter</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"135\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"220\">is</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"220\">AUX</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"305\">a</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"305\">DET</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"390\">series</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"390\">NOUN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"475\">of</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"475\">ADP</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"560\">fantasy</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"560\">NOUN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"645\">novels</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"645\">NOUN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"730\">written</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"730\">VERB</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"815\">by</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"815\">ADP</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"900\">British</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"900\">ADJ</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"985\">author</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"985\">NOUN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"1070\">J.</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"1070\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"1155\">K.</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"1155\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"217.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"1240\">Rowling.</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"1240\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-0\" stroke-width=\"2px\" d=\"M70,172.0 C70,129.5 120.0,129.5 120.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-0\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">compound</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M70,174.0 L62,162.0 78,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-1\" stroke-width=\"2px\" d=\"M155,172.0 C155,129.5 205.0,129.5 205.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-1\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">nsubj</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M155,174.0 L147,162.0 163,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-2\" stroke-width=\"2px\" d=\"M325,172.0 C325,129.5 375.0,129.5 375.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-2\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">det</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M325,174.0 L317,162.0 333,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-3\" stroke-width=\"2px\" d=\"M240,172.0 C240,87.0 380.0,87.0 380.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-3\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">attr</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M380.0,174.0 L388.0,162.0 372.0,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-4\" stroke-width=\"2px\" d=\"M410,172.0 C410,129.5 460.0,129.5 460.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-4\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">prep</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M460.0,174.0 L468.0,162.0 452.0,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-5\" stroke-width=\"2px\" d=\"M580,172.0 C580,129.5 630.0,129.5 630.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-5\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">compound</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M580,174.0 L572,162.0 588,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-6\" stroke-width=\"2px\" d=\"M495,172.0 C495,87.0 635.0,87.0 635.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-6\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">pobj</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M635.0,174.0 L643.0,162.0 627.0,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-7\" stroke-width=\"2px\" d=\"M665,172.0 C665,129.5 715.0,129.5 715.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-7\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">acl</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M715.0,174.0 L723.0,162.0 707.0,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-8\" stroke-width=\"2px\" d=\"M750,172.0 C750,129.5 800.0,129.5 800.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-8\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">agent</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M800.0,174.0 L808.0,162.0 792.0,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-9\" stroke-width=\"2px\" d=\"M920,172.0 C920,129.5 970.0,129.5 970.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-9\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">amod</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M920,174.0 L912,162.0 928,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-10\" stroke-width=\"2px\" d=\"M1005,172.0 C1005,44.5 1235.0,44.5 1235.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-10\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">compound</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M1005,174.0 L997,162.0 1013,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-11\" stroke-width=\"2px\" d=\"M1090,172.0 C1090,87.0 1230.0,87.0 1230.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-11\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">compound</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M1090,174.0 L1082,162.0 1098,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-12\" stroke-width=\"2px\" d=\"M1175,172.0 C1175,129.5 1225.0,129.5 1225.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-12\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">compound</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M1175,174.0 L1167,162.0 1183,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-9e9f8a21c9924a7c910c06df311c1b32-0-13\" stroke-width=\"2px\" d=\"M835,172.0 C835,2.0 1240.0,2.0 1240.0,172.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-9e9f8a21c9924a7c910c06df311c1b32-0-13\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">pobj</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M1240.0,174.0 L1248.0,162.0 1232.0,162.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sen = nlp(\"Harry Potter is a series of fantasy novels written by British author J. K. Rowling.\")\n",
    "displacy.render(sen, style='dep', jupyter=True, options={'distance': 85})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Chunking\n",
    "\n",
    "https://spacy.io/usage/linguistic-features#noun-chunks\n",
    "\n",
    "We need noun chunks. Noun chunks are noun phrases - not a single word, but a short phrase which describes the noun. For example, \"the blue skies\" or \"the world’s largest conglomerate\".\n",
    "\n",
    "eg: In the following code snippet, “Tall big tree is in the vast garden” → The words “tall” and “big” describe the noun “tree”, and “vast” describes the noun “garden”.\n",
    "\n",
    "To get the noun chunks in a document, simply iterate over doc.noun_chunks:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall Street Journal \t NP \t Journal\n",
      "an interesting piece \t NP \t piece\n",
      "crypto currencies \t NP \t currencies\n"
     ]
    }
   ],
   "source": [
    "doc = nlp(\"Wall Street Journal just published an interesting piece on crypto currencies\")\n",
    "for chunk in doc.noun_chunks:\n",
    "    print(chunk.text, '\\t', chunk.label_, '\\t', chunk.root.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    text → text: The original noun chunk text.\n",
    "    root.text → root text: The original text of the word connecting the noun chunk to the rest of the parse.\n",
    "    root.dep_ → root dep: Dependency relation connecting the root to its head.\n",
    "    root.head.text → root head text: The text of the root token’s head."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Break!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets use our NLP skills to do a simple exploration!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "What does Trump talk about?\n",
    "\n",
    "It might be interesting to explore what does Trump even talk about? Is it always them 'Angry Dems'? Or is he a narcissist with too many mentions of The President and the USA?\n",
    "\n",
    "One way to explore this would be to mine out all the entities and noun chunks from all his tweets! Let's go ahead and do that with amazing ease using spaCy\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/envs/aie/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3063: DtypeWarning: Columns (0,1,2,3,4,6) have mixed types.Specify dtype option on import or set low_memory=False.\n",
      "  interactivity=interactivity, compiler=compiler, result=result)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEkCAYAAADTtG33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd1xTZ///8VcY4gIHIuKeBItaFygOsK6667jVKlpb72qHtdpWW+3w1tu2jk6r9utoraPi3tviTR0tggMXouJWXIgCRmTm/P7gl9RIwAQTEvDzfDx8FM7K55ykeXPOdc51qRRFURBCCCFM4GDrAoQQQhQeEhpCCCFMJqEhhBDCZBIaQgghTCahIYQQwmQSGkIIIUzmZOsCRLb79x+i1crdz9bi7l6ahASNrcsosuT4Wld+jq+Dg4py5UpZvBYJDTuh1SoSGlYmx9e65Phal70cX5U83CeEEPYvNS2DB8mpJi/v4KDC3b20xeuQM41n1L59e+Li4li6dCktWrTI93ben7aRu/cfWrAyIURREjIzmAeYHhrWIg3hQgghTCahIYQQwmQSGkIIIUwmoSGEEMJkEhpCCCFMJqEhhBDCZBIaQgghTCahIYQQwmQSGkIIIUwmoSGEEMJkEhpCCCFMJqEhhBDCZBIaQgghTCahIYQQwmQSGkIIIUwmgzAJIUQhIIMwCQMJCRq7Gc6xKPLwcCU+/oGtyyiy5Phal4eHq1mBYU1yeUoIIYTJJDSEEEKYTC5P2QlrXHsUhjw8XG1dQpGlzcywdQmigEho2ImT8z4hPTnB1mUIkS/NPv4FsI9r7sK65PKUEEIIk0loCCGEMJmEhhBCCJNJaAghhDBZoQuN69evU79+ffz9/UlNNd7wlpGRQZs2bVCr1cTGxuqnx8XFMXnyZDp06ECDBg3w8/Nj6NChbNmyxeh2JkyYgFqtZv369Ubnz549G7VazezZs599x4QQohAodKFRtWpVXnrpJZKSkti6davRZXbv3k18fDz+/v7Uq1cPgGPHjtG7d29WrFgBQKdOnWjYsCFRUVGMGzeOjz/+GOlRRQgh8lboQgNg6NChAPoAeFJISAgAwcHBAKSlpTF27FiSk5MZNmwYu3fv5ocffmDRokWsX78ed3d3Nm3axKpVqwpmB4QQopAqlKEREBBA3bp1OXXqFCdOnDCYd/bsWQ4fPkzFihXp2LEjADt27ODmzZtUqVKF8ePH4+joqF/e29ub0aNHA7Bo0aKC2wkhhCiECmVowD9nEbqzCh3d7wMHDsTJKfvZxUOHDgHQs2dPnJ2dc2yrb9++qFQqrly5wu3bt61ZthBCFGqFNjR69+6Nq6sr27dvJzExEQCNRsPmzZtxdnZmwIAB+mV1QVC1alWj23JxcaFixYoGywohhMip0IZGyZIl6devH2lpaaxbtw6AjRs3kpKSQocOHfQhYE1ardbqryGEEPak0IYGZF+icnBwYOXKlWi1Wn3DuO7SlY6npyeQfbuuMWlpady5c8dgWUB/KSslJcXoejdu3Hi2HRBCiEKmUIdG9erVCQwM5OrVq3z//fecP3+eevXq4e/vb7Ccn58fAFu3biUzMzPHdjZs2ICiKNSoUcMgNHQ/X7x4Mcc6qampREZGWnJ3hBDC7hXq0AAYMmQIAAsXLgRg0KBBOZbp2rUrXl5eXL9+ne+++87gstL58+f1D+cNHz7cYL2WLVsCsGnTJoPgSE1NZfLkyXKmIYR47hT6rtHbtGlDrVq1uHTpEqVKleKVV17JsYyLiws//vgjI0aMYNGiRYSGhtKwYUOSkpKIiIggIyODV155hYEDBxqs17x5c1566SXCwsLo27cvzZo1w8nJiVOnTqFSqejbt2+uT4sLIURRVOjPNFQqFa1atQKy76gqXdr4YEaNGzdm48aNvPrqq2RlZbF7926OHz9O48aN+eabb5gxYwYqlSrHerNmzWLkyJGUL1+eiIgIoqOjCQoKYsOGDVSuXNmq+yaEEPZGpRTyvjPS09N56aWXuHv3Ltu2baNu3bq2LkmI5442M4OE+zIIk7V4eLgSH//ArHUcHFRWGRG00F+eCgkJ4e7du7Rt27ZQB0ZCggattlDnt13Lz/90wnTZQ+lKaDwPzAoNjUbDuXPnKFmyJD4+Pgbz7t+/zzfffMOhQ4dIT0+nbdu2fPjhh5QvX96iBUP23Uy//vord+7c4cCBAzg7OzNu3DiLv44QQghDZrVprFy5kuDgYNauXWswPT09ncGDB7NhwwauXbvG7du3WbduHcOGDSMtLc2iBQPEx8ezdu1aDh48iI+PD3Pnzs0RYkIIISzPrDONAwcOANCrVy+D6Zs2beLSpUu4urry7rvv4uLiws8//8z58+cJCQnhjTfesFzFQIsWLTh79qxFt2lr1rj2KAxlX0Kxf2np6SQnWf6PLSEswazQuHr1KoB+jAqdHTt2oFKp+OCDDxg8eDAAVapU4a233mLXrl0WD42iaNyaKdzV3LN1GcIOLH5jFiChIeyTWZen7t+/j6urKyVKlNBPy8rK4ujRo6hUKrp166af3rZtWxwdHY0+TV2YtG/fHrVanWsXJEII8TwxKzS0Wm2OIVZPnz5Namoq3t7elC1b9p8NOzjg6uqaa79NhdnThoEVQoiiyqzLU56enly7do3z58/rb2/ds2cPkP309OMURUGj0Vjl7qmCtHjxYjIyMgz6pBJCiOeVWWca/v7+KIrC1KlTuXDhAuHh4YSEhKBSqQgKCjJY9sKFC2RmZhb6L9vq1atTp04do4M3CSHE88as0HjzzTcpUaIEkZGR9OjRg+HDh5OcnIyvry9t27Y1WDYsLAyVSkXjxo0tWnBBe7xN4/r166jVajZs2ADAxIkTUavV+n9yuUoIUdSZdXmqZs2a/PLLL0ybNo3o6GhKlChBYGAgn3/+ucFyiqKwZs0aFEXR9wtVFJQsWZI+ffpw5MgRrl69StOmTalRo4Z+fvXq1W1YnRBCWJ/Z3Yg0a9aMtWvXkpWVhaOjo9FlFEVh2bJlAFSoUOHZKrQj5cuXZ/r06UyYMIGrV6/Sv39/+vbta+uyhBCiwOS776ncAgOy75wq7G0ZQgghcjKrTWPSpEmcOHHCWrUIIYSwc2aFxurVqxk4cCA9e/Zk8eLF3LsnTzALIcTzxKzQ6NSpE46OjsTGxjJjxgwCAwN5//332bt3r8EQqkIIIYoms9o0Zs+ezf3799m0aRMbNmzg7Nmz7N69mz/++IMKFSrQp08f+vbtS82aNa1UrhBCCFsye7jXcuXK8frrr7Np0ybWrVvHoEGDcHNzIz4+noULF9K1a1eCg4PZsGEDjx49skbNNqd70C8rK8vGlQghRMF6pjHCfX19+c9//sP+/fv57rvvCAgIQKVSceTIET799FPatGnD559/ztGjRy1Vr13Q3Rl24cIFG1cihBAFyyLDvRYrVozu3bvTvXt3rl+/zscff8zRo0dJSUlh3bp1rFu3jrp16/Laa6/Rt2/fPG/XLQw6dOjA3LlzWbJkCbGxsXh6eqJSqejXrx9Nmza1dXlCCGE1KkVRLDIw9aVLl1i/fj2bNm0iPj4eRVFwdHSkUaNGnD59mrS0NFQqFT4+PixYsAAPDw9LvKzVtW/fnri4OPbs2UPVqlX103fu3MmiRYuIjY3V9+Q7bdo0edhPPLPCOAiTjMFuXfk5vg4OKqsM7vZMofHw4UO2b9/OunXrOH78OJD9NHi1atXo168fffv2pWLFijx48ID169ezYMEC7t27R48ePfjmm28sthNFQUKCBq3WIvktjJAvNeuS42td9hQa+bo8FRERwfr169m9ezepqakoikKxYsXo2LEj/fv3JyAgwGB5V1dXhg0bRmBgIN27d+evv/6ySPFCCCEKllmhMWfOHDZu3EhcXBy6E5R69erRr18/evfubTAIkzG1atXCw8ODO3fu5L9iIYQQNmN2aACUKFGCbt260b9/f7O7Pq9fv36R6sRQCCGeJ2a1afTv35/+/fvTvXt3SpUqZc26hCh0MlLTSHyQbusybELaNKyr0LZprFmzxuIFiGxhH47j0d0EW5chnkG3pb/Bcxoa4vnxTA/3CSGEeL4808N9Go2G+Ph4Hj16RF5XuXx9fZ/lZYQQQtiJfIXGypUrCQkJ4fz583mGBYBKpeL06dP5Kk4IIYR9Mfvy1NixY5kyZQrnzp3DyclJHxqurq6oVCoURUFRFJydnSlTpgxubm4WL9oUWVlZ+Pn54evri0ajMZi3Z88e1Go1arWavXv3GszTaDT4+vri5+dn0N17SkoKCxcu1HcV0qhRI7p3787s2bN5+PBhgeyTEELYmlmhsXHjRnbu3EnZsmVZuHChviNCd3d3IiMjiYqKYv78+fj6+uLk5MTkyZM5ePCgVQp/GkdHR/z9/cnMzCQyMtJgXnh4uNGfASIjI8nMzKRFixY4OGQfnlu3btG/f3++/fZbbty4QZMmTWjdujVJSUnMmTOHQYMGkZSUZP2dEkIIGzM7NFQqFePHj6dt27Y4ORle3XJxcSEoKIiVK1fSoEEDxo8fT3R0tEULNofuyfQng+HgwYNUqFABd3d3/v77b4N5umV16yqKwtixYzl//jxDhgwhLCyMX3/9lf/7v/8jNDSUXr16cfbsWaZNm1YAeySEELZlVmicOXMGgK5duxpMf3LUPmdnZz777DMyMjL45ZdfnrHE/DMWGvHx8cTGxtKyZUtatmzJuXPnSEj451ZX3ZlRy5YtAdi3bx9RUVE0btyYzz77jOLFi+uXLV68OFOmTMHd3Z0tW7bI2YYQosgzKzQePnyIq6srJUqU0E8rVqyYvpfXx6nVakqVKsWRI0eevcp8qlOnDhUrViQ2Npb4+Hjgn1AICAggICAARVH0oXL37l3OnTuHp6cnderUAbJDA6Bz5876y1WPK1myJA0aNCAzM5OTJ08WxG4JIYTNmBUa7u7uZGZmGkwrV64caWlp+i9lHUVRSE9P5/79+89e5TN48mxD999WrVrRqlUrAP0lqscDRefatWsAzJw5U994/uQ/XWP6vXv3CmCPhBDCdsy65bZSpUrcvn2be/fuUb58eSC7L6k7d+4QFhbGgAED9MseOHCAjIwMKlasaNmKzdSqVSs2bdpEeHg4vXr14uDBg9SoUYPKlSsDUL16dX1YPNmeAf8M6erv70+VKlXyfC3dNoUQoqgyKzSaNGnC8ePHOXz4MJ07dwage/fu/Pnnn8ycOZPMzEzq16/PuXPn+PHHH1GpVLRr184adZtMFwAHDx7k6tWrxMXF8eqrrxrMX7VqFZcvXzYaGl5eXgB06dKF4ODgAqxcCCHsj1mXp7p27YqzszPbtm3TT+vZsyetW7dGo9EwdepUBg8ezOTJk7l//z5eXl68//77Fi/aHJ6entSqVYsbN26wYsUKwDAUdD+vWrWKuLg4ateurR8DHCAwMBDIHqlPCCGed2aFRqNGjThx4gSzZs0ymD5v3jzGjRtHgwYNqFChArVq1WLIkCGsWbPGLrpB17VdLF++HAcHB/2dUZB9l5RKpWL58uUAOQaQ6tixI76+vkRGRjJp0iQSExNzbD8+Pp7Vq1dbcQ+EEMI+PFPfUzrOzs68+eabvPnmm5bYnMUFBASwfPly0tLS8PX1NRgsqly5ctSvX1/f1YkuYHQcHBz4+eefGTFiBKtWrWLr1q34+Pjg5eVFWloaly9f5vz587i7uxu06QghRFFkkdCwd7qnu7VabY4zCcgOldOnT+Pg4IC/v3+O+ZUqVWLt2rWsXbuWHTt2cO7cOU6cOEHZsmWpWLEib7zxBp06dSqIXRFCCJsyaxCmvKSmphIXF4ezszNVqlTB0dHREpsVotCQQZhkECZrKTSDMGm1WuLi4oDs20mNBcGDBw/48ssv2b59u/4ZDjc3NwYNGsTo0aMlPEyUkKBBq7VIfgsj5EtNCMvIsyH84MGDdO7cmWHDhhl9Gjo9PZ3XXnuNzZs3k5GRoe/hNikpifnz5zNx4kSrFS6EEKLg5RkaERERKIpC7969UalUOeYvXbqUmJgYIPsZjqlTp/Ltt9/Srl07FEVhy5YtOXqYFUIIUXjleXkqKioqzwf0Vq9ejUqlokGDBoSEhOiDpUePHrzzzjuEhYWxceNGo43LQgghCp88Q+PWrVs4OjpSv379HPOuX7/O1atXUalUvPXWWznORN5++23CwsI4ceKEZSsuoqzRYCUMeXi4Gp2enpZJUvKjAq5GiMIpz9CIj4+ndOnSODs755h37Nix7A04OdG6desc81944QVUKhU3b960UKl5u3LlCp07d8bR0ZHIyEhKl875JbxgwQK+++47IPss6cUXX8yxTFhYGG+//TYeHh4cOHDA6nXrzP1mO0mJOXsLFtb36Vf/snUJQhQaebZpaLXaHEOl6ui6Aa9bt65BV+k6zs7OuLm5kZqaaoEyn65GjRpUqlSJrKwsDh8+bHSZiIgIoz8/TtcGI5fUhBAipzxDw93dnaysLK5evZpj3tGjR1GpVDRs2DDX9VNSUowGirX4+fkBxgMhMzOTo0ePUq9ePZydnXNtoNdNb9GihfUKFUKIQirP0GjUqBGAvqM/nYsXL3Lq1CkgZ19NOlevXiUjI6NAuwvXfdEbC4To6GhSUlJo27YtDRo04OjRoznGBtFoNPq7weRMQwghcsqzTaNv377s3LmTpUuXUqxYMYKCgrh9+zazZs1CURTKlCnDSy+9ZHRd3RgVPj4+lq86F7rQiImJQaPRGLRr6ILEz88PJycnoqKiiI6ONmjXOHLkCFlZWVSsWJFatWoZbPvGjRssWrSI/fv3c/PmTZycnFCr1QwcOJDevXsXwN4JIYTt5XmmERgYSLdu3cjKymLBggUEBwfz4YcfcvnyZVQqFWPGjDEYM/txmzZtQqVS6S8ZFYTq1avj5eVltF0jMjISBwcHmjdvnutlLN3vT55lhIeH07NnT5YtWwZA27ZtadSoEWfOnOGTTz6RhxiFEM+Np3aNPnPmTN566y1cXV31T3yXL1+ezz77jMGDBxtd59ixYxw5cgQHBwfat29v8aLzoguExy9RZWVlcfToUdRqNW5ubjRt2hRHR0cOHTpksK7u98e7Tr916xajR48mNTWVmTNnsmvXLubOncvixYvZtm0b3t7erF+/nk2bNhXA3gkhhG09NTScnJz44IMPCA8PJzQ0lNDQUP766y+GDBmS6zpqtZoDBw6wf/9+3N3dLVrw0xhr14iOjkaj0dC8eXMASpcuTf369fWXoyC7PUPXPfrjZxqLFy/mwYMHvPnmm7zyyisGr1W5cmWmTp0KwO+//269nRJCCDth8iBMjo6OVK1alapVqxrtUuRxJUqUoEKFCvpxxAuSLjROnz6tv13Y2G20fn5+PHz4kOjoaCC7PSMzM5NKlSpRo0YN/XJ79+4Fsod7NaZRo0YUL16c6OhoMjIyLL9DQghhR8waua8wqFatGpUrVyYrK4sjR44A2ZedVCqV/kwDct6eq7s09WR7xvXr1wHo3bs3arU6x7/69euTmppKVlYWSUlJVt8/IYSwpXwPwnTmzBl27drF6dOnuXfvHgDly5fnhRdeoEuXLqjVaosVaS4/Pz82bdpEREQEbdq04ciRI9StW9fgzKd58+aoVCoiIyMZMWJErs9naLVaALp3706xYsXyfF1jT84LIURRYnZoJCYmMmnSJP744w8AnhzDad++fcybN49OnTrx3//+12Bo1YLSokULNm3aRGRkJDExMTx48IAePXoYLFOmTBm8vb05cuQIDx480F+mejI0PD09iYuL47333qN27doFtg9CCGGPzAqNlJQUhg4dyvnz51EUBbVajZ+fH56engDcuXOHQ4cOcebMGf744w8uX77MqlWrCvSpcPjnEtPp06cJCwszmPY4Pz8/zp49y9KlS8nMzKRy5cpUq1bNYJnAwEBWrFjBzp07effdd61fvBBC2DGzQmP+/PnExsbi5ubGzJkzc+0yfe/evYwfP57Y2FjmzZvHBx98YIlaTVatWjWqVKlCXFyc/tkKY8+L+Pn58fvvv7NkyRLAeLC8+eabbN68mZ9//ply5crRv39/nJwMD9vZs2e5du0aHTt2tMLeCCGE/TCrIXzHjh2oVCqmTp2aa2AABAUFMXXqVBRFYceOHc9aY77oQiIpKYmaNWvi4eGR5zJgvL+pqlWrMmfOHEqUKMHkyZNp164dw4cPZ9y4cYwYMYKgoCB69erFzp07rbg3QghhH8wKjVu3blGsWDE6d+781GU7deqEi4sLt27dyndxz+LxAMjtqXR3d3eDdorc+ptq1aoV27ZtY+TIkbi7uxMVFcWuXbu4cOEC1atXZ9y4cYwZM8ayOyCEEHZIpTzZkp2HwMBAUlJScu16/EnNmjWjVKlS7Nu3L98FCmFtMgjTs/PwcCU+/oGtyyiy8nN8HRxUVhnczaw2jYCAADZv3sypU6do0KBBnsueOnWKhw8f0qlTp2cq8HmRkKBBqzU5v4WZ5EtNCMsw6/LUe++9h5ubGxMnTuTOnTu5LhcfH8/EiRMpU6YMo0aNeuYihRBC2AezzjSSk5OZMGECX3/9NV27dqVPnz45brmNjIxk48aNODg48Omnn5KcnKx/BuJxvr6+ltkDIYQQBcasNg0fHx99v1OKouTaB1Ve8wBUKpW+c0AhClJ6WhpJyem2LqPIkct/1lVo2zTKlCnz1M4KRf78MuNTkhMTbF1GkffhtPmAhIYQ+WVWaBgbe9sW1q9fz8SJE+nTpw/Tp0+3dTlCCPHcsNtebtu3b49ardb3MiuEEML27DY0hBBC2B8JDSGEECYzu2t0RVHYvHkzu3btIjY2luTk5DxHrFOpVPrBkEyha6/Q6dChg8H8PXv2GPyu0WiYO3cuu3bt4s6dO5QrV4727dvzwQcf5Not+4ULF/j11185ePAg8fHxFC9eHF9fX4YOHWrweg8fPiQwMJBHjx7xv//9j0qVKhndXt++fYmOjmbBggUEBQWZvK9CCFHYmBUaGo2GkSNHEhUVlWMcjdyYe7dV9erV6dOnD7t27SIlJYWXX36ZkiVL6uc//vODBw8YNGgQd+7coXnz5vrxMVauXMnJkydZtWpVjoGRtm3bxieffEJGRgb16tXjpZde4t69exw+fJjw8HDeffddfT9SpUqVom/fvixdupRVq1YZ7V/q2LFjREdHU61aNdq2bWvWvgohRGFjVmj88MMPHD16lOLFizNkyBACAgJwd3fH0dHRYgU1b96c5s2bExkZSUpKCh9//DFVq1Y1umxoaChBQUGsXLmSUqVKAXD79m0GDhxIdHQ0O3bsoFevXvrlz5w5wyeffIKzszNz5841OCuIjY1lxIgR/Pzzz7Ro0YKWLVsCEBwczLJly1izZg3vvvtujhAKCQkBYNCgQTg4yNU+IUTRZta33K5du1CpVEyfPp1x48bRunVrfHx8qFevXp7/rKVkyZJ89dVX+sCA7JH2goODAQgPDzdYft68eWRkZDB+/Pgcl5Hq1avHhAkTAFi+fLl+es2aNWnbti3x8fH60Qp17t27x44dO3BxcaFfv34W3TchhLBHZoXGgwcPcHFxMalr9ILQoEEDo+Nk6Lo7f7x/LK1Wy/79+1GpVHTp0sXo9nRdo0dFRRlMHzJkCAArVqwwmL527VrS09Pp3r27TYa1FUKIgmbW5akqVapw8+ZNu7kM4+XlZXR66dLZj86np//z5G9iYiIajQbI7q03L/fv3zf4PTAwkJo1axIZGcn58+epW7cuWq2WlStXAjB48OB874MQQhQmZoVGjx49mD17NuHh4U/94i0I5oRXVlYWAI6OjgbtHKZQqVQEBwfz1VdfERISwqRJk9i7dy9xcXE0bNiQhg0bmrU9IYQorMwKjX//+9/s3buXzz77jDlz5vDCCy9Yqy6LK1euHMWLFyc1NZUvvvjCoB3EFH379uWHH35g06ZNfPTRR/oGcF37iRBCPA/MCg0XFxd+++03vv76a/71r3/RqlUr6tWrR8WKFfNc74033jC7MN1dSrozhGfl5OREQEAAYWFh7Nq1i759+5q1funSpenTpw/Lly9n7ty5HDhwgLJly9KtWzeL1CeEEIWB2Q/3HTp0iEOHDqHVavnrr7/466+/cl1W10V6fkKjYsWKXL58mQsXLlCjRg2z1zdm1KhRHDhwgK+++goXFxe6detm8ByJoiicPHmS5ORk2rRpk2P94OBgQkJC+PXXXwHo168fLi4uFqlNCCEKA7NCIzw8nFGjRpGVlUWxYsXw9fXF3d3dKg3jnTp1IjIyknHjxtGmTRtcXV0BGDduXL632bBhQ2bMmMGnn37Khx9+yHfffUedOnUoU6YM9+/fJyYmhoSEBEaMGGE0NOrUqUPr1q05cOAADg4ODBo0KN+1CCFEYWRWaMydO5fMzEzatGnD999/j5ubm7XqYsiQIWg0GrZs2UJYWJj+Tqh33nnnmbbbvXt3GjZsyNKlS/n77785dOgQABUqVKB+/foEBQXx8ssv57p+q1atOHDgAIGBgVSrVu2ZahFCiMLGrJH7mjVrRkpKCvv376dChQrWrMtu9e7dm5iYGOlnqpCSkfusQ0bus65CO3Kfs7Mzrq6uz21g/PHHH8TExFCnTh0CAwMtuu2EBA1arcn5LcwkX2pCWIZZodGoUSMOHDhAYmLic/ME9P379/n2229JSkpi7969AHz88ccy7K0Q4rlkVgv2yJEjUalUzJkzx1r12J2HDx+ydu1awsLCqFq1KjNmzKBdu3a2LksIIWzCrDYNgO3bt/PZZ58REBDAyJEj8fb2NuiuXAhby0zP5H7SI4NpcnnKuuT4WlehbdNo2rQpABkZGYSFhREWFgZAiRIlcl3H3EGYnlenlx0k40GqrcsoEl58t52tSxCiyDIrNFJSUsyaDuYPwmRNarUagLNnz9q4EiGEKJzMCo0tW7ZYqw4hhBCFgFmhYc0BlYQQQtg/+xgYQwghRKFg96GRlZWFn58fvr6++kGUdPbs2YNarUatVuufodDRaDT4+vri5+eHVqvNsd3t27czcOBAmjRpQpMmTRg2bBiHDx/OtY64uDgmT55Mh4TehOEAABucSURBVA4daNCgAX5+fgwdOlQu2QkhnitmXZ768ssv8/Uin3/+eb7Wg+xBk/z9/QkNDSUyMpL27dvr5z0+Bnh4eLhBtx6RkZFkZmbSokWLHB0qzpo1i3nz5tGsWTOCgoI4e/YsBw8e5MiRIyxbtowmTZoYLH/s2DFGjBhBcnIyVatWpVOnTiQlJREZGUlkZCT79+9nxowZdtXoL4QQ1mBWaPz++++oVCpye7TjyS9NXdfozxIakD08a2hoKOHh4QahcfDgQSpUqICiKPz9998G6+gCxdgIgyEhIaxZs4YGDRoA2eOH/+c//2H16tX89NNP/Pbbb/pl09LSGDt2LMnJyQwbNoxPPvkER0dHAM6dO8frr7/Opk2baNq0Ka+++uoz7acQQtg7s0LjaeNiaDQaTp48yZkzZyhTpgx9+vSxyF/fui/+x88s4uPjiY2NpUePHiiKwvbt20lISMDd3R3IDhSAli1b5tje6NGj9YEB2cPGjhkzhtWrV3P48GEyMjL0g0Dt2LGDmzdvUqVKFcaPH68PDABvb29Gjx7N5MmTWbRokYSGEKLIMys0PvnkE5OW27dvHx9++CE3btzgp59+yldhj6tTpw4VK1YkNjaW+Ph4PDw89KEQEBCAoihs27aN8PBwevTowd27dzl37hyenp7UqVMnx/aMdQNSoUIFypQpQ1JSEomJiXh4eADou07v2bOnPkge17dvX6ZMmcKVK1e4ffs2np6ez7y/Qghhr6zSEB4YGMgXX3zBH3/8wfLlyy2yzSfPNnT/bdWqFa1atQLQX6J6PFCMqVy5stHppUtnP3Kflpamn3b79m0AqlatanQdFxcX/XC3umWFEKKostrdU127dsXR0ZE1a9ZYZHu6YNCFxcGDB6lRowaVK1emSpUqVK9eXR8WebVnAFYZaVAIIZ4HVvv2LFasGC4uLly5csUi29MFwMGDB7l69SpxcXEGoRAQEEBcXByXL19+amiYQ3e56fr160bnp6WlcefOHYNlhRCiqLJaaFy8eJGHDx8abQfID09PT2rVqsWNGzdYsWIFQI7QAFi1ahVxcXHUrl3bIl/ifn5+AGzdupXMzMwc8zds2ICiKNSoUUNCQwhR5FklNC5dusT48eNRqVQ0btzYYtvVXaJavnw5Dg4OBndGtWzZEpVKpW9DscRZBmRfZvPy8uL69et89913Bg8Knj9/ntmzZwMwfPhwi7yeEELYM7Punnr77bfznJ+ens6tW7e4fPkyWq2WYsWK8d577z1TgY8LCAhg+fLlpKWl4evrazB6YLly5ahfvz6nT58G/gmYZ+Xi4sKPP/7IiBEjWLRoEaGhoTRs2JCkpCQiIiLIyMjglVdeYeDAgRZ5PSGEsGdmhcaff/5p8rJ169Zl0qRJNGrUyNyacqV7ulur1Ro9kwgICOD06dM4ODjg7+9vsddt3LgxGzduZMGCBezfv5/du3dTvHhxGjduzIABA+jZs+czP4/ywtCcz5OI/MlMz3kZUQhhGWaN3Pe022cdHR1xc3PD29ubunXrPnNxz5OEBA1arVmDKAozyMhy1iXH17oK7ch9wcHBFi9ACCFE4WFWaMyaNQuAAQMG4OXlZZWChBBC2C+zQmP+/Pk4ODhYtHFbCCFE4WFWaJQvX56MjAyDTvuEZVjj2mNRkZ6eQVJSqq3LEEJgZmg0bNiQP//8Uzrms4KFCxeSnJxs6zLs0kcffQRIaAhhD8x6uG/48OGoVCp++OEHa9UjhBDCjpkVGn5+fnz55Zfs2LGDESNGEB4ezqNHj6xVmxBCCDtj1uWppk2bAtnjdh84cIADBw4A2U9N59ZzrEql4siRI89YphBCCHtgVmikpKQYnZ6amvv15oIcN1utVgNw9uxZ1q9fz++//87FixcpVaoUQUFBjBs3jvLly5OWlsb8+fPZtm0bN27cwN3dnVdeeYX33nsv1w4W9+/fz/Llyzlx4gTJycmULVsWf39/3nrrLf3rCiFEUWdWaGzZssVadVjUN998w5IlS/D396dt27ZERUWxbt06Tp06xYoVK/j3v//NxYsX8fPzo3r16hw6dIh58+Zx7949pk6dmmN7X375JcuWLcPJyYmGDRvi6enJ1atX2bZtG6GhocyePZugoCAb7KkQQhQss0KjXr161qrDojZu3MimTZv0Q70mJSUxcOBAzp49y6uvvoqrqyt79uzB1dUVgJiYGP71r3+xZs0a3n77bapUqaLf1ooVK1i2bBn16tVj1qxZBsPHhoaGMmbMGMaNG0doaChlypQp2B0VQogClmdD+Guvvcb7779fULVYzPvvv2/w5V6mTBleffVVILs786lTp+oDA6B+/foEBgaiKIp+THDIbruZO3cuAD/++GOO8cY7duzIwIEDSU5OZvPmzdbcJSGEsAt5hkZkZCRHjx4tqFospm3btjmm1ahRA8geH/zJL3+AmjVrAuhH4YPsM5D4+Hjq1auXaweMukGajh079qxlCyGE3TPr8lRhUalSpRzTSpYsmeu8x+enpaXpp127dg2A2NjYpzZ237t3L1+1CiFEYVIkQyO323+fNu9JulH6PD09nzqoU+3atU3erhBCFFZFMjQsRXdW4uHhwfTp021cjRBC2J5VxggvKho1akTZsmWJiYnhypUrti5HCCFs7qlnGhqNhokTJ+b7BVQqFV9//XW+17clZ2dn3n33Xb7++mtGjRrF119/nWP42vT0dPbv30/NmjWNNrALIURR8tTQSEtLY+PGjfnauKIohTo0AIYNG8aNGzdYvHgx/fv3R61WU716dZydnbl9+zYxMTGkpKSwcOFCCQ0hRJH31NBwcnKicePGBVGL3Zo4cSIdO3ZkxYoVHD16lD///JPixYvj4eFBu3btaN++Pc2bN7d1mUIIYXUqRVGU3Gb6+PhQoUIFfceEQtiCJQZh8vBwJT7+gYUqEk+S42td+Tm+Dg4qqwzuJndP2YmEBA1aba75LYQQdkHunhJCCGEyCQ0hhBAmk8tTdsIa1x7tRWZGOvcT056+oBDC7klo2ImIPT+S9ijR1mVYRWCPyYCEhhBFQZ6hcebMmYKqo0A8PrKfEEII80mbhhBCCJNJaAghhDCZhIYQQgiTPbcN4du3b2fJkiWcO3cOyO7RdvTo0bl2B5KSksLy5cvZuXMnly5dIjMzk2rVqtGlSxeGDx9OqVKlCrJ8IYSwiefyTGPWrFl89NFHODs7ExQURKVKlTh48CCvv/46UVFROZa/desW/fv359tvv+XGjRs0adKE1q1bk5SUxJw5cxg0aBBJSUk22BMhhChYz+WZRkhICGvWrKFBgwZA9gh9//nPf1i9ejU//fQTv/32m35ZRVEYO3Ys58+fZ8iQIYwfP57ixYsDkJqayhdffMHmzZuZNm2aDNQkhCjynsszjdGjR+sDA7KHgB0zZgwAhw8fJiMjQz9v3759REVF0bhxYz777DN9YAAUL16cKVOm4O7uzpYtW+RsQwhR5D2XodGuXbsc0ypUqECZMmVIT08nMfGfh+z27dsHQOfOnY2OL16yZEkaNGhAZmYmJ0+etFrNQghhD57Ly1OVK1c2Or106dIkJSWRlvbP08vXrl0DYObMmcycOTPP7d67d89yRQohhB16LkPD2BlDbrKysgDw9/enSpUqeS6bWxgJIURR8VyGhjm8vLwA6NKlC8HBwTauRgghbOu5bNMwR2BgIAA7d+60cSVCCGF7EhpP0bFjR3x9fYmMjGTSpEkGjeQ68fHxrF692gbVCSFEwZLLU0/h4ODAzz//zIgRI1i1ahVbt27Fx8cHLy8v0tLSuHz5MufPn8fd3Z0BAwbYulwhhLAqCQ0TVKpUibVr17J27Vp27NjBuXPnOHHiBGXLlqVixYq88cYbdOrUydZlCiGE1akURVFsXYQo2uxh5D4PD1fi4x/YtIaiTI6vdeXn+Do4qKwyIqicadiJhAQNWq3ktxDCvklDuBBCCJNJaAghhDCZXJ6yE9a49mgLaRmZJCc+snUZQggrkdCwE1N3HeR+im0biy3h+z5Bti5BCGFFcnkqD+3bt0etVhMREWHrUoQQwi5IaJhp9uzZqNVqZs+ebetShBCiwEloCCGEMJmEhhBCCJNJQ7gZ1Gq1/uc5c+YwZ84c/e/vvfceo0ePtkVZQghRYCQ0zNCnTx9iYmI4c+YMPj4+1K9fXz/v8Z+FEKKoktAww/Tp05k9ezZnzpyhY8eOcmYhhHjuSJuGEEIIk0loCCGEMJmEhhBCCJNJaAghhDCZhIYQQgiTSWiYydnZGYDMzEwbVyKEEAVPQsNMnp6eAFy8eNHGlQghRMGT5zTM1KZNG0qUKMHu3bsJDg6mevXqODg40L59ezp06GDr8oQQwqokNMzk4eHBvHnzmDt3LjExMRw5cgRFUahUqZKEhhCiyFMpiqLYughRdNjryH0eHq7Exz+wdRlFlhxf68rP8XVwUFllRFA507AT9+8/RKstGvnt4KCydQlG2WtdRYUcX+sy9/ha6/2QMw0hhBAmk7unhBBCmExCQwghhMkkNIQQQphMQkMIIYTJJDSEEEKYTEJDCCGEySQ0hBBCmExCQwghhMkkNIQQQphMQkMIIYTJJDSEEEKYTEJDCCGEySQ0hBBCmEy6RrehLVu2sGLFCs6ePYtWq6VWrVr069ePQYMG4eAgeT5hwgQ2bNiQ6/xatWqxc+fOHNO1Wi0rVqxg3bp1XLp0CQcHB9RqNYMHD6ZHjx55vmZRe08uXrzI/v37OXnyJKdOneLy5csoisKsWbPo0qVLnuvm91js27ePxYsXc+rUKdLS0qhWrRrdu3fn3//+N8WKFct1vePHj7NgwQKOHj2KRqPBy8uLjh078s477+Dq6prvY2Bt+TnG+f1sg+0/3xIaNjJlyhRCQkJwcXEhICAAJycnwsPD+e9//0t4eDg//fRTofySsoamTZtSo0aNHNM9PDxyTMvKyuK9997jf//7H6VLl6Z169akp6cTHh7ORx99xLFjx/j888+Nvk5RfE9WrFjB0qVLzV4vv8di4cKFfPvttzg6OuLv74+bmxuHDh3ixx9/5M8//2Tx4sWUKFEix3pbt27l448/Jisri6ZNm+Lp6cnx48f59ddfCQ0NZcWKFbi7u+frGFhbfo8xmPfZBjv5fCuiwO3cuVPx9vZWWrdurVy6dEk/PT4+Xunatavi7e2tLF682HYF2olPPvlE8fb2VtatW2fyOr/++qvi7e2tdOvWTYmPj9dPv3TpktKqVSvF29tb+eOPP3KsV1Tfk9WrVyszZsxQtm3bply5ckUZMmSI4u3trezYsSPXdfJ7LE6cOKGo1WrlxRdfVI4dO6afrtFolODgYMXb21v56quvcqx38+ZNpVGjRoqPj4/Be5ORkaGMHTtW8fb2Vt599918HgHry88xzs9nW1Hs4/MtoWEDffr0Uby9vZUNGzbkmBcREaF/c7OysmxQnf0w93+szMxMJSAgQPH29lYiIyNzzF+/fr3i7e2t9OvXL8e85+U9MeULLb/HYvTo0Yq3t7cye/bsHOtdvXpV8fHxUXx9fZWkpCSDedOnT1e8vb2VCRMm5FjvwYMHStOmTRVvb28lNjbW1N20KWuFhr18vgvXuXYRcOvWLaKjo3F2djZ6vdPf3x9PT0/i4+M5duyYDSosvKKiokhISKBSpUr4+fnlmN+lSxecnZ05efIkt2/f1k+X9+Qf+T0W6enp7Nu3D4BevXrlWK9atWo0btyYjIwM9u7dazAvNDQ01/VKly7NSy+9ZLDc88pePt8SGgXs9OnTANSrV4/ixYsbXaZhw4YAxMTEFFhd9iwiIoJp06bxxRdf8OOPP7J//360Wm2O5XTHS3f8nlSiRAnq1q1rsCzIe/K4/B6LS5cu8ejRI8qWLUv16tXzXE/3GgAajYarV68azDdlvaLC1M822M/nWxrCC9j169cBqFy5cq7LeHl5GSz7vNu4cWOOaXXr1uX7779HrVbrp5l6bGNiYgyOrbwn/8jvsdD9rJtnjG6bcXFxOdZzc3OjdOnSea5XFI+9qZ9tsJ/Pt5xpFLCUlBQAo3eQ6JQqVQqAhw8fFkhN9srHx4fPP/+c7du3ExUVxf79+5k/fz4+Pj6cP3+eN954w+A03JRjW7JkScDw2Mp78o/8HgtrHntj6xV25n62wX4+33KmIezW66+/bvB7yZIlqVixIq1atWLo0KEcO3aM+fPnM2nSJNsUKEQ+FebPtpxpFDDdXwKPHj3KdRld2uvSXxgqVqwYI0eOBDBoVDXl2Or+6nr82Mp78o/8HgtrHntj6xVVuX22wX4+3xIaBaxKlSoA3LhxI9dlbt26ZbCsyKl27doABqfw+T228p7841mP4c2bN3NdTzfP2HrJycloNJo816tatepT6y8KjH22wX4+3xIaBeyFF14AIDY2ltTUVKPLnDx5EoD69esXWF2FTWJiImD4l5Hu2OqO35MePXpEbGyswbKP/yzvSf6PRe3atSlevDiJiYn6u6GedOLEiRzrubq66u+2yu19M7ZeUWbssw328/mW0ChgXl5e+Pr6kpGRYbRvmcjISG7duoWHhwdNmjSxQYWFw44dOwBo0KCBflqTJk0oX748t27d4tChQznW2blzJxkZGTRs2BBPT0/9dHlP/pHfY1GsWDECAwMB2Lx5c471rl27xrFjx3B2dqZdu3YG8zp06JDrehqNhrCwMAA6deqU7/0qTIx9tsF+Pt8SGjagu2b57bffcuXKFf30hIQEpkyZAsCIESMKXT9HlhQTE0NYWBhZWVkG0zMzM1m0aBHLli0DDBsUHR0defPNNwGYPHkyCQkJ+nmXL1/mu+++A+Dtt9/O8Xrynvwjv8dixIgRqFQqfvnlF/3ZAWRfL//000/RarUMHjwYNzc3g/WGDRtG8eLF2bhxI3v27NFPz8zMZNKkSWg0Gjp27Kh/BqGwy89nG+zn861SFEV56lLC4iZPnsyKFStwcXGhVatW+s7DdP+D/PTTTzg6Otq6TJsJDQ1l1KhRlC1blhdeeIHy5cuTmJjIuXPnuHPnDg4ODnz00Uf6/4l0srKyGDVqFGFhYZQuXZqAgAAyMzP5+++/SUtLY+jQobl26FYU35Po6Gj9lwLA+fPnefjwITVr1qRMmTL66atXrzZYL7/H4vEOC1u2bImrqyuHDh0iISGBF198kSVLluTZYaFWq6VZs2ZUrFiR48ePExcXR40aNey6w0Jzj3F+P9tgH59vCQ0b2rJlC8uXL+fcuXNotVpq165dqLvhtqRr166xdOlSTp48SVxcHImJiahUKipVqkSzZs0IDg7Ocfquo9VqCQkJYf369Vy8eNGg6+iePXvm+bpF7T2JiIjgtddee+pyZ8+ezTEtv8di3759/PbbbwZdo/fo0cOkrtHnz59v0DV6p06d7L5rdHOP8bN8tsH2n28JDSGEECYrfH86CSGEsBkJDSGEECaT0BBCCGEyCQ0hhBAmk9AQQghhMgkNIYQQJpPQEEIIYTIJDSGEECaTQZiEEDlEREQQGRlJ/fr16dixo63LEXZEzjSEEDlERkYyZ84cQkNDbV2KsDMSGkIIIUwmoSGEEMJk0mGhEPl04cIFlixZQkREBLdu3cLZ2RkvLy9atGhB7969c/RUevr0aRYtWqTvKrxUqVI0aNCAAQMG8PLLLxt9DbVaDcCePXuMDnd6/fp1/SBGT/ZUO3ToUCIjI5k2bRrdunVj4cKFbN26lZs3b1KqVClatmzJmDFjqFmzptHt5Sa3WsTzQRrChciHZcuWMW3aNP1AOiVLlkSlUnHu3DnOnTvH2bNn9YPpAKxatYrJkyej1WoBcHNz48GDBxw4cIADBw7Qq1cvpk+fbpXxOjQaDYMGDeL06dMUK1YMBwcH7t27x/bt2/n7779Zs2aNfshVR0dHKlSoQEpKCikpKbi4uOTolrywjSkiLEsuTwlhph07dvDll1+SlZXFyy+/zPbt24mKiuLQoUNERETwzTff4Ovrq1/+6NGj+sB4+eWX2bt3L4cOHeLQoUOMHTsWlUrF5s2bmT9/vlXqnT17NklJSfzyyy8cO3aMqKgoli9fTqVKlUhMTNSP+AbZQ4P+9ddfDB8+HIBu3brx119/Gfzz8vKySp2icJDQEMIMGRkZTJs2DYAePXrw008/UadOHf38smXL0qtXLyZMmKCfNmvWLLRaLU2bNuWHH36gUqVKAJQqVYp33nlHPxTnwoUL0Wg0Fq85PT2d3377jbZt2+Lo6IiDgwPNmzfn008/BeB///sf6enpFn9dUTRJaAhhhvDwcG7fvo2joyMff/zxU5dPTEwkIiICgLfeesvopZ0RI0bg4uJCSkoKe/futXjNL7/8MjVq1MgxvX379qhUKtLT07l69arFX1cUTRIaQpjh+PHjAPj4+ODp6fnU5WNiYlAUBZVKhZ+fn9FlXF1d9ZezoqOjLVfs/9ewYUOj052dnfXjbiclJVn8dUXRJKEhhBnu3r0LYPJ1/Xv37gHZwVCqVKlcl9Ndsrp///4zVphTXq/r4uICQGZmpsVfVxRNEhpCFABpMxBFhYSGEGaoUKECADdu3DBp+fLlywOQmpqqP+sw5tatWwCUK1fOYLquDSQtLc3oetZoOBciLxIaQpjhxRdfBLIfpLt9+/ZTl69fvz4qlQqAgwcPGl3mwYMH+raMx2/VBfTPSOT2WidPnjStcDPpapZnf8WTJDSEMENAQACenp5kZWUxc+bMpy5ftmxZWrRoAcAvv/yif7jvcQsXLiQtLY2SJUsSFBRkMM/b2xvIfgr7Senp6SxZsiQ/u/FUpUuXBiA5Odkq2xeFl4SGEGZwdnbWP4OxdetWxowZw4ULF/TzExMTWb16NV9++aV+2pgxY3BwcCA6OpoPPvhAfynq4cOHzJs3jwULFgAwcuRI/Ze1TteuXQFYvXo169at07eNxMbGMmLECO7cuWOV/axXrx6Q/WDi5cuXrfIaonCSvqeEyIfffvuNmTNn6s8cSpYsiZOTk/4vc39/f4NuRFauXMmUKVPQarWoVCrc3NzQaDT6bkh69uzJjBkzcjzHkZGRQXBwsP5WXycnJ4oXL45Go6Fs2bJ89dVXjBo1Csi776m+ffsa3Y/27dsTFxfH0qVL9WdEutft1q0bV69eRaVSUa5cOUqUKAFASEiI/m4v8fyRvqeEyIc33niDgIAAfYeF8fHxODs7o1aradGiBX369DFY/tVXX6Vhw4YsWrSIyMhI7t+/r38+Y8CAAXTp0sXo6zg7O7No0SJ+/vlndu7cyZ07dyhRogSdO3fWh4U1ODs7s3jxYmbNmkVERAR3797VN+TL7bnPNznTEEIIYTJp0xBCCGEyCQ0hhBAmk9AQQghhMgkNIYQQJpPQEEIIYTIJDSGEECaT0BBCCGEyCQ0hhBAmk9AQQghhMgkNIYQQJvt/oMZM+FPbRCsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import seaborn as sns #for visualization\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "plt.style.use('seaborn')\n",
    "sns.set(font_scale=2)\n",
    "\n",
    "tweets = pd.read_csv(\"../data/dataset/trump_tweets.csv\")\n",
    "\n",
    "text = tweets['text'].str.cat(sep=' ')\n",
    "# spaCy enforces a max limit of 1000000 characters for NER and similar use cases.\n",
    "# Since `text` might be longer than that, we will slice it off here\n",
    "max_length = 1000000-1\n",
    "text = text[:max_length]\n",
    "\n",
    "# removing URLs and '&amp' substrings using regex\n",
    "import re\n",
    "url_reg  = r'[a-z]*[:.]+\\S+'\n",
    "text   = re.sub(url_reg, '', text)\n",
    "noise_reg = r'\\&amp'\n",
    "text   = re.sub(noise_reg, '', text)\n",
    "\n",
    "doc = nlp(text)\n",
    "\n",
    "items_of_interest = list(doc.noun_chunks)\n",
    "# each element in this list is spaCy's inbuilt `Span`, which is not useful for us\n",
    "items_of_interest = [str(x) for x in items_of_interest]\n",
    "# so we've converted it to string\n",
    "\n",
    "df_nouns = pd.DataFrame(items_of_interest, columns=[\"TrumpSays\"])\n",
    "plt.figure(figsize=(5,4))\n",
    "sns.countplot(y=\"TrumpSays\",\n",
    "             data=df_nouns,\n",
    "             order=df_nouns[\"TrumpSays\"].value_counts().iloc[:10].index)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### WordCloud\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-22-c6a2c052fd41>, line 6)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-22-c6a2c052fd41>\"\u001b[0;36m, line \u001b[0;32m6\u001b[0m\n\u001b[0;31m    trump_topics = ???\u001b[0m\n\u001b[0m                   ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "from spacy.lang.en.stop_words import STOP_WORDS\n",
    "from wordcloud import WordCloud\n",
    "\n",
    "#Can you plot the entities that Trump mentions the most?\n",
    "# Visit https://www.kaggle.com/nirant/hitchhiker-s-guide-to-nlp-in-spacy \n",
    "trump_topics = ???\n",
    "\n",
    "plt.figure(figsize=(10,5))\n",
    "wordcloud = WordCloud(background_color=\"white\",\n",
    "                      stopwords = STOP_WORDS,\n",
    "                      max_words=45,\n",
    "                      max_font_size=30,\n",
    "                      random_state=42\n",
    "                     ).generate(str(trump_topics))\n",
    "plt.imshow(wordcloud)\n",
    "plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dependency Parsing\n",
    "\n",
    "Dependency parsing is the process of assigning syntactic dependency labels that describe the relationships between individual tokens, like subject or object.\n",
    "\n",
    "Dependency parsing analyzes the grammatical structure of a sentence. It establishes a \"tree\" like structure between a \"root\" word and those that are related to it by branches of some manner.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall/NNP<--compound-- Street/NNP\n",
      "Street/NNP<--compound-- Journal/NNP\n",
      "Journal/NNP<--nsubj-- published/VBD\n",
      "just/RB<--advmod-- published/VBD\n",
      "published/VBD<--ROOT-- published/VBD\n",
      "an/DT<--det-- piece/NN\n",
      "interesting/JJ<--amod-- piece/NN\n",
      "piece/NN<--dobj-- published/VBD\n",
      "on/IN<--prep-- piece/NN\n",
      "crypto/NNP<--compound-- currencies/NNS\n",
      "currencies/NNS<--pobj-- on/IN\n"
     ]
    }
   ],
   "source": [
    "doc = nlp('Wall Street Journal just published an interesting piece on crypto currencies')\n",
    " \n",
    "for token in doc:\n",
    "    print(\"{0}/{1}<--{2}-- {3}/{4}\".format(\n",
    "        token.text, token.tag_, token.dep_, token.head.text, token.head.tag_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xml:lang=\"en\" id=\"1390aa432a6d4f33b28509aa7155b496-0\" class=\"displacy\" width=\"1370\" height=\"317.0\" direction=\"ltr\" style=\"max-width: none; height: 317.0px; color: #000000; background: #ffffff; font-family: Arial; direction: ltr\">\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"50\">Wall</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"50\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"170\">Street</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"170\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"290\">Journal</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"290\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"410\">just</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"410\">ADV</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"530\">published</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"530\">VERB</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"650\">an</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"650\">DET</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"770\">interesting</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"770\">ADJ</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"890\">piece</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"890\">NOUN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"1010\">on</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"1010\">ADP</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"1130\">crypto</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"1130\">PROPN</tspan>\n",
       "</text>\n",
       "\n",
       "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"227.0\">\n",
       "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"1250\">currencies</tspan>\n",
       "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"1250\">NOUN</tspan>\n",
       "</text>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-1390aa432a6d4f33b28509aa7155b496-0-0\" stroke-width=\"2px\" d=\"M70,182.0 C70,122.0 160.0,122.0 160.0,182.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-1390aa432a6d4f33b28509aa7155b496-0-0\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">compound</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M70,184.0 L62,172.0 78,172.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-1390aa432a6d4f33b28509aa7155b496-0-1\" stroke-width=\"2px\" d=\"M190,182.0 C190,122.0 280.0,122.0 280.0,182.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-1390aa432a6d4f33b28509aa7155b496-0-1\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">compound</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M190,184.0 L182,172.0 198,172.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-1390aa432a6d4f33b28509aa7155b496-0-2\" stroke-width=\"2px\" d=\"M310,182.0 C310,62.0 525.0,62.0 525.0,182.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-1390aa432a6d4f33b28509aa7155b496-0-2\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">nsubj</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M310,184.0 L302,172.0 318,172.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-1390aa432a6d4f33b28509aa7155b496-0-3\" stroke-width=\"2px\" d=\"M430,182.0 C430,122.0 520.0,122.0 520.0,182.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-1390aa432a6d4f33b28509aa7155b496-0-3\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">advmod</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M430,184.0 L422,172.0 438,172.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-1390aa432a6d4f33b28509aa7155b496-0-4\" stroke-width=\"2px\" d=\"M670,182.0 C670,62.0 885.0,62.0 885.0,182.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-1390aa432a6d4f33b28509aa7155b496-0-4\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">det</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M670,184.0 L662,172.0 678,172.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-1390aa432a6d4f33b28509aa7155b496-0-5\" stroke-width=\"2px\" d=\"M790,182.0 C790,122.0 880.0,122.0 880.0,182.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-1390aa432a6d4f33b28509aa7155b496-0-5\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">amod</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M790,184.0 L782,172.0 798,172.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-1390aa432a6d4f33b28509aa7155b496-0-6\" stroke-width=\"2px\" d=\"M550,182.0 C550,2.0 890.0,2.0 890.0,182.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-1390aa432a6d4f33b28509aa7155b496-0-6\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">dobj</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M890.0,184.0 L898.0,172.0 882.0,172.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-1390aa432a6d4f33b28509aa7155b496-0-7\" stroke-width=\"2px\" d=\"M910,182.0 C910,122.0 1000.0,122.0 1000.0,182.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-1390aa432a6d4f33b28509aa7155b496-0-7\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">prep</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M1000.0,184.0 L1008.0,172.0 992.0,172.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-1390aa432a6d4f33b28509aa7155b496-0-8\" stroke-width=\"2px\" d=\"M1150,182.0 C1150,122.0 1240.0,122.0 1240.0,182.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-1390aa432a6d4f33b28509aa7155b496-0-8\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">compound</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M1150,184.0 L1142,172.0 1158,172.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "\n",
       "<g class=\"displacy-arrow\">\n",
       "    <path class=\"displacy-arc\" id=\"arrow-1390aa432a6d4f33b28509aa7155b496-0-9\" stroke-width=\"2px\" d=\"M1030,182.0 C1030,62.0 1245.0,62.0 1245.0,182.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
       "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
       "        <textPath xlink:href=\"#arrow-1390aa432a6d4f33b28509aa7155b496-0-9\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">pobj</textPath>\n",
       "    </text>\n",
       "    <path class=\"displacy-arrowhead\" d=\"M1245.0,184.0 L1253.0,172.0 1237.0,172.0\" fill=\"currentColor\"/>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "displacy.render(doc, style='dep', jupyter=True, options={'distance': 120})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Word Vectors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "spaCy comes shipped with a Word Vector model as well. We’ll need to download a larger model for that:\n",
    "\n",
    "`python -m spacy download en_core_web_lg`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 2.0228e-01 -7.6618e-02  3.7032e-01  3.2845e-02 -4.1957e-01  7.2069e-02\n",
      " -3.7476e-01  5.7460e-02 -1.2401e-02  5.2949e-01 -5.2380e-01 -1.9771e-01\n",
      " -3.4147e-01  5.3317e-01 -2.5331e-02  1.7380e-01  1.6772e-01  8.3984e-01\n",
      "  5.5107e-02  1.0547e-01  3.7872e-01  2.4275e-01  1.4745e-02  5.5951e-01\n",
      "  1.2521e-01 -6.7596e-01  3.5842e-01 -4.0028e-02  9.5949e-02 -5.0690e-01\n",
      " -8.5318e-02  1.7980e-01  3.3867e-01  1.3230e-01  3.1021e-01  2.1878e-01\n",
      "  1.6853e-01  1.9874e-01 -5.7385e-01 -1.0649e-01  2.6669e-01  1.2838e-01\n",
      " -1.2803e-01 -1.3284e-01  1.2657e-01  8.6723e-01  9.6721e-02  4.8306e-01\n",
      "  2.1271e-01 -5.4990e-02 -8.2425e-02  2.2408e-01  2.3975e-01 -6.2260e-02\n",
      "  6.2194e-01 -5.9900e-01  4.3201e-01  2.8143e-01  3.3842e-02 -4.8815e-01\n",
      " -2.1359e-01  2.7401e-01  2.4095e-01  4.5950e-01 -1.8605e-01 -1.0497e+00\n",
      " -9.7305e-02 -1.8908e-01 -7.0929e-01  4.0195e-01 -1.8768e-01  5.1687e-01\n",
      "  1.2520e-01  8.4150e-01  1.2097e-01  8.8239e-02 -2.9196e-02  1.2151e-03\n",
      "  5.6825e-02 -2.7421e-01  2.5564e-01  6.9793e-02 -2.2258e-01 -3.6006e-01\n",
      " -2.2402e-01 -5.3699e-02  1.2022e+00  5.4535e-01 -5.7998e-01  1.0905e-01\n",
      "  4.2167e-01  2.0662e-01  1.2936e-01 -4.1457e-02 -6.6777e-01  4.0467e-01\n",
      " -1.5218e-02 -2.7640e-01 -1.5611e-01 -7.9198e-02  4.0037e-02 -1.2944e-01\n",
      " -2.4090e-04 -2.6785e-01 -3.8115e-01 -9.7245e-01  3.1726e-01 -4.3951e-01\n",
      "  4.1934e-01  1.8353e-01 -1.5260e-01 -1.0808e-01 -1.0358e+00  7.6217e-02\n",
      "  1.6519e-01  2.6526e-04  1.6616e-01 -1.5281e-01  1.8123e-01  7.0274e-01\n",
      "  5.7956e-03  5.1664e-02 -5.9745e-02 -2.7551e-01 -3.9049e-01  6.1132e-02\n",
      "  5.5430e-01 -8.7997e-02 -4.1681e-01  3.2826e-01 -5.2549e-01 -4.4288e-01\n",
      "  8.2183e-03  2.4486e-01 -2.2982e-01 -3.4981e-01  2.6894e-01  3.9166e-01\n",
      " -4.1904e-01  1.6191e-01 -2.6263e+00  6.4134e-01  3.9743e-01 -1.2868e-01\n",
      " -3.1946e-01 -2.5633e-01 -1.2220e-01  3.2275e-01 -7.9933e-02 -1.5348e-01\n",
      "  3.1505e-01  3.0591e-01  2.6012e-01  1.8553e-01 -2.4043e-01  4.2886e-02\n",
      "  4.0622e-01 -2.4256e-01  6.3870e-01  6.9983e-01 -1.4043e-01  2.5209e-01\n",
      "  4.8984e-01 -6.1067e-02 -3.6766e-01 -5.5089e-01 -3.8265e-01 -2.0843e-01\n",
      "  2.2832e-01  5.1218e-01  2.7868e-01  4.7652e-01  4.7951e-02 -3.4008e-01\n",
      " -3.2873e-01 -4.1967e-01 -7.5499e-02 -3.8954e-01 -2.9622e-02 -3.4070e-01\n",
      "  2.2170e-01 -6.2856e-02 -5.1903e-01 -3.7774e-01 -4.3477e-03 -5.8301e-01\n",
      " -8.7546e-02 -2.3929e-01 -2.4711e-01 -2.5887e-01 -2.9894e-01  1.3715e-01\n",
      "  2.9892e-02  3.6544e-02 -4.9665e-01 -1.8160e-01  5.2939e-01  2.1992e-01\n",
      " -4.4514e-01  3.7798e-01 -5.7062e-01 -4.6946e-02  8.1806e-02  1.9279e-02\n",
      "  3.3246e-01 -1.4620e-01  1.7156e-01  3.9981e-01  3.6217e-01  1.2816e-01\n",
      "  3.1644e-01  3.7569e-01 -7.4690e-02 -4.8480e-02 -3.1401e-01 -1.9286e-01\n",
      " -3.1294e-01 -1.7553e-02 -1.7514e-01 -2.7587e-02 -1.0000e+00  1.8387e-01\n",
      "  8.1434e-01 -1.8913e-01  5.0999e-01 -9.1960e-03 -1.9295e-03  2.8189e-01\n",
      "  2.7247e-02  4.3409e-01 -5.4967e-01 -9.7426e-02 -2.4540e-01 -1.7203e-01\n",
      " -8.8650e-02 -3.0298e-01 -1.3591e-01 -2.7765e-01  3.1286e-03  2.0556e-01\n",
      " -1.5772e-01 -5.2308e-01 -6.4701e-01 -3.7014e-01  6.9393e-02  1.1401e-01\n",
      "  2.7594e-01 -1.3875e-01 -2.7268e-01  6.6891e-01 -5.6454e-02  2.4017e-01\n",
      " -2.6730e-01  2.9860e-01  1.0083e-01  5.5592e-01  3.2849e-01  7.6858e-02\n",
      "  1.5528e-01  2.5636e-01 -1.0772e-01 -1.2359e-01  1.1827e-01 -9.9029e-02\n",
      " -3.4328e-01  1.1502e-01 -3.7808e-01 -3.9012e-02 -3.4593e-01 -1.9404e-01\n",
      " -3.3580e-01 -6.2334e-02  2.8919e-01  2.8032e-01 -5.3741e-01  6.2794e-01\n",
      "  5.6955e-02  6.2147e-01 -2.5282e-01  4.1670e-01 -1.0108e-02 -2.5434e-01\n",
      "  4.0003e-01  4.2432e-01  2.2672e-01  1.7553e-01  2.3049e-01  2.8323e-01\n",
      "  1.3882e-01  3.1218e-03  1.7057e-01  3.6685e-01  2.5247e-03 -6.4009e-01\n",
      " -2.9765e-01  7.8943e-01  3.3168e-01 -1.1966e+00 -4.7156e-02  5.3175e-01]\n"
     ]
    }
   ],
   "source": [
    "nlp = spacy.load('en_core_web_lg')\n",
    "print(nlp.vocab['banana'].vector)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Computing Similarity**\n",
    "\n",
    "Based on the word embeddings, spaCy offers a similarity interface for all of it’s building blocks: Token, Span, Doc and Lexeme. Here’s how to use that similarity interface:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'dog' is 0.66185343 similar to 'animal' and 0.23552851 similar to 'fruit'\n",
      "'banana' is 0.24272855 similar to 'animal' and 0.67148364 similar to 'fruit'\n"
     ]
    }
   ],
   "source": [
    "banana = nlp.vocab['banana']\n",
    "dog = nlp.vocab['dog']\n",
    "fruit = nlp.vocab['fruit']\n",
    "animal = nlp.vocab['animal']\n",
    " \n",
    "print(\"'dog' is %s similar to 'animal' and %s similar to 'fruit'\" % (dog.similarity(animal), dog.similarity(fruit)))\n",
    "print(\"'banana' is %s similar to 'animal' and %s similar to 'fruit'\" % (banana.similarity(animal), banana.similarity(fruit)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s now use this technique on entire texts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8901766262114666\n",
      "0.9115828449161616\n",
      "0.7822956256736615\n",
      "0.7133323899064792\n",
      "0.6526212010025575\n"
     ]
    }
   ],
   "source": [
    "target = nlp(\"Cats are beautiful animals.\")\n",
    " \n",
    "doc1 = nlp(\"Dogs are awesome.\")\n",
    "doc2 = nlp(\"Some gorgeous creatures are felines.\")\n",
    "doc3 = nlp(\"Dolphins are swimming mammals.\")\n",
    "doc4 = nlp(\"Snoopy is a very smart dog.\")\n",
    "doc5 = nlp(\"Tomorrow it will rain a lot in Berlin.\")\n",
    " \n",
    "print(target.similarity(doc1))\n",
    "print(target.similarity(doc2))\n",
    "print(target.similarity(doc3))\n",
    "print(target.similarity(doc4))\n",
    "print(target.similarity(doc5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\"king\" - \"man\" + \"woman\" = \"queen\"?\n",
    "\n",
    "There’s a really famous example of word embedding math: \"king\" - \"man\" + \"woman\" = \"queen\". Let’s test that out:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "maybe_queen : [ 5.14087021e-01 -2.78459996e-01  2.42767006e-01  4.54899669e-02\n",
      " -2.59425014e-01 -3.19999963e-01  3.23920012e-01 -6.71030045e-01\n",
      " -9.98499990e-02  1.91499996e+00 -5.68080008e-01 -2.74451017e-01\n",
      " -1.49906695e-01  8.01083148e-02 -2.34764010e-01 -1.10950008e-01\n",
      " -1.02593988e-01  8.53819966e-01 -2.68564999e-01  3.85140002e-01\n",
      " -1.36149988e-01  6.35029972e-01 -7.62044966e-01 -2.52770007e-01\n",
      " -6.75969958e-01  3.89851004e-01 -2.89680034e-01  1.75860003e-01\n",
      " -5.16229987e-01  5.21373034e-01 -1.89909995e-01  6.73759937e-01\n",
      "  1.17550008e-01 -4.69896019e-01  5.88999987e-01  1.29447982e-01\n",
      " -5.71900010e-01 -5.47450066e-01 -4.84210014e-01  5.85503951e-02\n",
      "  4.82379973e-01 -2.86769986e-01 -2.01718003e-01 -4.74729985e-01\n",
      "  3.43068987e-01 -2.28827983e-01 -1.76439017e-01  6.05450034e-01\n",
      "  2.07139999e-01 -2.89762974e-01 -7.63288975e-01  4.37090009e-01\n",
      " -2.06220001e-01 -4.20252979e-01  1.98040009e-01  3.18709970e-01\n",
      " -9.51815993e-02 -3.23054016e-01 -6.02343976e-01  2.33427018e-01\n",
      " -2.15409994e-02 -6.29774988e-01  3.72432500e-01  3.41740012e-01\n",
      "  5.81782043e-01  7.02129960e-01  7.19299972e-01  3.28493983e-01\n",
      "  3.36353004e-01  1.06999278e-03 -5.53239942e-01 -2.46219993e-01\n",
      " -6.37116969e-01 -1.72280014e-01  8.97620022e-01 -1.38548493e-01\n",
      " -5.71600199e-02  6.41870022e-01  3.89845997e-01 -3.98499995e-01\n",
      " -7.28532076e-01  9.17530134e-02 -3.40600014e-01  3.46671015e-01\n",
      " -2.63424516e-01  3.68355006e-01  8.78340006e-01 -1.57473043e-01\n",
      " -4.29450005e-01 -4.91259992e-01 -1.23234093e-02  3.27509999e-01\n",
      "  1.44889995e-01 -3.27081025e-01  9.45929945e-01 -8.07909966e-01\n",
      " -2.07101002e-01 -8.87000561e-03 -5.59080057e-02  7.93069959e-01\n",
      "  3.58245999e-01  6.05069995e-01  1.01848006e-01 -1.89061001e-01\n",
      "  1.09030008e-02 -7.64109969e-01 -5.05369961e-01 -1.11367017e-01\n",
      "  6.56607985e-01 -1.48448005e-01  1.30866021e-01  6.62039995e-01\n",
      " -1.54300034e-02 -4.17466015e-01 -4.54553008e-01 -5.05975008e-01\n",
      "  4.15473014e-01  4.00425017e-01  7.88707018e-01 -5.19399941e-02\n",
      " -3.91889989e-01  8.31609964e-02  4.58730012e-01  1.23339996e-01\n",
      "  2.39246994e-01  3.81098986e-01  1.86000004e-01  2.69684941e-02\n",
      " -5.55605292e-01  2.53284007e-01 -6.67639971e-01 -5.55985987e-01\n",
      " -3.71130019e-01 -6.53919995e-01 -1.09452009e-01 -6.04629993e-01\n",
      " -4.62760001e-01  3.97581995e-01 -3.26649994e-01  2.60998994e-01\n",
      " -2.09120011e+00 -2.76019007e-01  2.68036008e-01 -3.35714996e-01\n",
      " -4.75513011e-01 -2.83890069e-02  4.40270007e-01  2.24150002e-01\n",
      " -4.50639009e-01 -6.16590083e-01  1.10599995e-01 -3.00589710e-01\n",
      "  1.24530017e-01  2.99279988e-01  3.03467005e-01 -3.42969984e-01\n",
      "  3.93694013e-01 -5.84149957e-01 -1.88180000e-01  2.98162013e-01\n",
      " -1.80879980e-01 -3.70599926e-02  4.09860015e-02 -8.07899833e-02\n",
      "  3.92280012e-01 -4.94572997e-01  4.01719987e-01  8.48469973e-01\n",
      " -1.94183022e-01  4.29439992e-01 -6.07819974e-01 -9.71959978e-02\n",
      "  3.55786979e-01 -1.79980025e-02 -5.83269954e-01 -2.50129998e-01\n",
      "  2.80330002e-01 -3.72725993e-01 -7.41009951e-01  1.03881419e-01\n",
      "  8.04000199e-02 -1.64650023e-01  1.09247290e-01 -5.68639994e-01\n",
      "  4.11399961e-01  5.69249988e-01 -2.14549989e-01 -1.56975001e-01\n",
      "  9.64879990e-02  2.01149940e-01 -9.81989980e-01 -9.00639057e-01\n",
      "  1.57496989e-01 -1.24968991e-01  9.11729932e-02 -5.17108977e-01\n",
      "  6.34269863e-02  1.72169998e-01 -2.36945987e-01 -7.58899987e-01\n",
      "  5.74868977e-01  6.10739946e-01  8.88329893e-02 -2.59585023e-01\n",
      " -9.03399587e-02 -8.53200257e-02  1.69609979e-01 -7.29799643e-03\n",
      " -2.05680996e-01 -1.93440005e-01 -4.92264986e-01  3.19920003e-01\n",
      " -3.66147995e-01  5.69279015e-01  6.27799928e-02  7.91899860e-02\n",
      " -3.93792808e-01  4.87831026e-01 -3.85988951e-02  7.52799988e-01\n",
      "  1.74212992e-01 -6.07100964e-01  4.81240004e-01  1.49755001e-01\n",
      "  4.32273030e-01  2.77104974e-01  4.56589013e-01 -3.32702011e-01\n",
      " -2.80999988e-01  6.35839045e-01  1.15425006e-01  7.80760050e-02\n",
      "  3.17489982e-01 -4.80073988e-01  4.07790095e-02 -8.21070611e-01\n",
      " -1.63500011e-03 -3.97460014e-01 -9.85880196e-02 -5.31642020e-01\n",
      " -4.52499986e-02 -4.23010021e-01  1.44284993e-01 -7.62080014e-01\n",
      "  2.15179995e-01 -7.05516994e-01  6.44015014e-01 -9.44310054e-02\n",
      " -5.36169946e-01 -1.31442308e+00  4.51058030e-01  1.44240022e-01\n",
      "  3.84460092e-02 -1.80320218e-02 -2.95219988e-01  4.90060002e-01\n",
      "  3.83020639e-02 -1.70519948e-02 -7.32708037e-01  5.04490495e-01\n",
      "  1.77098006e-01  5.36670089e-02 -2.40814000e-01 -8.20799917e-02\n",
      "  2.19249994e-01 -4.58490014e-01  3.68449986e-01  3.09300005e-01\n",
      " -1.21967995e+00 -2.55998999e-01 -8.38758051e-01 -1.99926004e-01\n",
      " -3.38140011e-01 -8.05199146e-03  1.42598450e-02 -3.56069952e-01\n",
      "  8.31499994e-02  2.89311975e-01  5.29001653e-03 -1.11837029e-01\n",
      "  1.28127396e+00  8.09929967e-01  5.58990002e-01 -2.18623012e-01\n",
      " -1.70580015e-01  7.43115008e-01 -1.40369982e-01  2.97093987e-01\n",
      " -3.28552961e-01 -3.10106993e-01  1.80748999e-01  3.05629998e-01\n",
      "  2.17199922e-02 -4.68929976e-01 -1.95840016e-01  6.82327509e-01\n",
      " -2.89168000e-01 -7.09619969e-02  8.64340067e-01 -3.79067004e-01]\n",
      "['King', 'KING', 'king', 'KIng', 'Queen', 'QUEEN', 'queen', 'Prince', 'PRINCE', 'prince']\n"
     ]
    }
   ],
   "source": [
    "from scipy import spatial\n",
    " \n",
    "cosine_similarity = lambda x, y: 1 - spatial.distance.cosine(x, y)\n",
    " \n",
    "man = nlp.vocab['man'].vector\n",
    "woman = nlp.vocab['woman'].vector\n",
    "queen = nlp.vocab['queen'].vector\n",
    "king = nlp.vocab['king'].vector\n",
    " \n",
    "# We now need to find the closest vector in the vocabulary to the result of \"man\" - \"woman\" + \"queen\"\n",
    "maybe_queen = king - man + woman\n",
    "\n",
    "# print(\"maybe_queen : {}\".format(maybe_queen))\n",
    "\n",
    "computed_similarities = []\n",
    " \n",
    "for word in nlp.vocab:\n",
    "    if word.has_vector:  # Ignore words without vectors\n",
    "        similarity = cosine_similarity(maybe_queen, word.vector)\n",
    "        computed_similarities.append((word, similarity))\n",
    " \n",
    "computed_similarities = sorted(computed_similarities, key=lambda item: -item[1])\n",
    "\n",
    "print([w[0].text for w in computed_similarities[:10]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " **Refer this link for more on similarity : https://medium.com/better-programming/the-beginners-guide-to-similarity-matching-using-spacy-782fc2922f7c**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**References**\n",
    "- https://github.com/pyladies-bcn/spacy-workshop\n",
    "- https://nlpforhackers.io/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**For further reading**\n",
    "- https://github.com/explosion/spacy-notebooks\n",
    "- Trump dataset exploration @ https://www.kaggle.com/nirant/hitchhiker-s-guide-to-nlp-in-spacy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}