takluyver/Twitter archive.ipynb

## Twitter archive.ipynb
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inspecting archived tweets\n",
    "\n",
    "You can [download an archive of your tweets](https://twitter.com/settings/your_twitter_data) from Twitter. This may be a preliminary towards deleting old tweets, for instance.\n",
    "\n",
    "But looking at the raw Javascript files it gives you isn't terribly convenient. We can see them better with help from pandas. This notebook looks only at `tweet.js`, which contains tweets you have sent (including replies and retweets). There are other files that might be interesting too, like `following.js` and `like.js`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from datetime import datetime\n",
    "from IPython.display import HTML\n",
    "import json\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, load the data. This assumes you've downloaded your own archive and unzipped it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "817 entries\n",
      "\n",
      "Keys: contributors coordinates created_at display_text_range entities favorite_count favorited full_text geo id id_str in_reply_to_screen_name in_reply_to_status_id in_reply_to_status_id_str in_reply_to_user_id in_reply_to_user_id_str lang place retweet_count retweeted source truncated\n"
     ]
    }
   ],
   "source": [
    "ARCHIVE = \"twitter-archive-2018-08-04/tweet.js\"\n",
    "\n",
    "with open(ARCHIVE, 'r') as f:\n",
    "    data = json.loads(f.read().split('=', 1)[1])\n",
    "\n",
    "print(f\"{len(data)} entries\")\n",
    "print()\n",
    "print(\"Keys:\", *sorted(data[0].keys()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the records into pandas and do some cleanup."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame.from_records(data)\n",
    "\n",
    "# Delete some columns that don't appear to be useful to us at the moment\n",
    "df.pop('entities')\n",
    "df.pop('display_text_range')\n",
    "df.pop('id')  # IDs formatted as floats - use _str fields instead\n",
    "df.pop('in_reply_to_user_id')\n",
    "df.pop('in_reply_to_status_id')\n",
    "\n",
    "# Counts are stored as strings, with a decimal point. Why?!\n",
    "df['favorite_count'] = [int(float(n)) for n in df['favorite_count']]\n",
    "df['retweet_count'] = [int(float(n)) for n in df['retweet_count']]\n",
    "\n",
    "# Parse timestamps to a standard form. Twitter, please read https://xkcd.com/1179/ .\n",
    "df['timestamp'] = [datetime.strptime(s, '%a %b %d %H:%M:%S %z %Y') for s in df.created_at]\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summarising information\n",
    "\n",
    "Which sources do I tweet from?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>                      668\n",
       "<a href=\"https://about.twitter.com/products/tweetdeck\" rel=\"nofollow\">TweetDeck</a>      76\n",
       "<a href=\"https://dev.twitter.com/docs/tfw\" rel=\"nofollow\">Twitter for Websites</a>       67\n",
       "<a href=\"http://twitter.com/download/android\" rel=\"nofollow\">Twitter for Android</a>      2\n",
       "<a href=\"http://www.cloudhopper.com/\" rel=\"nofollow\">Cloudhopper</a>                      2\n",
       "<a href=\"http://tweetmeme.com\" rel=\"nofollow\">TweetMeme</a>                               1\n",
       "<a href=\"http://www.bbc.co.uk\" rel=\"nofollow\">BBC Website</a>                             1\n",
       "Name: source, dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.source.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How many people favourited my tweets? Am I a social media genius? (I'm not)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADh9JREFUeJzt3X+sX/Vdx/Hne+1QQrVzA29MixZCgzZU0d7AjIu5JZm5CKXLQpTaETCFmxlJZoLRakwWTYjsD9QNMcsNEGaCXBt0o4UujMzeoAmZpRtLqYh2pIY2s9dZd/USHOl8+8c9nV8bbu/3fn/wvefN85GQ3vO558fnDYcXX97nfM+JzESSVNd7Rj0BSdJwGfSSVJxBL0nFGfSSVJxBL0nFGfSSVJxBL0nFGfSSVNzAgz4ifiIiPhsRT0bErw16/5KklYluvhkbEY8CNwNzmXlNx/gk8GlgDfBwZt7f8bv3AH+emR9bbv+XXnppbtq0aeWzB9544w0uueSSnrZtg8r1WVt7Va6vTbUdOXLkW5l52bIrZuayfwE/D/wM8HLH2BrgG8CVwEXA14Etze9uAb4I/Eo3+9+2bVv26tChQz1v2waV67O29qpcX5tqA17MLjK2q9ZNZj4PnDlv+DrgeGa+lplvATPAzmb9/Zl5I7C7m/1LkoZnbR/bbgBe71g+CVwfERPAR4HvAw4utXFETAFTAGNjY8zOzvY0iYWFhZ63bYPK9Vlbe1Wur2Jt/QT928rMWWC2i/WmgWmA8fHxnJiY6Ol4s7Oz9LptG1Suz9raq3J9FWvr566bU8DlHcsbmzFJ0irST9AfBjZHxBURcRFwG7B/JTuIiB0RMT0/P9/HNCRJF9JV0EfEE8ALwNURcTIi9mTmWeAe4FngFWBfZh5bycEz80BmTq1fv36l85YkdamrHn1m7lpi/CAXuOAqSRq9kT4CwdaNJA3fwO+6WYnMPAAcGB8fv7vXfRw9Nc+de58Z4Ky6d+L+m0ZyXElaCR9qJknFGfSSVJw9ekkqbqRB7+2VkjR8tm4kqTiDXpKKM+glqTgvxkpScV6MlaTibN1IUnEGvSQVZ9BLUnFejJWk4rwYK0nF2bqRpOIMekkqzqCXpOIMekkqzqCXpOK8vVKSivP2SkkqztaNJBVn0EtScQa9JBVn0EtScQa9JBVn0EtScQa9JBXnF6YkqTi/MCVJxdm6kaTiDHpJKs6gl6TiDHpJKs6gl6TiDHpJKs6gl6TiDHpJKs6gl6TiDHpJKs5n3UhScT7rRpKKs3UjScUZ9JJUnEEvScUZ9JJUnEEvScUZ9JJUnEEvScUZ9JJUnEEvScUZ9JJUnEEvScUZ9JJUnEEvScUZ9JJU3Nph7DQiPgLcBPwg8EhmfmkYx5EkLa/rT/QR8WhEzEXEy+eNT0bEqxFxPCL2AmTmFzLzbuDjwC8PdsqSpJVYSevmMWCycyAi1gAPATcCW4BdEbGlY5Xfa34vSRqRroM+M58Hzpw3fB1wPDNfy8y3gBlgZyz6FPDFzPzq4KYrSVqpyMzuV47YBDydmdc0y7cCk5l5V7N8O3A98E/AHcBh4KXM/Ozb7GsKmAIYGxvbNjMz01MBc2fmOf1mT5v2beuG4b8CcWFhgXXr1g39OKNgbe1Vub421bZ9+/YjmTm+3HpDuRibmZ8BPrPMOtPANMD4+HhOTEz0dKwHH3+KB44OpYxlndg9MfRjzM7O0uvfm9XO2tqrcn0Va+v39spTwOUdyxubMUnSKtFv0B8GNkfEFRFxEXAbsL/bjSNiR0RMz8/P9zkNSdJSVnJ75RPAC8DVEXEyIvZk5lngHuBZ4BVgX2Ye63afmXkgM6fWrx9+r1uS3q26bm5n5q4lxg8CBwc2I0nSQPkIBEkqbqRBb49ekoZvpEFvj16Shs/WjSQVZ9BLUnH26CWpOHv0klScrRtJKs6gl6Ti7NFLUnH26CWpOFs3klScQS9JxRn0klScQS9JxXnXjSQV5103klScrRtJKs6gl6TiDHpJKs6gl6TivOtGkorzrhtJKs7WjSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQV5xemJKk4vzAlScXZupGk4gx6SSrOoJek4gx6SSrOoJek4gx6SSrOoJek4gx6SSpu7agn0Gab9j4z9GPcu/Usd553nBP33zT040qqw0/0klScz7qRpOJ81o0kFWfrRpKKM+glqTiDXpKKM+glqTiDXpKKM+glqTiDXpKKM+glqTiDXpKKM+glqTiDXpKKM+glqTiDXpKKM+glqTiDXpKKG3jQR8SVEfFIRDw56H1Lklauq6CPiEcjYi4iXj5vfDIiXo2I4xGxFyAzX8vMPcOYrCRp5br9RP8YMNk5EBFrgIeAG4EtwK6I2DLQ2UmS+tZV0Gfm88CZ84avA443n+DfAmaAnQOenySpT5GZ3a0YsQl4OjOvaZZvBSYz865m+XbgeuCTwH3Ah4GHM/MPl9jfFDAFMDY2tm1mZqanAubOzHP6zZ42bYWxi1lV9W3dMLj3+y4sLLBu3bqB7W81qVwb1K6vTbVt3779SGaOL7fe2kEfODP/Hfh4F+tNA9MA4+PjOTEx0dPxHnz8KR44OvAyVo17t55dVfWd2D0xsH3Nzs7S6z/31a5ybVC7voq19XPXzSng8o7ljc2YJGkV6SfoDwObI+KKiLgIuA3Yv5IdRMSOiJien5/vYxqSpAvp9vbKJ4AXgKsj4mRE7MnMs8A9wLPAK8C+zDy2koNn5oHMnFq/fnB9X0nS/9dV8zczdy0xfhA4ONAZSZIGaqSPQLB1I0nDN9Kgt3UjScPnQ80kqTiDXpKKs0cvScXZo5ek4mzdSFJxBr0kFWfQS1JxXoyVpOK8GCtJxdm6kaTiDHpJKs6gl6TivBgrScV5MVaSirN1I0nFGfSSVJxBL0nFGfSSVJxBL0nFrR3lwSNiB7DjqquuGuU01AKb9j4zkuOeuP+mkRxXGiRvr5Sk4mzdSFJxBr0kFWfQS1JxBr0kFWfQS1JxBr0kFedjiiWpOO+jl6TibN1IUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQV5ztjtSKDfHfrvVvPcueI3gUrvZv4rBtJKs7WjSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVN/AXj0TEJcCfAW8Bs5n5+KCPIUnqXlef6CPi0YiYi4iXzxufjIhXI+J4ROxthj8KPJmZdwO3DHi+kqQV6rZ18xgw2TkQEWuAh4AbgS3ArojYAmwEXm9W++5gpilJ6lVXQZ+ZzwNnzhu+Djiema9l5lvADLATOMli2He9f0nS8ERmdrdixCbg6cy8plm+FZjMzLua5duB64HfBv4U+G/g75bq0UfEFDAFMDY2tm1mZqanAubOzHP6zZ42bYWxiylbXxtq27qht/cZLywssG7dur6OffTUfF/b96qbmgdR39sZVc2d3unzstdzDGD79u1HMnN8ufUGfjE2M98AfrWL9aaBaYDx8fGcmJjo6XgPPv4UDxwdeBmrxr1bz5atrw21ndg90dN2s7Oz9HpOn3Pn3mf62r5X3dQ8iPrezqhq7vROn5e9nmMr0U9r5RRwecfyxmZMkrSK9BP0h4HNEXFFRFwE3AbsX8kOImJHREzPz4/+f9ckqapub698AngBuDoiTkbEnsw8C9wDPAu8AuzLzGMrOXhmHsjMqfXre+9RSZIurKtGVGbuWmL8IHBwoDOSJA2Utz9KUnEjDXp79JI0fCMNenv0kjR8tm4kqbiuvxk71ElE/BvwLz1ufinwrQFOZ7WpXJ+1tVfl+tpU249l5mXLrbQqgr4fEfFiN18BbqvK9Vlbe1Wur2Jttm4kqTiDXpKKqxD006OewJBVrs/a2qtyfeVqa32PXpJ0YRU+0UuSLqDVQb/EO2tb6e3eyxsR74+I5yLin5s/f2iUc+xVRFweEYci4h8i4lhEfKIZr1Lf90fE30fE15v6fr8ZvyIivtKcn3/ZPOW1lSJiTUR8LSKebpZL1BYRJyLiaES8FBEvNmMlzstOrQ36C7yztq0e47z38gJ7gS9n5mbgy81yG50F7s3MLcAHgV9v/llVqe87wA2Z+VPAtcBkRHwQ+BTwx5l5FfAfwJ4RzrFfn2DxKbXnVKpte2Ze23FLZZXz8ntaG/Qs/c7aVlrivbw7gc81P38O+Mg7OqkBycxvZuZXm5//i8XA2ECd+jIzF5rF9zZ/JXAD8GQz3tr6ImIjcBPwcLMcFKltCSXOy05tDvoNwOsdyyebsUrGMvObzc//CoyNcjKD0Lx7+KeBr1Covqa18RIwBzwHfAP4dvPeBmj3+fknwG8B/9Msf4A6tSXwpYg40rzHGgqdl+es7hd26nsyMyOi1bdIRcQ64K+A38jM/1z8YLio7fVl5neBayPifcDngR8f8ZQGIiJuBuYy80hETIx6PkPwocw8FRE/DDwXEf/Y+cu2n5fntPkT/bvhnbWnI+JHAJo/50Y8n55FxHtZDPnHM/Ovm+Ey9Z2Tmd8GDgE/C7wvIs59mGrr+flzwC0RcYLF9ugNwKepURuZear5c47F/0BfR8Hzss1B3/c7a1tgP3BH8/MdwFMjnEvPmp7uI8ArmflHHb+qUt9lzSd5IuJi4MMsXoc4BNzarNbK+jLzdzJzY2ZuYvHfsb/JzN0UqC0iLomIHzj3M/ALwMsUOS87tfoLUxHxiyz2D9cAj2bmfSOeUs+a9/JOsPjkvNPAJ4EvAPuAH2Xx6Z6/lJnnX7Bd9SLiQ8DfAkf5vz7v77LYp69Q30+yeNFuDYsfnvZl5h9ExJUsfgp+P/A14GOZ+Z3RzbQ/TevmNzPz5gq1NTV8vllcC/xFZt4XER+gwHnZqdVBL0laXptbN5KkLhj0klScQS9JxRn0klScQS9JxRn0klScQS9JxRn0klTc/wLqPHawoqjR4wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax = df.favorite_count.hist()\n",
    "ax.set_yscale('log')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When have I been most active on twitter?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f05cd2361d0>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFD1JREFUeJzt3XGMnPV95/H3tzgkhm1tiNMtZ1tdrkFUHNtr8B6lJRftxr0eCVHMH1EE4lpTUVmn0h5tHCXOVWrUP5BIG5qmurtIbqBxLykbwiUHhaQN57IXVSq0NiFZA0nxEZPgM3Z6AaeboiZuv/fHPG5Gzq5n53me2Xny4/2SRjvzPM/8ns/OM/PZZ559ZjcyE0lSuX5g3AEkSaNl0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCDSz6iLgrIk5ExKFl5u2OiIyITdXtiIjfj4jDEfHFiLhiFKElSau3mj36jwDXnDkxIrYCPwd8tW/ym4BLqssu4EPNI0qSmlg3aIHM/FxETC0z6wPAu4D7+qbtAP4oex+3fSQiNkbERZl57Gzr2LRpU05NLbeKtfGtb32L888/f2zrX4m5htPVXNDdbOYaTtdyHTx48G8z8zWDlhtY9MuJiB3A0cz8QkT0z9oMfK3v9nPVtO8p+ojYRW+vn8nJSd7//vfXidKKpaUlJiYmxrb+lZhrOF3NBd3NZq7hdC3X3Nzcs6taMDMHXoAp4FB1/TzgUWBDdfsIsKm6/gDw+r777QdmBo2/bdu2HKeHH354rOtfibmG09Vcmd3NZq7hdC0XcCBX0eF19uh/DLgYOL03vwV4LCKuBI4CW/uW3VJNkySNydCnV2bmYmb+cGZOZeYUvcMzV2Tm88D9wC9UZ99cBZzMAcfnJUmjtZrTK+8G/hK4NCKei4ibz7L4p4FngMPAHwC/3EpKSVJtqznr5oYB86f6ridwS/NYkqS2+MlYSSqcRS9JhbPoJalwFr0kFa7WJ2Oll5OpPQ82HmP39CluGnKcI7df23i9ErhHL0nFs+glqXAWvSQVzqKXpMJZ9JJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYUbWPQRcVdEnIiIQ33TficivhQRX4yIT0XExr5574mIwxHx5Yj496MKLklandXs0X8EuOaMaQ8Bl2fmTwB/A7wHICIuA64H/lV1n/8WEee0llaSNLSBRZ+ZnwO+cca0z2bmqermI8CW6voOYD4z/yEzvwIcBq5sMa8kaUiRmYMXipgCHsjMy5eZ9yfAxzPzoxHxX4BHMvOj1bw7gc9k5r3L3G8XsAtgcnJy2/z8fJPvo5GlpSUmJibGtv6VmGs4o8q1ePRk4zEm18Pxl4a7z/TmDY3XO8hyj1kb329dp7/nl9tzrK65ubmDmTkzaLl1TVYSEb8BnAI+Nux9M3MvsBdgZmYmZ2dnm0RpZGFhgXGufyXmGs6oct2058HGY+yePsUdi8O93I7cONt4vYMs95i18f3Wdfp7frk9x0atdtFHxE3AW4Dt+d23BUeBrX2LbammSZLGpNbplRFxDfAu4K2Z+fd9s+4Hro+IV0bExcAlwF81jylJqmvgHn1E3A3MApsi4jngvfTOsnkl8FBEQO+4/H/MzCci4h7gSXqHdG7JzH8cVXhJ0mADiz4zb1hm8p1nWf424LYmoaQzTa3iuPHu6VNjPb4sdZWfjJWkwln0klQ4i16SCmfRS1LhLHpJKpxFL0mFs+glqXAWvSQVzqKXpMJZ9JJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYUbWPQRcVdEnIiIQ33TLoyIhyLi6errBdX0iIjfj4jDEfHFiLhilOElSYOtZo/+I8A1Z0zbA+zPzEuA/dVtgDcBl1SXXcCH2okpSaprYNFn5ueAb5wxeQewr7q+D7iub/ofZc8jwMaIuKitsJKk4UVmDl4oYgp4IDMvr26/mJkbq+sBvJCZGyPiAeD2zPyLat5+4N2ZeWCZMXfR2+tncnJy2/z8fDvfUQ1LS0tMTEyMbf0rMdd3LR49OXCZyfVw/KU1CFNDnWzTmzeMJkyf5bblah7rUTn9PfvcX525ubmDmTkzaLl1TVeUmRkRg39afO/99gJ7AWZmZnJ2drZplNoWFhYY5/pXYq7vumnPgwOX2T19ijsWGz+lR6JOtiM3zo4mTJ/ltuVqHutROf09+9xvV91XxfGIuCgzj1WHZk5U048CW/uW21JNG5mpFp6Uu6dP1XpyH7n92sbrlqRRq3t65f3Azur6TuC+vum/UJ19cxVwMjOPNcwoSWpg4B59RNwNzAKbIuI54L3A7cA9EXEz8Czw9mrxTwNvBg4Dfw/84ggyS5KGMLDoM/OGFWZtX2bZBG5pGkqS1B4/GStJhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuG6+YdBJL0snf6TJnX/LEldpf85E/foJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYWz6CWpcBa9JBXOopekwln0klQ4i16SCmfRS1LhGhV9RPx6RDwREYci4u6IeFVEXBwRj0bE4Yj4eESc21ZYSdLwahd9RGwG/hMwk5mXA+cA1wPvAz6Qma8FXgBubiOoJKmepodu1gHrI2IdcB5wDHgjcG81fx9wXcN1SJIaiMysf+eIW4HbgJeAzwK3Ao9Ue/NExFbgM9Ue/5n33QXsApicnNw2Pz9fK8Pi0ZP1wveZXA/HXxr+ftObNzRe99ksLS0xMTEx0nXUMY5cq9nOdbfjWqiTbdTPL1h+W7bxmmpqrbflah/rrr0m5+bmDmbmzKDlav8rwYi4ANgBXAy8CHwCuGa198/MvcBegJmZmZydna2Vo41/N7Z7+hR3LA7/UBy5cbbxus9mYWGBuo/LKI0j12q2c93tuBbqZBv18wuW35Zr+S/8VrLW23K1j3VXX5ODNDl087PAVzLz65n5HeCTwNXAxupQDsAW4GjDjJKkBpoU/VeBqyLivIgIYDvwJPAw8LZqmZ3Afc0iSpKaqF30mfkovV+6PgYsVmPtBd4NvCMiDgOvBu5sIackqaZGB8Ey873Ae8+Y/AxwZZNxJWktTa3y9xK7p0+1/juMI7df2+p4y/GTsZJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYWz6CWpcBa9JBXOopekwln0klS4tfs365KGstr/etTEKP5jkrrHPXpJKpxFL0mF89BNA6N+a73S2+q1+GfCksrRaI8+IjZGxL0R8aWIeCoifjoiLoyIhyLi6errBW2FlSQNr+mhmw8Cf5qZPw78a+ApYA+wPzMvAfZXtyVJY1K76CNiA/AG4E6AzPx2Zr4I7AD2VYvtA65rGlKSVF+TPfqLga8DfxgRn4+ID0fE+cBkZh6rlnkemGwaUpJUX2RmvTtGzACPAFdn5qMR8UHgm8CvZubGvuVeyMzvOU4fEbuAXQCTk5Pb5ufna+VYPHqy1v36Ta6H4y81HqZ1K+Wa3rxh7cP0WVpaYmJiYk3XuZrt3NXtCN3NZq7hjCJXk9fz3NzcwcycGbRck6L/EeCRzJyqbv9besfjXwvMZuaxiLgIWMjMS8821szMTB44cKBWjjbOfNk9fYo7Frt3AtJKucZ91s3CwgKzs7Nrus7VbOeubkfobjZzDWcUuZq8niNiVUVf+9BNZj4PfC0iTpf4duBJ4H5gZzVtJ3Bf3XVIkppr+qPpV4GPRcS5wDPAL9L74XFPRNwMPAu8veE6JEkNNCr6zHwcWO5tw/Ym40qS2uOfQJCkwln0klQ4i16SCmfRS1LhLHpJKpxFL0mFs+glqXAWvSQVzqKXpMJZ9JJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYVrXPQRcU5EfD4iHqhuXxwRj0bE4Yj4eESc2zymJKmuNvbobwWe6rv9PuADmfla4AXg5hbWIUmqqVHRR8QW4Frgw9XtAN4I3Fstsg+4rsk6JEnNNN2j/z3gXcA/VbdfDbyYmaeq288BmxuuQ5LUQGRmvTtGvAV4c2b+ckTMAu8EbgIeqQ7bEBFbgc9k5uXL3H8XsAtgcnJy2/z8fK0ci0dP1rpfv8n1cPylxsO0bqVc05s3rH2YPktLS0xMTKzpOleznbu6HaG72cw1nFHkavJ6npubO5iZM4OWW1d7DXA18NaIeDPwKuCHgA8CGyNiXbVXvwU4utydM3MvsBdgZmYmZ2dna4W4ac+Dte7Xb/f0Ke5YbPJQjMZKuY7cOLv2YfosLCxQd3vVtZrt3NXtCN3NZq7hjCLXWryeax+6ycz3ZOaWzJwCrgf+PDNvBB4G3lYtthO4r3FKSVJtoziP/t3AOyLiML1j9neOYB2SpFVq5T1IZi4AC9X1Z4Ar2xhXktScn4yVpMJZ9JJUOItekgrXvfOX1GlTex5k9/SpVk5rlbQ23KOXpMJZ9JJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYWz6CWpcBa9JBXOopekwln0klS42kUfEVsj4uGIeDIinoiIW6vpF0bEQxHxdPX1gvbiSpKG1WSP/hSwOzMvA64CbomIy4A9wP7MvATYX92WJI1J7aLPzGOZ+Vh1/e+Ap4DNwA5gX7XYPuC6piElSfW1cow+IqaA1wGPApOZeaya9Tww2cY6JEn1RGY2GyBiAvjfwG2Z+cmIeDEzN/bNfyEzv+c4fUTsAnYBTE5Obpufn6+1/sWjJ+sF7zO5Ho6/1HiY1q2Ua3rzhrUPU1k8evL77vHqgq5mM9dwRpGryet5bm7uYGbODFquUdFHxCuAB4A/y8zfraZ9GZjNzGMRcRGwkJmXnm2cmZmZPHDgQK0MU3serHW/frunT3HH4rrG47RtpVxHbr92DGl6pvY8+H33eHVBV7OZazijyNXk9RwRqyr6JmfdBHAn8NTpkq/cD+ysru8E7qu7DklSc01+NF0N/DywGBGPV9P+M3A7cE9E3Aw8C7y9WURJUhO1iz4z/wKIFWZvrzuuJKldfjJWkgpn0UtS4Sx6SSpc985f0kBtnFIq6eXDPXpJKpxFL0mFs+glqXAWvSQVzqKXpMJZ9JJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYUbWdFHxDUR8eWIOBwRe0a1HknS2Y2k6CPiHOC/Am8CLgNuiIjLRrEuSdLZjWqP/krgcGY+k5nfBuaBHSNalyTpLEZV9JuBr/Xdfq6aJklaY5GZ7Q8a8Tbgmsz8per2zwM/lZm/0rfMLmBXdfNS4MutB1m9TcDfjnH9KzHXcLqaC7qbzVzD6VquH83M1wxaaN2IVn4U2Np3e0s17Z9l5l5g74jWP5SIOJCZM+POcSZzDaeruaC72cw1nK7mGmRUh27+GrgkIi6OiHOB64H7R7QuSdJZjGSPPjNPRcSvAH8GnAPclZlPjGJdkqSzG9WhGzLz08CnRzV+yzpxCGkZ5hpOV3NBd7OZazhdzXVWI/llrCSpO/wTCJJUuCKLPiK2RsTDEfFkRDwREbdW0y+MiIci4unq6wXV9B+PiL+MiH+IiHeeMdZdEXEiIg51JddK43Qk26si4q8i4gvVOL/VhVx9450TEZ+PiAe6kisijkTEYkQ8HhEHOpRrY0TcGxFfioinIuKnx50rIi6tHqfTl29GxK+NO1c179erMQ5FxN0R8aq6uVqXmcVdgIuAK6rrPwj8Db0/xfDbwJ5q+h7gfdX1Hwb+DXAb8M4zxnoDcAVwqCu5VhqnI9kCmKiuvwJ4FLhq3Ln6xnsH8MfAA114vKp5R4BNHXzu7wN+qbp+LrCxC7n6xjwHeJ7eueTjft5vBr4CrK9u3wPc1MY2beNS5B59Zh7LzMeq638HPEVvQ+yg9+Sl+npdtcyJzPxr4DvLjPU54BtdynWWcbqQLTNzqbr5iupS+xdBbW7LiNgCXAt8uG6eUeRqU1u5ImIDvZ2cO6vlvp2ZL4471xm2A/8nM5/tSK51wPqIWAecB/zfurnaVmTR94uIKeB19PYsJzPzWDXreWByTLFay3XGOJ3IVh0eeRw4ATyUma1ka+Ex+z3gXcA/tZGnxVwJfDYiDkbvE+NdyHUx8HXgD6tDXR+OiPM7kKvf9cDdbWRqmiszjwLvB74KHANOZuZn28rWVNFFHxETwP8Afi0zv9k/L3vvr8ZyylFbuc42zjizZeY/ZuZP0vtE9JURcfm4c0XEW4ATmXmwaZY2c1Ven5lX0Ptrr7dExBs6kGsdvUOWH8rM1wHfoncIY9y5To9zLvBW4BNNM7WRqzqGv4PeD8h/AZwfEf+hjWxtKLboI+IV9DbcxzLzk9Xk4xFxUTX/Inp7nN+XuVYYpxPZTqve6j8MXNOBXFcDb42II/T+muobI+KjHch1em+QzDwBfIreX38dd67ngOf63o3dS6/4x53rtDcBj2Xm8SaZWsz1s8BXMvPrmfkd4JPAzzTN1pYiiz4igt6xxacy83f7Zt0P7Kyu7wTu+37MdZZxupDtNRGxsbq+Hvh3wJfGnSsz35OZWzJzit5b/j/PzNp7XC0+XudHxA+evg78HFD7DK8WH6/nga9FxKXVpO3Ak+PO1ecGWjhs02KurwJXRcR51Zjb6R3v74YzfztbwgV4Pb23Wl8EHq8ubwZeDewHngb+F3BhtfyP0NuD+SbwYnX9h6p5d9M75vadavrN48610jhdeMyAnwA+X41zCPjNLuQ6Y8xZmp9109bj9S+BL1SXJ4Df6EKuat5PAgeqsf4ncEFHcp0P/D9gQ8e64rfo7dQcAv478Mqm+dq6+MlYSSpckYduJEnfZdFLUuEsekkqnEUvSYWz6CWpcBa9JBXOopekwln0klS4/w+Cg+PlkZBqNwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.timestamp.hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View all tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Sort first to get tweets in reverse chronological order\n",
    "for month, group in df.sort_values('timestamp', ascending=False).groupby(df.timestamp.dt.to_period('M'), sort=False):\n",
    "    html = f'<h2>{month}</h2>\\n'\n",
    "    table_rows = []\n",
    "    for ix, tweet in group.iterrows():\n",
    "        reply = \"↩\" if tweet['in_reply_to_screen_name'] else \" \"\n",
    "        table_rows.append(f\"<tr><td>{reply}</td><td>{tweet['full_text']}</td></tr>\\n\")\n",
    "    html += '<table>\\n' + ''.join(table_rows) + '</table>'\n",
    "    display(HTML(html))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
	{
	"cells": [
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"# Inspecting archived tweets\n",
	"\n",
	"You can [download an archive of your tweets](https://twitter.com/settings/your_twitter_data) from Twitter. This may be a preliminary towards deleting old tweets, for instance.\n",
	"\n",
	"But looking at the raw Javascript files it gives you isn't terribly convenient. We can see them better with help from pandas. This notebook looks only at `tweet.js`, which contains tweets you have sent (including replies and retweets). There are other files that might be interesting too, like `following.js` and `like.js`."
	]
	},
	{
	"cell_type": "code",
	"execution_count": 1,
	"metadata": {},
	"outputs": [],
	"source": [
	"%matplotlib inline\n",
	"from datetime import datetime\n",
	"from IPython.display import HTML\n",
	"import json\n",
	"import pandas as pd"
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"First, load the data. This assumes you've downloaded your own archive and unzipped it:"
	]
	},
	{
	"cell_type": "code",
	"execution_count": 2,
	"metadata": {},
	"outputs": [
	{
	"name": "stdout",
	"output_type": "stream",
	"text": [
	"817 entries\n",
	"\n",
	"Keys: contributors coordinates created_at display_text_range entities favorite_count favorited full_text geo id id_str in_reply_to_screen_name in_reply_to_status_id in_reply_to_status_id_str in_reply_to_user_id in_reply_to_user_id_str lang place retweet_count retweeted source truncated\n"
	]
	}
	],
	"source": [
	"ARCHIVE = \"twitter-archive-2018-08-04/tweet.js\"\n",
	"\n",
	"with open(ARCHIVE, 'r') as f:\n",
	" data = json.loads(f.read().split('=', 1)[1])\n",
	"\n",
	"print(f\"{len(data)} entries\")\n",
	"print()\n",
	"print(\"Keys:\", *sorted(data[0].keys()))"
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"Load the records into pandas and do some cleanup."
	]
	},
	{
	"cell_type": "code",
	"execution_count": null,
	"metadata": {},
	"outputs": [],
	"source": [
	"df = pd.DataFrame.from_records(data)\n",
	"\n",
	"# Delete some columns that don't appear to be useful to us at the moment\n",
	"df.pop('entities')\n",
	"df.pop('display_text_range')\n",
	"df.pop('id') # IDs formatted as floats - use _str fields instead\n",
	"df.pop('in_reply_to_user_id')\n",
	"df.pop('in_reply_to_status_id')\n",
	"\n",
	"# Counts are stored as strings, with a decimal point. Why?!\n",
	"df['favorite_count'] = [int(float(n)) for n in df['favorite_count']]\n",
	"df['retweet_count'] = [int(float(n)) for n in df['retweet_count']]\n",
	"\n",
	"# Parse timestamps to a standard form. Twitter, please read https://xkcd.com/1179/ .\n",
	"df['timestamp'] = [datetime.strptime(s, '%a %b %d %H:%M:%S %z %Y') for s in df.created_at]\n",
	"\n",
	"df.head()"
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"## Summarising information\n",
	"\n",
	"Which sources do I tweet from?"
	]
	},
	{
	"cell_type": "code",
	"execution_count": 4,
	"metadata": {},
	"outputs": [
	{
	"data": {
	"text/plain": [
	"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a> 668\n",
	"<a href=\"https://about.twitter.com/products/tweetdeck\" rel=\"nofollow\">TweetDeck</a> 76\n",
	"<a href=\"https://dev.twitter.com/docs/tfw\" rel=\"nofollow\">Twitter for Websites</a> 67\n",
	"<a href=\"http://twitter.com/download/android\" rel=\"nofollow\">Twitter for Android</a> 2\n",
	"<a href=\"http://www.cloudhopper.com/\" rel=\"nofollow\">Cloudhopper</a> 2\n",
	"<a href=\"http://tweetmeme.com\" rel=\"nofollow\">TweetMeme</a> 1\n",
	"<a href=\"http://www.bbc.co.uk\" rel=\"nofollow\">BBC Website</a> 1\n",
	"Name: source, dtype: int64"
	]
	},
	"execution_count": 4,
	"metadata": {},
	"output_type": "execute_result"
	}
	],
	"source": [
	"df.source.value_counts()"
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"How many people favourited my tweets? Am I a social media genius? (I'm not)"
	]
	},
	{
	"cell_type": "code",
	"execution_count": 5,
	"metadata": {
	"scrolled": true
	},
	"outputs": [
	{
	"data": {
	"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADh9JREFUeJzt3X+sX/Vdx/Hne+1QQrVzA29MixZCgzZU0d7AjIu5JZm5CKXLQpTaETCFmxlJZoLRakwWTYjsD9QNMcsNEGaCXBt0o4UujMzeoAmZpRtLqYh2pIY2s9dZd/USHOl8+8c9nV8bbu/3fn/wvefN85GQ3vO558fnDYcXX97nfM+JzESSVNd7Rj0BSdJwGfSSVJxBL0nFGfSSVJxBL0nFGfSSVJxBL0nFGfSSVNzAgz4ifiIiPhsRT0bErw16/5KklYluvhkbEY8CNwNzmXlNx/gk8GlgDfBwZt7f8bv3AH+emR9bbv+XXnppbtq0aeWzB9544w0uueSSnrZtg8r1WVt7Va6vTbUdOXLkW5l52bIrZuayfwE/D/wM8HLH2BrgG8CVwEXA14Etze9uAb4I/Eo3+9+2bVv26tChQz1v2waV67O29qpcX5tqA17MLjK2q9ZNZj4PnDlv+DrgeGa+lplvATPAzmb9/Zl5I7C7m/1LkoZnbR/bbgBe71g+CVwfERPAR4HvAw4utXFETAFTAGNjY8zOzvY0iYWFhZ63bYPK9Vlbe1Wur2Jt/QT928rMWWC2i/WmgWmA8fHxnJiY6Ol4s7Oz9LptG1Suz9raq3J9FWvr566bU8DlHcsbmzFJ0irST9AfBjZHxBURcRFwG7B/JTuIiB0RMT0/P9/HNCRJF9JV0EfEE8ALwNURcTIi9mTmWeAe4FngFWBfZh5bycEz80BmTq1fv36l85YkdamrHn1m7lpi/CAXuOAqSRq9kT4CwdaNJA3fwO+6WYnMPAAcGB8fv7vXfRw9Nc+de58Z4Ky6d+L+m0ZyXElaCR9qJknFGfSSVJw9ekkqbqRB7+2VkjR8tm4kqTiDXpKKM+glqTgvxkpScV6MlaTibN1IUnEGvSQVZ9BLUnFejJWk4rwYK0nF2bqRpOIMekkqzqCXpOIMekkqzqCXpOK8vVKSivP2SkkqztaNJBVn0EtScQa9JBVn0EtScQa9JBVn0EtScQa9JBXnF6YkqTi/MCVJxdm6kaTiDHpJKs6gl6TiDHpJKs6gl6TiDHpJKs6gl6TiDHpJKs6gl6TiDHpJKs5n3UhScT7rRpKKs3UjScUZ9JJUnEEvScUZ9JJUnEEvScUZ9JJUnEEvScUZ9JJUnEEvScUZ9JJUnEEvScUZ9JJUnEEvScUZ9JJU3Nph7DQiPgLcBPwg8EhmfmkYx5EkLa/rT/QR8WhEzEXEy+eNT0bEqxFxPCL2AmTmFzLzbuDjwC8PdsqSpJVYSevmMWCycyAi1gAPATcCW4BdEbGlY5Xfa34vSRqRroM+M58Hzpw3fB1wPDNfy8y3gBlgZyz6FPDFzPzq4KYrSVqpyMzuV47YBDydmdc0y7cCk5l5V7N8O3A98E/AHcBh4KXM/Ozb7GsKmAIYGxvbNjMz01MBc2fmOf1mT5v2beuG4b8CcWFhgXXr1g39OKNgbe1Vub421bZ9+/YjmTm+3HpDuRibmZ8BPrPMOtPANMD4+HhOTEz0dKwHH3+KB44OpYxlndg9MfRjzM7O0uvfm9XO2tqrcn0Va+v39spTwOUdyxubMUnSKtFv0B8GNkfEFRFxEXAbsL/bjSNiR0RMz8/P9zkNSdJSVnJ75RPAC8DVEXEyIvZk5lngHuBZ4BVgX2Ye63afmXkgM6fWrx9+r1uS3q26bm5n5q4lxg8CBwc2I0nSQPkIBEkqbqRBb49ekoZvpEFvj16Shs/WjSQVZ9BLUnH26CWpOHv0klScrRtJKs6gl6Ti7NFLUnH26CWpOFs3klScQS9JxRn0klScQS9JxXnXjSQV5103klScrRtJKs6gl6TiDHpJKs6gl6TivOtGkorzrhtJKs7WjSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQV5xemJKk4vzAlScXZupGk4gx6SSrOoJek4gx6SSrOoJek4gx6SSrOoJek4gx6SSpu7agn0Gab9j4z9GPcu/Usd553nBP33zT040qqw0/0klScz7qRpOJ81o0kFWfrRpKKM+glqTiDXpKKM+glqTiDXpKKM+glqTiDXpKKM+glqTiDXpKKM+glqTiDXpKKM+glqTiDXpKKM+glqTiDXpKKG3jQR8SVEfFIRDw56H1Lklauq6CPiEcjYi4iXj5vfDIiXo2I4xGxFyAzX8vMPcOYrCRp5br9RP8YMNk5EBFrgIeAG4EtwK6I2DLQ2UmS+tZV0Gfm88CZ84avA443n+DfAmaAnQOenySpT5GZ3a0YsQl4OjOvaZZvBSYz865m+XbgeuCTwH3Ah4GHM/MPl9jfFDAFMDY2tm1mZqanAubOzHP6zZ42bYWxi1lV9W3dMLj3+y4sLLBu3bqB7W81qVwb1K6vTbVt3779SGaOL7fe2kEfODP/Hfh4F+tNA9MA4+PjOTEx0dPxHnz8KR44OvAyVo17t55dVfWd2D0xsH3Nzs7S6z/31a5ybVC7voq19XPXzSng8o7ljc2YJGkV6SfoDwObI+KKiLgIuA3Yv5IdRMSOiJien5/vYxqSpAvp9vbKJ4AXgKsj4mRE7MnMs8A9wLPAK8C+zDy2koNn5oHMnFq/fnB9X0nS/9dV8zczdy0xfhA4ONAZSZIGaqSPQLB1I0nDN9Kgt3UjScPnQ80kqTiDXpKKs0cvScXZo5ek4mzdSFJxBr0kFWfQS1JxXoyVpOK8GCtJxdm6kaTiDHpJKs6gl6TivBgrScV5MVaSirN1I0nFGfSSVJxBL0nFGfSSVJxBL0nFrR3lwSNiB7DjqquuGuU01AKb9j4zkuOeuP+mkRxXGiRvr5Sk4mzdSFJxBr0kFWfQS1JxBr0kFWfQS1JxBr0kFedjiiWpOO+jl6TibN1IUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQV5ztjtSKDfHfrvVvPcueI3gUrvZv4rBtJKs7WjSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVZ9BLUnEGvSQVN/AXj0TEJcCfAW8Bs5n5+KCPIUnqXlef6CPi0YiYi4iXzxufjIhXI+J4ROxthj8KPJmZdwO3DHi+kqQV6rZ18xgw2TkQEWuAh4AbgS3ArojYAmwEXm9W++5gpilJ6lVXQZ+ZzwNnzhu+Djiema9l5lvADLATOMli2He9f0nS8ERmdrdixCbg6cy8plm+FZjMzLua5duB64HfBv4U+G/g75bq0UfEFDAFMDY2tm1mZqanAubOzHP6zZ42bYWxiylbXxtq27qht/cZLywssG7dur6OffTUfF/b96qbmgdR39sZVc2d3unzstdzDGD79u1HMnN8ufUGfjE2M98AfrWL9aaBaYDx8fGcmJjo6XgPPv4UDxwdeBmrxr1bz5atrw21ndg90dN2s7Oz9HpOn3Pn3mf62r5X3dQ8iPrezqhq7vROn5e9nmMr0U9r5RRwecfyxmZMkrSK9BP0h4HNEXFFRFwE3AbsX8kOImJHREzPz4/+f9ckqapub698AngBuDoiTkbEnsw8C9wDPAu8AuzLzGMrOXhmHsjMqfXre+9RSZIurKtGVGbuWmL8IHBwoDOSJA2Utz9KUnEjDXp79JI0fCMNenv0kjR8tm4kqbiuvxk71ElE/BvwLz1ufinwrQFOZ7WpXJ+1tVfl+tpU249l5mXLrbQqgr4fEfFiN18BbqvK9Vlbe1Wur2Jttm4kqTiDXpKKqxD006OewJBVrs/a2qtyfeVqa32PXpJ0YRU+0UuSLqDVQb/EO2tb6e3eyxsR74+I5yLin5s/f2iUc+xVRFweEYci4h8i4lhEfKIZr1Lf90fE30fE15v6fr8ZvyIivtKcn3/ZPOW1lSJiTUR8LSKebpZL1BYRJyLiaES8FBEvNmMlzstOrQ36C7yztq0e47z38gJ7gS9n5mbgy81yG50F7s3MLcAHgV9v/llVqe87wA2Z+VPAtcBkRHwQ+BTwx5l5FfAfwJ4RzrFfn2DxKbXnVKpte2Ze23FLZZXz8ntaG/Qs/c7aVlrivbw7gc81P38O+Mg7OqkBycxvZuZXm5//i8XA2ECd+jIzF5rF9zZ/JXAD8GQz3tr6ImIjcBPwcLMcFKltCSXOy05tDvoNwOsdyyebsUrGMvObzc//CoyNcjKD0Lx7+KeBr1Covqa18RIwBzwHfAP4dvPeBmj3+fknwG8B/9Msf4A6tSXwpYg40rzHGgqdl+es7hd26nsyMyOi1bdIRcQ64K+A38jM/1z8YLio7fVl5neBayPifcDngR8f8ZQGIiJuBuYy80hETIx6PkPwocw8FRE/DDwXEf/Y+cu2n5fntPkT/bvhnbWnI+JHAJo/50Y8n55FxHtZDPnHM/Ovm+Ey9Z2Tmd8GDgE/C7wvIs59mGrr+flzwC0RcYLF9ugNwKepURuZear5c47F/0BfR8Hzss1B3/c7a1tgP3BH8/MdwFMjnEvPmp7uI8ArmflHHb+qUt9lzSd5IuJi4MMsXoc4BNzarNbK+jLzdzJzY2ZuYvHfsb/JzN0UqC0iLomIHzj3M/ALwMsUOS87tfoLUxHxiyz2D9cAj2bmfSOeUs+a9/JOsPjkvNPAJ4EvAPuAH2Xx6Z6/lJnnX7Bd9SLiQ8DfAkf5vz7v77LYp69Q30+yeNFuDYsfnvZl5h9ExJUsfgp+P/A14GOZ+Z3RzbQ/TevmNzPz5gq1NTV8vllcC/xFZt4XER+gwHnZqdVBL0laXptbN5KkLhj0klScQS9JxRn0klScQS9JxRn0klScQS9JxRn0klTc/wLqPHawoqjR4wAAAABJRU5ErkJggg==\n",
	"text/plain": [
	"<Figure size 432x288 with 1 Axes>"
	]
	},
	"metadata": {},
	"output_type": "display_data"
	}
	],
	"source": [
	"ax = df.favorite_count.hist()\n",
	"ax.set_yscale('log')"
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"When have I been most active on twitter?"
	]
	},
	{
	"cell_type": "code",
	"execution_count": 6,
	"metadata": {},
	"outputs": [
	{
	"data": {
	"text/plain": [
	"<matplotlib.axes._subplots.AxesSubplot at 0x7f05cd2361d0>"
	]
	},
	"execution_count": 6,
	"metadata": {},
	"output_type": "execute_result"
	},
	{
	"data": {
	"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFD1JREFUeJzt3XGMnPV95/H3tzgkhm1tiNMtZ1tdrkFUHNtr8B6lJRftxr0eCVHMH1EE4lpTUVmn0h5tHCXOVWrUP5BIG5qmurtIbqBxLykbwiUHhaQN57IXVSq0NiFZA0nxEZPgM3Z6AaeboiZuv/fHPG5Gzq5n53me2Xny4/2SRjvzPM/8ns/OM/PZZ559ZjcyE0lSuX5g3AEkSaNl0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCDSz6iLgrIk5ExKFl5u2OiIyITdXtiIjfj4jDEfHFiLhiFKElSau3mj36jwDXnDkxIrYCPwd8tW/ym4BLqssu4EPNI0qSmlg3aIHM/FxETC0z6wPAu4D7+qbtAP4oex+3fSQiNkbERZl57Gzr2LRpU05NLbeKtfGtb32L888/f2zrX4m5htPVXNDdbOYaTtdyHTx48G8z8zWDlhtY9MuJiB3A0cz8QkT0z9oMfK3v9nPVtO8p+ojYRW+vn8nJSd7//vfXidKKpaUlJiYmxrb+lZhrOF3NBd3NZq7hdC3X3Nzcs6taMDMHXoAp4FB1/TzgUWBDdfsIsKm6/gDw+r777QdmBo2/bdu2HKeHH354rOtfibmG09Vcmd3NZq7hdC0XcCBX0eF19uh/DLgYOL03vwV4LCKuBI4CW/uW3VJNkySNydCnV2bmYmb+cGZOZeYUvcMzV2Tm88D9wC9UZ99cBZzMAcfnJUmjtZrTK+8G/hK4NCKei4ibz7L4p4FngMPAHwC/3EpKSVJtqznr5oYB86f6ridwS/NYkqS2+MlYSSqcRS9JhbPoJalwFr0kFa7WJ2Oll5OpPQ82HmP39CluGnKcI7df23i9ErhHL0nFs+glqXAWvSQVzqKXpMJZ9JJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYUbWPQRcVdEnIiIQ33TficivhQRX4yIT0XExr5574mIwxHx5Yj496MKLklandXs0X8EuOaMaQ8Bl2fmTwB/A7wHICIuA64H/lV1n/8WEee0llaSNLSBRZ+ZnwO+cca0z2bmqermI8CW6voOYD4z/yEzvwIcBq5sMa8kaUiRmYMXipgCHsjMy5eZ9yfAxzPzoxHxX4BHMvOj1bw7gc9k5r3L3G8XsAtgcnJy2/z8fJPvo5GlpSUmJibGtv6VmGs4o8q1ePRk4zEm18Pxl4a7z/TmDY3XO8hyj1kb329dp7/nl9tzrK65ubmDmTkzaLl1TVYSEb8BnAI+Nux9M3MvsBdgZmYmZ2dnm0RpZGFhgXGufyXmGs6oct2058HGY+yePsUdi8O93I7cONt4vYMs95i18f3Wdfp7frk9x0atdtFHxE3AW4Dt+d23BUeBrX2LbammSZLGpNbplRFxDfAu4K2Z+fd9s+4Hro+IV0bExcAlwF81jylJqmvgHn1E3A3MApsi4jngvfTOsnkl8FBEQO+4/H/MzCci4h7gSXqHdG7JzH8cVXhJ0mADiz4zb1hm8p1nWf424LYmoaQzTa3iuPHu6VNjPb4sdZWfjJWkwln0klQ4i16SCmfRS1LhLHpJKpxFL0mFs+glqXAWvSQVzqKXpMJZ9JJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYUbWPQRcVdEnIiIQ33TLoyIhyLi6errBdX0iIjfj4jDEfHFiLhilOElSYOtZo/+I8A1Z0zbA+zPzEuA/dVtgDcBl1SXXcCH2okpSaprYNFn5ueAb5wxeQewr7q+D7iub/ofZc8jwMaIuKitsJKk4UVmDl4oYgp4IDMvr26/mJkbq+sBvJCZGyPiAeD2zPyLat5+4N2ZeWCZMXfR2+tncnJy2/z8fDvfUQ1LS0tMTEyMbf0rMdd3LR49OXCZyfVw/KU1CFNDnWzTmzeMJkyf5bblah7rUTn9PfvcX525ubmDmTkzaLl1TVeUmRkRg39afO/99gJ7AWZmZnJ2drZplNoWFhYY5/pXYq7vumnPgwOX2T19ijsWGz+lR6JOtiM3zo4mTJ/ltuVqHutROf09+9xvV91XxfGIuCgzj1WHZk5U048CW/uW21JNG5mpFp6Uu6dP1XpyH7n92sbrlqRRq3t65f3Azur6TuC+vum/UJ19cxVwMjOPNcwoSWpg4B59RNwNzAKbIuI54L3A7cA9EXEz8Czw9mrxTwNvBg4Dfw/84ggyS5KGMLDoM/OGFWZtX2bZBG5pGkqS1B4/GStJhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuG6+YdBJL0snf6TJnX/LEldpf85E/foJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYWz6CWpcBa9JBXOopekwln0klQ4i16SCmfRS1LhGhV9RPx6RDwREYci4u6IeFVEXBwRj0bE4Yj4eESc21ZYSdLwahd9RGwG/hMwk5mXA+cA1wPvAz6Qma8FXgBubiOoJKmepodu1gHrI2IdcB5wDHgjcG81fx9wXcN1SJIaiMysf+eIW4HbgJeAzwK3Ao9Ue/NExFbgM9Ue/5n33QXsApicnNw2Pz9fK8Pi0ZP1wveZXA/HXxr+ftObNzRe99ksLS0xMTEx0nXUMY5cq9nOdbfjWqiTbdTPL1h+W7bxmmpqrbflah/rrr0m5+bmDmbmzKDlav8rwYi4ANgBXAy8CHwCuGa198/MvcBegJmZmZydna2Vo41/N7Z7+hR3LA7/UBy5cbbxus9mYWGBuo/LKI0j12q2c93tuBbqZBv18wuW35Zr+S/8VrLW23K1j3VXX5ODNDl087PAVzLz65n5HeCTwNXAxupQDsAW4GjDjJKkBpoU/VeBqyLivIgIYDvwJPAw8LZqmZ3Afc0iSpKaqF30mfkovV+6PgYsVmPtBd4NvCMiDgOvBu5sIackqaZGB8Ey873Ae8+Y/AxwZZNxJWktTa3y9xK7p0+1/juMI7df2+p4y/GTsZJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYWz6CWpcBa9JBXOopekwln0klS4tfs365KGstr/etTEKP5jkrrHPXpJKpxFL0mF89BNA6N+a73S2+q1+GfCksrRaI8+IjZGxL0R8aWIeCoifjoiLoyIhyLi6errBW2FlSQNr+mhmw8Cf5qZPw78a+ApYA+wPzMvAfZXtyVJY1K76CNiA/AG4E6AzPx2Zr4I7AD2VYvtA65rGlKSVF+TPfqLga8DfxgRn4+ID0fE+cBkZh6rlnkemGwaUpJUX2RmvTtGzACPAFdn5qMR8UHgm8CvZubGvuVeyMzvOU4fEbuAXQCTk5Pb5ufna+VYPHqy1v36Ta6H4y81HqZ1K+Wa3rxh7cP0WVpaYmJiYk3XuZrt3NXtCN3NZq7hjCJXk9fz3NzcwcycGbRck6L/EeCRzJyqbv9besfjXwvMZuaxiLgIWMjMS8821szMTB44cKBWjjbOfNk9fYo7Frt3AtJKucZ91s3CwgKzs7Nrus7VbOeubkfobjZzDWcUuZq8niNiVUVf+9BNZj4PfC0iTpf4duBJ4H5gZzVtJ3Bf3XVIkppr+qPpV4GPRcS5wDPAL9L74XFPRNwMPAu8veE6JEkNNCr6zHwcWO5tw/Ym40qS2uOfQJCkwln0klQ4i16SCmfRS1LhLHpJKpxFL0mFs+glqXAWvSQVzqKXpMJZ9JJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYVrXPQRcU5EfD4iHqhuXxwRj0bE4Yj4eESc2zymJKmuNvbobwWe6rv9PuADmfla4AXg5hbWIUmqqVHRR8QW4Frgw9XtAN4I3Fstsg+4rsk6JEnNNN2j/z3gXcA/VbdfDbyYmaeq288BmxuuQ5LUQGRmvTtGvAV4c2b+ckTMAu8EbgIeqQ7bEBFbgc9k5uXL3H8XsAtgcnJy2/z8fK0ci0dP1rpfv8n1cPylxsO0bqVc05s3rH2YPktLS0xMTKzpOleznbu6HaG72cw1nFHkavJ6npubO5iZM4OWW1d7DXA18NaIeDPwKuCHgA8CGyNiXbVXvwU4utydM3MvsBdgZmYmZ2dna4W4ac+Dte7Xb/f0Ke5YbPJQjMZKuY7cOLv2YfosLCxQd3vVtZrt3NXtCN3NZq7hjCLXWryeax+6ycz3ZOaWzJwCrgf+PDNvBB4G3lYtthO4r3FKSVJtoziP/t3AOyLiML1j9neOYB2SpFVq5T1IZi4AC9X1Z4Ar2xhXktScn4yVpMJZ9JJUOItekgrXvfOX1GlTex5k9/SpVk5rlbQ23KOXpMJZ9JJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYWz6CWpcBa9JBXOopekwln0klS42kUfEVsj4uGIeDIinoiIW6vpF0bEQxHxdPX1gvbiSpKG1WSP/hSwOzMvA64CbomIy4A9wP7MvATYX92WJI1J7aLPzGOZ+Vh1/e+Ap4DNwA5gX7XYPuC6piElSfW1cow+IqaA1wGPApOZeaya9Tww2cY6JEn1RGY2GyBiAvjfwG2Z+cmIeDEzN/bNfyEzv+c4fUTsAnYBTE5Obpufn6+1/sWjJ+sF7zO5Ho6/1HiY1q2Ua3rzhrUPU1k8evL77vHqgq5mM9dwRpGryet5bm7uYGbODFquUdFHxCuAB4A/y8zfraZ9GZjNzGMRcRGwkJmXnm2cmZmZPHDgQK0MU3serHW/frunT3HH4rrG47RtpVxHbr92DGl6pvY8+H33eHVBV7OZazijyNXk9RwRqyr6JmfdBHAn8NTpkq/cD+ysru8E7qu7DklSc01+NF0N/DywGBGPV9P+M3A7cE9E3Aw8C7y9WURJUhO1iz4z/wKIFWZvrzuuJKldfjJWkgpn0UtS4Sx6SSpc985f0kBtnFIq6eXDPXpJKpxFL0mFs+glqXAWvSQVzqKXpMJZ9JJUOItekgpn0UtS4Sx6SSqcRS9JhbPoJalwFr0kFc6il6TCWfSSVDiLXpIKZ9FLUuEsekkqnEUvSYUbWdFHxDUR8eWIOBwRe0a1HknS2Y2k6CPiHOC/Am8CLgNuiIjLRrEuSdLZjWqP/krgcGY+k5nfBuaBHSNalyTpLEZV9JuBr/Xdfq6aJklaY5GZ7Q8a8Tbgmsz8per2zwM/lZm/0rfMLmBXdfNS4MutB1m9TcDfjnH9KzHXcLqaC7qbzVzD6VquH83M1wxaaN2IVn4U2Np3e0s17Z9l5l5g74jWP5SIOJCZM+POcSZzDaeruaC72cw1nK7mGmRUh27+GrgkIi6OiHOB64H7R7QuSdJZjGSPPjNPRcSvAH8GnAPclZlPjGJdkqSzG9WhGzLz08CnRzV+yzpxCGkZ5hpOV3NBd7OZazhdzXVWI/llrCSpO/wTCJJUuCKLPiK2RsTDEfFkRDwREbdW0y+MiIci4unq6wXV9B+PiL+MiH+IiHeeMdZdEXEiIg51JddK43Qk26si4q8i4gvVOL/VhVx9450TEZ+PiAe6kisijkTEYkQ8HhEHOpRrY0TcGxFfioinIuKnx50rIi6tHqfTl29GxK+NO1c179erMQ5FxN0R8aq6uVqXmcVdgIuAK6rrPwj8Db0/xfDbwJ5q+h7gfdX1Hwb+DXAb8M4zxnoDcAVwqCu5VhqnI9kCmKiuvwJ4FLhq3Ln6xnsH8MfAA114vKp5R4BNHXzu7wN+qbp+LrCxC7n6xjwHeJ7eueTjft5vBr4CrK9u3wPc1MY2beNS5B59Zh7LzMeq638HPEVvQ+yg9+Sl+npdtcyJzPxr4DvLjPU54BtdynWWcbqQLTNzqbr5iupS+xdBbW7LiNgCXAt8uG6eUeRqU1u5ImIDvZ2cO6vlvp2ZL4471xm2A/8nM5/tSK51wPqIWAecB/zfurnaVmTR94uIKeB19PYsJzPzWDXreWByTLFay3XGOJ3IVh0eeRw4ATyUma1ka+Ex+z3gXcA/tZGnxVwJfDYiDkbvE+NdyHUx8HXgD6tDXR+OiPM7kKvf9cDdbWRqmiszjwLvB74KHANOZuZn28rWVNFFHxETwP8Afi0zv9k/L3vvr8ZyylFbuc42zjizZeY/ZuZP0vtE9JURcfm4c0XEW4ATmXmwaZY2c1Ven5lX0Ptrr7dExBs6kGsdvUOWH8rM1wHfoncIY9y5To9zLvBW4BNNM7WRqzqGv4PeD8h/AZwfEf+hjWxtKLboI+IV9DbcxzLzk9Xk4xFxUTX/Inp7nN+XuVYYpxPZTqve6j8MXNOBXFcDb42II/T+muobI+KjHch1em+QzDwBfIreX38dd67ngOf63o3dS6/4x53rtDcBj2Xm8SaZWsz1s8BXMvPrmfkd4JPAzzTN1pYiiz4igt6xxacy83f7Zt0P7Kyu7wTu+37MdZZxupDtNRGxsbq+Hvh3wJfGnSsz35OZWzJzit5b/j/PzNp7XC0+XudHxA+evg78HFD7DK8WH6/nga9FxKXVpO3Ak+PO1ecGWjhs02KurwJXRcR51Zjb6R3v74YzfztbwgV4Pb23Wl8EHq8ubwZeDewHngb+F3BhtfyP0NuD+SbwYnX9h6p5d9M75vadavrN48610jhdeMyAnwA+X41zCPjNLuQ6Y8xZmp9109bj9S+BL1SXJ4Df6EKuat5PAgeqsf4ncEFHcp0P/D9gQ8e64rfo7dQcAv478Mqm+dq6+MlYSSpckYduJEnfZdFLUuEsekkqnEUvSYWz6CWpcBa9JBXOopekwln0klS4/w+Cg+PlkZBqNwAAAABJRU5ErkJggg==\n",
	"text/plain": [
	"<Figure size 432x288 with 1 Axes>"
	]
	},
	"metadata": {},
	"output_type": "display_data"
	}
	],
	"source": [
	"df.timestamp.hist()"
	]
	},
	{
	"cell_type": "markdown",
	"metadata": {},
	"source": [
	"## View all tweets"
	]
	},
	{
	"cell_type": "code",
	"execution_count": null,
	"metadata": {
	"scrolled": false
	},
	"outputs": [],
	"source": [
	"# Sort first to get tweets in reverse chronological order\n",
	"for month, group in df.sort_values('timestamp', ascending=False).groupby(df.timestamp.dt.to_period('M'), sort=False):\n",
	" html = f'<h2>{month}</h2>\\n'\n",
	" table_rows = []\n",
	" for ix, tweet in group.iterrows():\n",
	" reply = \"↩\" if tweet['in_reply_to_screen_name'] else \" \"\n",
	" table_rows.append(f\"<tr><td>{reply}</td><td>{tweet['full_text']}</td></tr>\\n\")\n",
	" html += '<table>\\n' + ''.join(table_rows) + '</table>'\n",
	" display(HTML(html))"
	]
	}
	],
	"metadata": {
	"kernelspec": {
	"display_name": "Python 3",
	"language": "python",
	"name": "python3"
	},
	"language_info": {
	"codemirror_mode": {
	"name": "ipython",
	"version": 3
	},
	"file_extension": ".py",
	"mimetype": "text/x-python",
	"name": "python",
	"nbconvert_exporter": "python",
	"pygments_lexer": "ipython3",
	"version": "3.6.6"
	}
	},
	"nbformat": 4,
	"nbformat_minor": 2
	}