Skip to content

Instantly share code, notes, and snippets.

@psychemedia

psychemedia/ipythonblocks_pandas_demo.ipynb

Last active September 9, 2023 23:56
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save psychemedia/9795643 to your computer and use it in GitHub Desktop.
Save psychemedia/9795643 to your computer and use it in GitHub Desktop.
Download ZIP
IPython notebook demonstrating the use of ipythonblocks to visualise pandas dataframe manipulations
Display the source blob
Display the rendered blob
Raw
ipythonblocks_pandas_demo.ipynb
{
"metadata": {
"name": ""
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"IPython and Pandas Blocks Demo"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For a new OU course on all matters data that we're working on at the moment, we're exploring the use of IPython notebooks and the `pandas` library.\n",
"\n",
"A few weeks ago, I came across another Python library called [ipythonblocks](https://github.com/jiffyclub/ipythonblocks) [[demo](http://nbviewer.ipython.org/github/jiffyclub/ipythonblocks/blob/master/demos/ipythonblocks_demo.ipynb)] that's been developed as a teaching tool around Python programming. The library provides an HTML grid containing squares whose colours are controlled by the programmer.\n",
"\n",
"When I first saw caught sight of this library, I thought it looked ideal as a tool for helping visualise the state of pandas dataframes and various transformations we might apply to them. So here are a few proof of concept doodles... Comments much appreciated via the partner blogpost to this gist/notebook - [Visualising Pandas DataFrames With IPythonBlocks \u2013 Proof of Concept](http://blog.ouseful.info/2014/03/26/visualising-pandas-dataframes-with-ipythonblocks-proof-of-concept/)."
]
},
{
"cell_type": "heading",
"level": 2,
"metadata": {},
"source": [
"Getting Started"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I'm using Anaconda as my Python/ipynb/matplotlib distro, which is a real pain to install things into. Easiest way to get the `ipythonblocks` library running is to download the `.py` file from github and install it locally..."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"from ipythonblocks import BlockGrid"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 265
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"#Here's a simple grid\n",
"grid = BlockGrid(10, 10, fill=(123, 234, 123))\n",
"grid"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocks4e0868af-9307-485c-ad8d-a97c79b3ec1c td {border: 1px solid white;}</style><table id=\"blocks4e0868af-9307-485c-ad8d-a97c79b3ec1c\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 3]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 4]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 5]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 6]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 7]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 8]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 9]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 3]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 4]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 5]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 6]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 7]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 8]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 9]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 3]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 4]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 5]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 6]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 7]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 8]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 9]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [3, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 3]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 4]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 5]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 6]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 7]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 8]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 9]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [4, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 3]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 4]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 5]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 6]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 7]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 8]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 9]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [5, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 3]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 4]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 5]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 6]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 7]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 8]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 9]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [6, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 3]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 4]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 5]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 6]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 7]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 8]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 9]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [7, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 3]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 4]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 5]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 6]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 7]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 8]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 9]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [8, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 3]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 4]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 5]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 6]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 7]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 8]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 9]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [9, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 3]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 4]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 5]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 6]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 7]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 8]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 9]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 266,
"text": [
"<ipythonblocks.BlockGrid at 0x10722ed10>"
]
}
],
"prompt_number": 266
},
{
"cell_type": "heading",
"level": 2,
"metadata": {},
"source": [
"Introducing pandas DateFrames"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import pandas as pd"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 267
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"df=pd.DataFrame({'group':['a','a','a','a','a','a','a','b','b','b','b','b','b','b'],\n",
" 'day':['Mon','Tues','Fri','Thurs','Sat','Sun','Weds','Fri','Sun','Thurs','Sat','Weds','Mon','Tues'],\n",
" 'amount':[1,2,4,2,1,1,2,4,5,3,4,2,1,3],\n",
" 'value':[2.1,np.nan,3,4.4,2.5,1,np.nan,np.nan,8.2,3.2,4.1,np.nan,1.2,1.4]\n",
" })"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 268
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"df"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>amount</th>\n",
" <th>day</th>\n",
" <th>group</th>\n",
" <th>value</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0 </th>\n",
" <td> 1</td>\n",
" <td> Mon</td>\n",
" <td> a</td>\n",
" <td> 2.1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1 </th>\n",
" <td> 2</td>\n",
" <td> Tues</td>\n",
" <td> a</td>\n",
" <td> NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2 </th>\n",
" <td> 4</td>\n",
" <td> Fri</td>\n",
" <td> a</td>\n",
" <td> 3.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3 </th>\n",
" <td> 2</td>\n",
" <td> Thurs</td>\n",
" <td> a</td>\n",
" <td> 4.4</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4 </th>\n",
" <td> 1</td>\n",
" <td> Sat</td>\n",
" <td> a</td>\n",
" <td> 2.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5 </th>\n",
" <td> 1</td>\n",
" <td> Sun</td>\n",
" <td> a</td>\n",
" <td> 1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6 </th>\n",
" <td> 2</td>\n",
" <td> Weds</td>\n",
" <td> a</td>\n",
" <td> NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7 </th>\n",
" <td> 4</td>\n",
" <td> Fri</td>\n",
" <td> b</td>\n",
" <td> NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8 </th>\n",
" <td> 5</td>\n",
" <td> Sun</td>\n",
" <td> b</td>\n",
" <td> 8.2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>9 </th>\n",
" <td> 3</td>\n",
" <td> Thurs</td>\n",
" <td> b</td>\n",
" <td> 3.2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10</th>\n",
" <td> 4</td>\n",
" <td> Sat</td>\n",
" <td> b</td>\n",
" <td> 4.1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>11</th>\n",
" <td> 2</td>\n",
" <td> Weds</td>\n",
" <td> b</td>\n",
" <td> NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>12</th>\n",
" <td> 1</td>\n",
" <td> Mon</td>\n",
" <td> b</td>\n",
" <td> 1.2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>13</th>\n",
" <td> 3</td>\n",
" <td> Tues</td>\n",
" <td> b</td>\n",
" <td> 1.4</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>14 rows \u00d7 4 columns</p>\n",
"</div>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 269,
"text": [
" amount day group value\n",
"0 1 Mon a 2.1\n",
"1 2 Tues a NaN\n",
"2 4 Fri a 3.0\n",
"3 2 Thurs a 4.4\n",
"4 1 Sat a 2.5\n",
"5 1 Sun a 1.0\n",
"6 2 Weds a NaN\n",
"7 4 Fri b NaN\n",
"8 5 Sun b 8.2\n",
"9 3 Thurs b 3.2\n",
"10 4 Sat b 4.1\n",
"11 2 Weds b NaN\n",
"12 1 Mon b 1.2\n",
"13 3 Tues b 1.4\n",
"\n",
"[14 rows x 4 columns]"
]
}
],
"prompt_number": 269
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can get the shape of a dataframe easily enough:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"df.shape"
],
"language": "python",
"metadata": {},
"outputs": [
{
"metadata": {},
"output_type": "pyout",
"prompt_number": 270,
"text": [
"(14, 4)"
]
}
],
"prompt_number": 270
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This gives us *(rows,columns)*. So let's visualise that:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"def pBlockGrid(df):\n",
" (y,x)=df.shape\n",
" return BlockGrid(x,y)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 271
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"pBlockGrid(df)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocksb530e41b-f84c-4c46-8ba7-dc85bcaf7806 td {border: 1px solid white;}</style><table id=\"blocksb530e41b-f84c-4c46-8ba7-dc85bcaf7806\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [0, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [0, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [0, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [1, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [1, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [1, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [2, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [2, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [2, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [3, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [3, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [3, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [3, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [4, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [4, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [4, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [4, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [5, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [5, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [5, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [5, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [6, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [6, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [6, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [6, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [7, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [7, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [7, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [7, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [8, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [8, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [8, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [8, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [9, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [9, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [9, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [9, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [10, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [10, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [10, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [10, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [11, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [11, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [11, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [11, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [12, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [12, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [12, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [12, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [13, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [13, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [13, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [13, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 272,
"text": [
"<ipythonblocks.BlockGrid at 0x1072148d0>"
]
}
],
"prompt_number": 272
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can add in some colour:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"def pBlockGrid2(df,fill=(123, 234, 123),*args,**kwargs):\n",
" (y,x)=df.shape\n",
" b=BlockGrid(x,y,fill=fill,**kwargs)\n",
" return b"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 273
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There's also an opportunity to tweak the blocksize..."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"pb=pBlockGrid2(df)\n",
"pb.block_size = 4\n",
"pb"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocks17607922-37e7-4829-98cc-43fef0581635 td {border: 1px solid white;}</style><table id=\"blocks17607922-37e7-4829-98cc-43fef0581635\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [3, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [4, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [5, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [6, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [6, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [7, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [7, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [8, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [8, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [9, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [9, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [10, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [10, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [10, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [10, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [11, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [11, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [11, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [11, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [12, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [12, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [12, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [12, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [13, 0]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [13, 1]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [13, 2]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [13, 3]&#10;Color: (123, 234, 123)\" style=\"width: 4px; height: 4px;background-color: rgb(123, 234, 123);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 274,
"text": [
"<ipythonblocks.BlockGrid at 0x107213790>"
]
}
],
"prompt_number": 274
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can check the type of each column in the `pandas` dataframe:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"df.dtypes"
],
"language": "python",
"metadata": {},
"outputs": [
{
"metadata": {},
"output_type": "pyout",
"prompt_number": 275,
"text": [
"amount int64\n",
"day object\n",
"group object\n",
"value float64\n",
"dtype: object"
]
}
],
"prompt_number": 275
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's start to tweak the function so we can display the columns by data type:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"def pBlockGrid3(df,fill=(123, 234, 123),blocksize=10,*args,**kwargs):\n",
" b=pBlockGrid2(df,fill=fill)\n",
" type_colours={'int64':(255, 0, 0),'float64':(0, 0, 255),'object':(128,128,0)}\n",
" for i,j in enumerate(df.dtypes):\n",
" b[:, i] = type_colours[j.name]\n",
" b.block_size = blocksize\n",
" return b"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 276
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"pBlockGrid3(df)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocks34f44ac1-d1a2-4931-a02f-8b9a48368acf td {border: 1px solid white;}</style><table id=\"blocks34f44ac1-d1a2-4931-a02f-8b9a48368acf\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [0, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [1, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [2, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [3, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [3, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [4, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [4, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [4, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [4, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [5, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [5, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [5, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [5, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [6, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [6, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [6, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [6, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [7, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [7, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [7, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [7, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [8, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [8, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [8, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [8, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [9, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [9, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [9, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [9, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [10, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [10, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [10, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [10, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [11, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [11, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [11, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [11, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [12, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [12, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [12, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [12, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [13, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [13, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [13, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [13, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 277,
"text": [
"<ipythonblocks.BlockGrid at 0x10722ac50>"
]
}
],
"prompt_number": 277
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Some of the cells are `NaN` or unknown values - let's detect them and colour them black:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"def pBlock_colourNA(df,b,color_NA=(0,0,0),*args,**kwargs):\n",
" (y,x)=df.shape\n",
" for i in range(x):\n",
" for j in range(y):\n",
" if pd.isnull(df.iloc[j,i]):\n",
" b[j,i]=color_NA\n",
" return b\n",
"\n",
"def pBlockGrid4(df,fill=(123, 234, 123),*args,**kwargs):\n",
" b=pBlockGrid3(df,fill=fill)\n",
" b=pBlock_colourNA(df,b,*args,**kwargs)\n",
" return b"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 278
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"pBlockGrid4(df,blocksize=5)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocks4b66e257-713c-48e6-b5f7-19d5deebc74b td {border: 1px solid white;}</style><table id=\"blocks4b66e257-713c-48e6-b5f7-19d5deebc74b\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [0, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [1, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 3]&#10;Color: (0, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [2, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [3, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [3, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [4, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [4, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [4, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [4, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [5, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [5, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [5, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [5, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [6, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [6, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [6, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [6, 3]&#10;Color: (0, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [7, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [7, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [7, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [7, 3]&#10;Color: (0, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [8, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [8, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [8, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [8, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [9, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [9, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [9, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [9, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [10, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [10, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [10, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [10, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [11, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [11, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [11, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [11, 3]&#10;Color: (0, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [12, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [12, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [12, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [12, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [13, 0]&#10;Color: (255, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [13, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [13, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [13, 3]&#10;Color: (0, 0, 255)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 255);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 279,
"text": [
"<ipythonblocks.BlockGrid at 0x10720c710>"
]
}
],
"prompt_number": 279
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If we transpose the dataframe, we should be able to see the change in shape:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"df.transpose()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>0</th>\n",
" <th>1</th>\n",
" <th>2</th>\n",
" <th>3</th>\n",
" <th>4</th>\n",
" <th>5</th>\n",
" <th>6</th>\n",
" <th>7</th>\n",
" <th>8</th>\n",
" <th>9</th>\n",
" <th>10</th>\n",
" <th>11</th>\n",
" <th>12</th>\n",
" <th>13</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>amount</th>\n",
" <td> 1</td>\n",
" <td> 2</td>\n",
" <td> 4</td>\n",
" <td> 2</td>\n",
" <td> 1</td>\n",
" <td> 1</td>\n",
" <td> 2</td>\n",
" <td> 4</td>\n",
" <td> 5</td>\n",
" <td> 3</td>\n",
" <td> 4</td>\n",
" <td> 2</td>\n",
" <td> 1</td>\n",
" <td> 3</td>\n",
" </tr>\n",
" <tr>\n",
" <th>day</th>\n",
" <td> Mon</td>\n",
" <td> Tues</td>\n",
" <td> Fri</td>\n",
" <td> Thurs</td>\n",
" <td> Sat</td>\n",
" <td> Sun</td>\n",
" <td> Weds</td>\n",
" <td> Fri</td>\n",
" <td> Sun</td>\n",
" <td> Thurs</td>\n",
" <td> Sat</td>\n",
" <td> Weds</td>\n",
" <td> Mon</td>\n",
" <td> Tues</td>\n",
" </tr>\n",
" <tr>\n",
" <th>group</th>\n",
" <td> a</td>\n",
" <td> a</td>\n",
" <td> a</td>\n",
" <td> a</td>\n",
" <td> a</td>\n",
" <td> a</td>\n",
" <td> a</td>\n",
" <td> b</td>\n",
" <td> b</td>\n",
" <td> b</td>\n",
" <td> b</td>\n",
" <td> b</td>\n",
" <td> b</td>\n",
" <td> b</td>\n",
" </tr>\n",
" <tr>\n",
" <th>value</th>\n",
" <td> 2.1</td>\n",
" <td> NaN</td>\n",
" <td> 3</td>\n",
" <td> 4.4</td>\n",
" <td> 2.5</td>\n",
" <td> 1</td>\n",
" <td> NaN</td>\n",
" <td> NaN</td>\n",
" <td> 8.2</td>\n",
" <td> 3.2</td>\n",
" <td> 4.1</td>\n",
" <td> NaN</td>\n",
" <td> 1.2</td>\n",
" <td> 1.4</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>4 rows \u00d7 14 columns</p>\n",
"</div>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 280,
"text": [
" 0 1 2 3 4 5 6 7 8 9 10 11 \\\n",
"amount 1 2 4 2 1 1 2 4 5 3 4 2 \n",
"day Mon Tues Fri Thurs Sat Sun Weds Fri Sun Thurs Sat Weds \n",
"group a a a a a a a b b b b b \n",
"value 2.1 NaN 3 4.4 2.5 1 NaN NaN 8.2 3.2 4.1 NaN \n",
"\n",
" 12 13 \n",
"amount 1 3 \n",
"day Mon Tues \n",
"group b b \n",
"value 1.2 1.4 \n",
"\n",
"[4 rows x 14 columns]"
]
}
],
"prompt_number": 280
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"pBlockGrid4(df.transpose(),blocksize=5)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocks1f271b3f-7dd2-4f51-bb65-4f3e69034ab5 td {border: 1px solid white;}</style><table id=\"blocks1f271b3f-7dd2-4f51-bb65-4f3e69034ab5\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 3]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 4]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 5]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 6]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 7]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 8]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 9]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 10]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 11]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 12]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [0, 13]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 3]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 4]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 5]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 6]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 7]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 8]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 9]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 10]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 11]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 12]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [1, 13]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 1]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 3]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 4]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 5]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 6]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 7]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 8]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 9]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 10]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 11]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 12]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [2, 13]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td></tr><tr><td title=\"Index: [3, 0]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 1]&#10;Color: (0, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [3, 2]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 3]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 4]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 5]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 6]&#10;Color: (0, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [3, 7]&#10;Color: (0, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [3, 8]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 9]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 10]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 11]&#10;Color: (0, 0, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [3, 12]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td><td title=\"Index: [3, 13]&#10;Color: (128, 128, 0)\" style=\"width: 10px; height: 10px;background-color: rgb(128, 128, 0);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 281,
"text": [
"<ipythonblocks.BlockGrid at 0x10722fd90>"
]
}
],
"prompt_number": 281
},
{
"cell_type": "heading",
"level": 3,
"metadata": {},
"source": [
"Joining Dataframes: .concat()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's create a couple of dataframes that we can try appending together:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"df1=pd.DataFrame({\"colA\":['A','B','C','D'],\"colB\":np.arange(4)})\n",
"df1"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>colA</th>\n",
" <th>colB</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td> A</td>\n",
" <td> 0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td> B</td>\n",
" <td> 1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td> C</td>\n",
" <td> 2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td> D</td>\n",
" <td> 3</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>4 rows \u00d7 2 columns</p>\n",
"</div>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 282,
"text": [
" colA colB\n",
"0 A 0\n",
"1 B 1\n",
"2 C 2\n",
"3 D 3\n",
"\n",
"[4 rows x 2 columns]"
]
}
],
"prompt_number": 282
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"df2=pd.DataFrame({\"colA\":['D','E','F'],\"colB\":np.arange(14,17)})\n",
"df2"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>colA</th>\n",
" <th>colB</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td> D</td>\n",
" <td> 14</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td> E</td>\n",
" <td> 15</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td> F</td>\n",
" <td> 16</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>3 rows \u00d7 2 columns</p>\n",
"</div>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 283,
"text": [
" colA colB\n",
"0 D 14\n",
"1 E 15\n",
"2 F 16\n",
"\n",
"[3 rows x 2 columns]"
]
}
],
"prompt_number": 283
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This is probably a bit clunky as far as \"proper\" Python hackers go, but this is still proof of concept...\n",
"\n",
"What happens if we create a function that does a pandas `.concat()` and returns the resulting dataframe along with its block representation:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"def pBlockGrid5(df1,df2,join='outer',fill=(123, 234, 123),*args,**kwargs):\n",
" df=pd.concat([df1,df2],join=join)\n",
" b=pBlockGrid2(df,fill=fill)\n",
" for block in b:\n",
" if block.row < len(df1): block.set_colors(255,0,0)\n",
" else: block.set_colors(0,0,255)\n",
" b=pBlock_colourNA(df,b,*args,**kwargs)\n",
" return (df,b)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 284
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The idea behind the colouring is to show which dataframe provide what rows. The function should probably be generalised to accept a list of dataframes to merge, and then colour them all differently (which will require setting up a colour ramp)."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"dfd,bb=pBlockGrid5(df1,df2)\n",
"bb"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocks1818af91-95fd-4f71-97bf-d842201bba75 td {border: 1px solid white;}</style><table id=\"blocks1818af91-95fd-4f71-97bf-d842201bba75\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [0, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [1, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [2, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td></tr><tr><td title=\"Index: [3, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [3, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td></tr><tr><td title=\"Index: [4, 0]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [4, 1]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [5, 0]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [5, 1]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [6, 0]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [6, 1]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 285,
"text": [
"<ipythonblocks.BlockGrid at 0x10722f4d0>"
]
}
],
"prompt_number": 285
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I'm wondering now: could the pandas DataFrame() object and associated operators (such as merge(), concat(), etc be extended to support ipythonblocks style visualisations?!\n",
"\n",
"Speaking of merge, let's see if we can get something going there so we can inner, outer, left and right join dataframes - and visualise what happens as a result..."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"df3=pd.DataFrame({\"colA\":['F','G'],\"colB\":np.arange(16,18),\"colC\":np.arange(26,28)})\n",
"df3"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\">\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>colA</th>\n",
" <th>colB</th>\n",
" <th>colC</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td> F</td>\n",
" <td> 16</td>\n",
" <td> 26</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td> G</td>\n",
" <td> 17</td>\n",
" <td> 27</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>2 rows \u00d7 3 columns</p>\n",
"</div>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 286,
"text": [
" colA colB colC\n",
"0 F 16 26\n",
"1 G 17 27\n",
"\n",
"[2 rows x 3 columns]"
]
}
],
"prompt_number": 286
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"dfd,bb=pBlockGrid5(df1,df3)\n",
"bb"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocksb3c35075-9aa7-4aa3-865a-6aee89f4ab05 td {border: 1px solid white;}</style><table id=\"blocksb3c35075-9aa7-4aa3-865a-6aee89f4ab05\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [0, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [0, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [1, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [1, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [2, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [2, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [3, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [3, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [3, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [4, 0]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [4, 1]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [4, 2]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [5, 0]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [5, 1]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [5, 2]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 287,
"text": [
"<ipythonblocks.BlockGrid at 0x10721b3d0>"
]
}
],
"prompt_number": 287
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This visualisation tells us that the top four rows include data from the first data frame, and the rightmost column for those rows are filled with NAs. The bottom two rows are complete and come from the second data frame; maybe the second dataframe introduced an additional column?\n",
"\n",
"How about we just look at missing data?"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"b=pBlockGrid2(dfd,fill=(123, 234, 123))\n",
"pBlock_colourNA(dfd,b)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocksbb10f52f-ed37-49d7-b4dd-e7263ec3c543 td {border: 1px solid white;}</style><table id=\"blocksbb10f52f-ed37-49d7-b4dd-e7263ec3c543\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [0, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [1, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [2, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [3, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [3, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [4, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [4, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr><tr><td title=\"Index: [5, 0]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 1]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td><td title=\"Index: [5, 2]&#10;Color: (123, 234, 123)\" style=\"width: 20px; height: 20px;background-color: rgb(123, 234, 123);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 288,
"text": [
"<ipythonblocks.BlockGrid at 0x107246350>"
]
}
],
"prompt_number": 288
},
{
"cell_type": "heading",
"level": 3,
"metadata": {},
"source": [
"Joining Dataframes: .merge()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The merge joins are akin to SQL joins - inner, outer, left and right.\n",
"\n",
"I'm going to try to colour things so we can see which dataframe (left or right) the columns came in from..."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"def pBlockGrid6(df1,df2,fill=(123, 234, 123),*args,**kwargs):\n",
" df=pd.merge(df1,df2,**kwargs)\n",
" b=pBlockGrid2(df,fill=fill)\n",
" for block in b:\n",
" label=df.columns.tolist()[block.col]\n",
" if 'on' in kwargs and label in kwargs['on']: block.set_colors(0,255,0)\n",
" #This following bit assumes that the left N cols in the merged table are from the left table\n",
" elif block.col < len(df1.columns): block.set_colors(255,0,0)\n",
" else: block.set_colors(0,0,255)\n",
" b=pBlock_colourNA(df,b)\n",
" return (df,b)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 236
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"dfd,bb=pBlockGrid6(df2,df3,how='inner',on=['colA','colB'])\n",
"print(dfd)\n",
"bb"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
" colA colB colC\n",
"0 F 16 26\n",
"\n",
"[1 rows x 3 columns]\n"
]
},
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocks67cb26b1-eb27-408a-a25c-7f4e36cc0ebb td {border: 1px solid white;}</style><table id=\"blocks67cb26b1-eb27-408a-a25c-7f4e36cc0ebb\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [0, 1]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [0, 2]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 237,
"text": [
"<ipythonblocks.BlockGrid at 0x10721ac50>"
]
}
],
"prompt_number": 237
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"So we read green as common join columns, blue as right dataframe only cells, and red as left dataframe only data cells."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"dfd,bb=pBlockGrid6(df2,df3,how='left',on=['colA'])\n",
"print(dfd)\n",
"bb"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
" colA colB_x colB_y colC\n",
"0 D 14 NaN NaN\n",
"1 E 15 NaN NaN\n",
"2 F 16 16 26\n",
"\n",
"[3 rows x 4 columns]\n"
]
},
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocks0417943b-dea5-4650-bac6-be59ba106198 td {border: 1px solid white;}</style><table id=\"blocks0417943b-dea5-4650-bac6-be59ba106198\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [0, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [0, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [0, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [1, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [1, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [1, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [2, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [2, 2]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [2, 3]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 238,
"text": [
"<ipythonblocks.BlockGrid at 0x10721aed0>"
]
}
],
"prompt_number": 238
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"dfd,bb=pBlockGrid6(df2,df3,how='right',on=['colA'])\n",
"print(dfd)\n",
"bb"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
" colA colB_x colB_y colC\n",
"0 F 16 16 26\n",
"1 G NaN 17 27\n",
"\n",
"[2 rows x 4 columns]\n"
]
},
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocks19d603a0-6d44-430d-bff1-970ed7bcab40 td {border: 1px solid white;}</style><table id=\"blocks19d603a0-6d44-430d-bff1-970ed7bcab40\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [0, 1]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [0, 2]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [0, 3]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [1, 1]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [1, 2]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [1, 3]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 239,
"text": [
"<ipythonblocks.BlockGrid at 0x10721ac10>"
]
}
],
"prompt_number": 239
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Getting the logic for colouring the outer join clear defeats me for now...:-( I'm not totally convinced it'll always work? Maybe it would help my thinking if we reorder the columns so that columns contributed by the right table are on the right of the joined dataframe, then we should be able to colour columns we know have come from the right table?\n",
"\n",
"Anyway, let's give it a go..."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"dfd,bb=pBlockGrid6(df2,df3,how='outer',on=['colB'])\n",
"print(dfd)\n",
"bb"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
" colA_x colB colA_y colC\n",
"0 D 14 NaN NaN\n",
"1 E 15 NaN NaN\n",
"2 F 16 F 26\n",
"3 NaN 17 G 27\n",
"\n",
"[4 rows x 4 columns]\n"
]
},
{
"html": [
"<style type=\"text/css\">table.blockgrid {border: none;} .blockgrid tr {border: none;} .blockgrid td {padding: 0px;} #blocks9e273568-0285-4320-a882-9cdd228e04e9 td {border: 1px solid white;}</style><table id=\"blocks9e273568-0285-4320-a882-9cdd228e04e9\" class=\"blockgrid\"><tbody><tr><td title=\"Index: [0, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [0, 1]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [0, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [0, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [1, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [1, 1]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [1, 2]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [1, 3]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td></tr><tr><td title=\"Index: [2, 0]&#10;Color: (255, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(255, 0, 0);\"></td><td title=\"Index: [2, 1]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [2, 2]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [2, 3]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr><tr><td title=\"Index: [3, 0]&#10;Color: (0, 0, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 0);\"></td><td title=\"Index: [3, 1]&#10;Color: (0, 255, 0)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 255, 0);\"></td><td title=\"Index: [3, 2]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td><td title=\"Index: [3, 3]&#10;Color: (0, 0, 255)\" style=\"width: 20px; height: 20px;background-color: rgb(0, 0, 255);\"></td></tr></tbody></table>"
],
"metadata": {},
"output_type": "pyout",
"prompt_number": 289,
"text": [
"<ipythonblocks.BlockGrid at 0x10721b510>"
]
}
],
"prompt_number": 289
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Seems to work.. erm, maybe... But I suspect things will start breaking if we allow `on_left` and `on-right` join column declarations...:-(\n",
"\n",
"But the principle is there...\n",
"\n",
"And once again I wonder - could this sort of visualisation be built into pandas as a useful teaching tool? How easy would it be to create a patch library to extend pandas with this sort of functionality?\n",
"\n",
"For example, here are some thoughts on a possible pandas interface:\n",
"\n",
"* DataFrame()*.blocks()* to show the blocks\n",
"* .cat(*blocks=True*) and .merge(*blocks=True*) to return (df, blocks)\n",
"* DataFrame().blocks(*blockProperties={}*) and eg .merge(blocks=True, *blockProperties={}*)\n",
"* blockProperties: showNA=True|False, color_base=(), color_NA=(), color_left=(), color_right=(), color_gradient=[] (eg for a .cat() on many dataframes), colorView=structure|datatypes|missing (the colorView reveals the <em>datatypes</em> of the columns, the <em>structure</em> origins of cells returned from a .merge() or .cat(), or a view of <em>missing</em> data (reveal NA/NaN etc over a base color), colorTypes={} (to set the colors for different datatype)"
]
}
],
"metadata": {}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment