conormm

R to python data wrangling snippets

The dplyr package in R makes data wrangling significantly easier. The beauty of dplyr is that, by design, the options available are limited. Specifically, a set of key verbs form the core of the package. Using these verbs you can solve a wide range of data problems effectively in a shorter timeframe. Whilse transitioning to Python I have greatly missed the ease with which I can think through and solve problems using dplyr in R. The purpose of this document is to demonstrate how to execute the key dplyr verbs when manipulating data using Python (with the pandas package).

dplyr is organised around six key verbs:

dannguyen

Iowa Liquor Sales dataset via Socrata/data.iowa.gov

(preliminary exploration)

The state of Iowa has released an 800MB+ dataset of more than 3 million rows showing weekly liquor sales, broken down by liquor category, vendor, and product name, e.g. STRAIGHT BOURBON WHISKIES, Jim Beam Brands, Maker's Mark

This dataset contains the spirits purchase information of Iowa Class “E” liquor licensees by product and date of purchase from January 1, 2014 to current. The dataset can be used to analyze total spirits sales in Iowa of individual products at the store level.

You can view the dataset via Socrata

yihui

<script type="text/javascript">
SyntaxHighlighter.autoloader(
  "r  path/to/your/syntaxhighlighter/scripts/shBrushR.js",
  "plain  path/to/your/syntaxhighlighter/scripts/shBrushPlain.js",
  "sql  path/to/your/syntaxhighlighter/scripts/shBrushSql.js",
  "js  path/to/your/syntaxhighlighter/scripts/shBrushJScript.js",
  "html xml  path/to/your/syntaxhighlighter/scripts/shBrushXml.js"
);
SyntaxHighlighter.defaults["toolbar"] = false;
SyntaxHighlighter.all();