Skip to content

Instantly share code, notes, and snippets.

@maptastik

maptastik/data-cleaning.md

Created April 19, 2016 04:11
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save maptastik/f480830b0f95761897c730a53f496ff1 to your computer and use it in GitHub Desktop.
Save maptastik/f480830b0f95761897c730a53f496ff1 to your computer and use it in GitHub Desktop.
Download ZIP
Open Refine Stuff
Raw
data-cleaning.md

Some Google Refine snippets and resources to help with cleaning PVA data

Google Refine recipes

Re-add the leading 0 to the 7-digit property numbers

  1. Numeric facet on propertyand filter for 7-digit values only
  2. Run a transform on the faceted property data. Concatenate a 0 to the front of the 7-digit values
    • '0' + value
  3. Remove the facet.

Get rid of dollar signs ($) and commas (,)

value.replace('$','').replace(',','')

Get rid of tilde-A (Ã)

value.replace('Ã','')

Concatenating across columns. This can be useful for creating unique IDs when there are multiple records for a property

cells['column name'].value + cells['other column name'].value

Find and delete duplicates

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment