Skip to content

Instantly share code, notes, and snippets.

@eyeseast

eyeseast/make.md

Last active August 29, 2015 14:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save eyeseast/6932bc97746fb1566d63 to your computer and use it in GitHub Desktop.
Save eyeseast/6932bc97746fb1566d63 to your computer and use it in GitHub Desktop.
Download ZIP
How to make make...
Raw
make.md

Using Massachusetts school enrollment data as an example:

  1. Loop through a list of years, downloading a spreadsheet for each one.

  2. Each spreadsheet is a weird Excel-as-HTML format, so I need to process each with a Python script to convert to CSV format.

  3. I only want Boston schools, not the whole state, so I need to filter each file using csvgrep

  4. Run the resulting files through another Python script to load them into a database using dataset

  5. Export aggregates and query results on the combined data using datafreeze

@eyeseast
Copy link
Author

Jeff Larson's answer: https://gist.github.com/thejefflarson/1221ad1984eba794fd9b. Essentially, use make to decide which Python scripts to run, based on which files exist.

@eyeseast
Copy link
Author

The twitter conversation that started all this: https://twitter.com/eyeseast/status/502946780146044928

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment