Skip to content

Instantly share code, notes, and snippets.

@eyeseast

eyeseast/make.md

Last active Aug 29, 2015
Embed
What would you like to do?
How to make make...

Using Massachusetts school enrollment data as an example:

  1. Loop through a list of years, downloading a spreadsheet for each one.

  2. Each spreadsheet is a weird Excel-as-HTML format, so I need to process each with a Python script to convert to CSV format.

  3. I only want Boston schools, not the whole state, so I need to filter each file using csvgrep

  4. Run the resulting files through another Python script to load them into a database using dataset

  5. Export aggregates and query results on the combined data using datafreeze

@eyeseast

This comment has been minimized.

Copy link
Owner Author

@eyeseast eyeseast commented Aug 23, 2014

Jeff Larson's answer: https://gist.github.com/thejefflarson/1221ad1984eba794fd9b. Essentially, use make to decide which Python scripts to run, based on which files exist.

@eyeseast

This comment has been minimized.

Copy link
Owner Author

@eyeseast eyeseast commented Aug 23, 2014

The twitter conversation that started all this: https://twitter.com/eyeseast/status/502946780146044928

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment