Skip to content

Instantly share code, notes, and snippets.

@rebeccawilliams

rebeccawilliams/alldata.py

Last active August 29, 2015 14:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rebeccawilliams/5698b255e8c7b63f2c7c to your computer and use it in GitHub Desktop.
Save rebeccawilliams/5698b255e8c7b63f2c7c to your computer and use it in GitHub Desktop.
Download ZIP
get all of the data in many files
Raw
alldata.py
import urllib
dataset = 137
url = 'http://catalog.data.gov/api/3/action/package_search?rows=1000&start=%s' % (dataset)
while dataset > 0:
print "downloading all the Data.gov metadata"
urllib.urlretrieve(url, "datagov%s.json" % (dataset))
dataset = dataset - 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment