Skip to content

Instantly share code, notes, and snippets.

@bodnarbm
Created March 30, 2017 00:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bodnarbm/5c8fc72e3b4ee5e6669c8c93bd16e3a4 to your computer and use it in GitHub Desktop.
Save bodnarbm/5c8fc72e3b4ee5e6669c8c93bd16e3a4 to your computer and use it in GitHub Desktop.
Given a comments.json (output from a the /documents.json for regulations.gov api), download all the attachments for the comments.
require 'JSON'
require 'open-uri'
api_key = "REPLACE_WITH_DATA_GOV_API_KEY"
file = File.open('comments.json')
comments_json = JSON.load(file)["documents"]
comments_with_attachments = comments_json.select { |comment| comment["attachmentCount"] > 0 }
comments_with_attachments.each do |comment|
comment_id = comment["documentId"]
comment_metadata = JSON.load(open("http://api.data.gov:80/regulations/v3/document.json?api_key=#{api_key}&documentId=#{comment_id}"))
attachments = comment_metadata["attachments"]
attachments.each do |attachment|
attachment_order_number = attachment["attachmentOrderNumber"]
attachmentURL = attachment["fileFormats"].first
file_ext = /contentType=([^&]+)/.match(attachmentURL)[1]
open("#{comment_id}_#{attachment_order_number}.#{file_ext}", "wb") do |file|
file << open("#{attachmentURL}&api_key=#{api_key}").read
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment