Skip to content

Instantly share code, notes, and snippets.

@fijimunkii
Created March 28, 2014 21:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save fijimunkii/9843681 to your computer and use it in GitHub Desktop.
Save fijimunkii/9843681 to your computer and use it in GitHub Desktop.
ruby: parse csv and get coordinates of addresses
require 'csv'
require 'geocoder'
csv_dir = File.join(__dir__, 'csv/*.csv')
Geocoder.configure(:always_raise => [Geocoder::OverQueryLimitError])
CSV::Converters[:quotes] = lambda{|s|
begin
return '' if s.to_s.empty?
return s if s.start_with?('"')
'"' + s + '"'
rescue ArgumentError
s
end
}
sql_batch = []
print "\nCrunching..."
Dir.glob(csv_dir) do |csv_file|
rows = CSV.read(csv_file, { :headers => true, :force_quotes => true, :converters => [:quotes] })
sql = 'INSERT INTO radiostations (Name, CallSign, Location, Zip, Twiter, Facebook, Phone, Latitude, Longitude) VALUES'
rows.each do |row|
begin
print '.'
coords = Geocoder.coordinates(row.to_s.split(',')[2])
rescue Geocoder::OverQueryLimitError
print '*'
sleep 10
retry
end
coords.nil? ? sql += " (#{row},\"\",\"\")," : sql += " (#{row}, \"#{coords[0]}\", \"#{coords[1]}\"),"
end
sql_batch << (sql[0..-2] + ';')
end
sql_output = sql_batch.join('').gsub!(/"""/, '"')
File.open('sql_batch.txt', 'a') { |f| f.write(sql_output) }
print "\nDone."
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment