Skip to content

Instantly share code, notes, and snippets.

@TSMMark
Last active March 4, 2022 20:17
Show Gist options
  • Save TSMMark/6642751 to your computer and use it in GitHub Desktop.
Save TSMMark/6642751 to your computer and use it in GitHub Desktop.
Process a remote CSV with SmarterCSV (also example with Paperclip and Amazon S3 AWS)
# this is the code I used to determine the csv_path to use when using Paperclip with Amazon S3 AWS
# necessary because my development environment doesn't upload to S3
# get an object that has_attachment with Paperclip
object = ModelWithPaperclip.last
# use url if no file exists at path
csv_path = File.exists?(object.csv.path) ? object.csv.path : object.csv.url
# this should probably be implemented as a model method
# how to read and process a CSV that has to be accessed over HTTP using smarter_csv
# I didn't have to specify this in my Rails setup, but you might
require 'open-uri'
# URL that points to where the CSV is hosted
csv_path = 'http://example.com/path/to/csv/file.csv'
# open the CSV file using open-uri
# you may have to specify encoding
csv_file = open(csv_path,'r')
# whatever options (optional)
csv_options = {
chunk_size: 100
}
# SmarterCSV can actually handle any object that responds to readline
# I didn't see that in the docs and I had to look at the source, which is why I'm writing this gist
SmarterCSV.process(csv_file, csv_options) do |chunk|
chunk.each do |row|
# do some stuff with row
end
end
@mklemme
Copy link

mklemme commented Jun 4, 2015

Awesome! I was trying to figure out why it couldn't find my csv uploaded to s3. +1

@deanfields
Copy link

thank you :)

@oojewale
Copy link

This is awesome. Though I did not use it for SmarterCSV. It gave an idea on how to pass remote files to google_drive library's upload method. Thanks!

@jolekszyk
Copy link

Thanks!

@denmarkmeralpis
Copy link

Awesome! Thanks for the gist!

@mrudult
Copy link

mrudult commented Mar 4, 2022

In Ruby 3, instead of open use URI.open(path,'r')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment