Skip to content

Instantly share code, notes, and snippets.

@gouravtiwari
Created March 13, 2017 18:34
Show Gist options
  • Save gouravtiwari/0b825aabd45e79f8e0598db754b12534 to your computer and use it in GitHub Desktop.
Save gouravtiwari/0b825aabd45e79f8e0598db754b12534 to your computer and use it in GitHub Desktop.
CSV file sanitizer, which will remove error-rows and create a new sanitized file
require 'csv'
valid_rows = 0
invalid_rows = 0
error_file = File.open("/path/to/invalid.csv", "w")
correct_file = File.open("path/to/valid.csv", "a+")
File.open("path/to/original.csv").each do |line|
begin
print "."
if CSV.parse_line(line)
valid_file.write(line)
valid_rows += 1
end
rescue Exception => e
invalid_rows += 1
invalid_file.write(line)
e.backtrace.each{ |b| print b + "\n" }
end
end
p "valid_rows: #{valid_rows}"
p "invalid_rows: #{invalid_rows}"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment