Skip to content

Instantly share code, notes, and snippets.

@gwenshap
Created March 13, 2014 17:43
Show Gist options
  • Save gwenshap/9533211 to your computer and use it in GitHub Desktop.
Save gwenshap/9533211 to your computer and use it in GitHub Desktop.
Python for cleaning CSVs from in-record newlines
#!/usr/bin/python
import csv
import sys
import argparse
import io
csv.field_size_limit(sys.maxsize)
parser = argparse.ArgumentParser(description='Clean csv of in-line newlines')
parser.add_argument('infile',help='Path to input CSV file');
parser.add_argument('outfile',help='Path to output CSV file');
args = parser.parse_args();
inf = file(args.infile,'r')
outf = file(args.outfile,'w')
try:
reader = csv.reader(inf)
writer = csv.writer(outf)
for row in reader:
newrow = [col.replace('\r\n', '##').replace('\n','##') for col in row]
writer.writerow(newrow)
finally:
inf.close()
outf.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment