Skip to content

Instantly share code, notes, and snippets.

@cscherrer
Created December 4, 2017 22:58
Show Gist options
  • Save cscherrer/40443722307ac42d7926163f25e33270 to your computer and use it in GitHub Desktop.
Save cscherrer/40443722307ac42d7926163f25e33270 to your computer and use it in GitHub Desktop.
Split a (possibly very large) CSV file into a file for each column, using Julia
# https://stackoverflow.com/questions/632475/regex-to-pick-commas-outside-of-quotes
comma = r"(,)(?=(?:[^\"]|\"[^\"]*\")*$)"
open(datafile) do csv
header = split(readline(csv),comma)
numCols = length(header)
outs=[open("$f.txt","w") for f in header]
while !eof(csv)
vals = split(readline(csv), comma)
foreach(zip(outs,vals)) do outval
(out,val) = outval
write(out, val) + write(out, '\n')
end
end
map(close, outs)
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment