Skip to content

Instantly share code, notes, and snippets.

@arose13
Last active April 8, 2017 00:45
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save arose13/facfb91b609d453f3ad840417faa503a to your computer and use it in GitHub Desktop.
Save arose13/facfb91b609d453f3ad840417faa503a to your computer and use it in GitHub Desktop.
Out Of Core CSV Transposing. Constant memory use. Arbitrary CSV size.
import csv
def transpose_csv_out_of_core(csv_path, output_csv_path='transposed.csv', delimiter=','):
"""
On my laptop it can transpose at ~375,000 lines a sec
:param csv_path:
:param output_csv_path:
:param delimiter:
:return:
"""
transposed_iterator = zip(*csv.reader(open(csv_path)))
with open(output_csv_path, 'w') as out:
for row in transposed_iterator:
out.write(delimiter.join(row) + '\n')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment