Skip to content

Instantly share code, notes, and snippets.

@AlexDel
Created November 26, 2015 08:01
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save AlexDel/abe95cdfd47fce09a0ac to your computer and use it in GitHub Desktop.
Save AlexDel/abe95cdfd47fce09a0ac to your computer and use it in GitHub Desktop.
csv chunker
import csv
filename = 'C:\Users\AlexDel\Desktop\horeca backup\\final.csv'
chunksize = 1000
chunksdir = 'C:\Users\AlexDel\Desktop\horeca backup\\'
f = open(filename, 'r')
myreader = csv.reader(f, delimiter=',')
headings = myreader.next()
def chunks(l, n):
"""Yield successive n-sized chunks from l."""
for i in xrange(0, len(l), n):
yield l[i:i+n]
rows = [i for i in myreader]
chunked_rows = [chunk for chunk in chunks(rows,chunksize)]
for i in range(len(chunked_rows)):
c = open('C:\Users\AlexDel\Desktop\horeca backup\\chunk_' + str(i) + '.csv', 'w+')
mywriter = csv.writer(c, delimiter=',')
mywriter.writerow(headings)
for chunk_row in chunked_rows[i]:
mywriter.writerow(chunk_row)
c.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment