Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
csv chunker
import csv
filename = 'C:\Users\AlexDel\Desktop\horeca backup\\final.csv'
chunksize = 1000
chunksdir = 'C:\Users\AlexDel\Desktop\horeca backup\\'
f = open(filename, 'r')
myreader = csv.reader(f, delimiter=',')
headings = myreader.next()
def chunks(l, n):
"""Yield successive n-sized chunks from l."""
for i in xrange(0, len(l), n):
yield l[i:i+n]
rows = [i for i in myreader]
chunked_rows = [chunk for chunk in chunks(rows,chunksize)]
for i in range(len(chunked_rows)):
c = open('C:\Users\AlexDel\Desktop\horeca backup\\chunk_' + str(i) + '.csv', 'w+')
mywriter = csv.writer(c, delimiter=',')
mywriter.writerow(headings)
for chunk_row in chunked_rows[i]:
mywriter.writerow(chunk_row)
c.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.