Skip to content

Instantly share code, notes, and snippets.

@rosdyana

rosdyana/split.py

Created March 27, 2020 02:25
Show Gist options
  • Save rosdyana/20c6cbe930958c0b431ffd875ef7ab4d to your computer and use it in GitHub Desktop.
Save rosdyana/20c6cbe930958c0b431ffd875ef7ab4d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
split.py
import pandas as pd
import sys
CHUNK_SIZE = int(sys.argv[2])
rows = pd.read_csv(sys.argv[1], chunksize=CHUNK_SIZE)
for i, chuck in enumerate(rows):
# i is for chunk number of each iteration
chuck.to_csv('out{}.csv'.format(i))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment