salma71/intuitive_read

Created February 8, 2021 13:59

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/salma71/3dbd7eb6429341357e6795302353bdf7.js"></script>
Save salma71/3dbd7eb6429341357e6795302353bdf7 to your computer and use it in GitHub Desktop.

Download ZIP

Saved from https://stackoverflow.com/questions/17444679/reading-a-huge-csv-file

Raw

intuitive_read

	import pandas as pd
	chunksize = 10 ** 8
	for chunk in pd.read_csv(filename, chunksize=chunksize):
	process(chunk)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment