Skip to content

Instantly share code, notes, and snippets.

@ericlagergren
Created July 9, 2014 07:24
Show Gist options
  • Save ericlagergren/6bc4f7c462703657c25a to your computer and use it in GitHub Desktop.
Save ericlagergren/6bc4f7c462703657c25a to your computer and use it in GitHub Desktop.
Count lines in large csv files
#!/usr/bin/env python
def bufcount(filename):
f = open(filename)
lines = 0
buf_size = 1024 * 1024
read_f = f.read # loop optimization
buf = read_f(buf_size)
while buf:
lines += buf.count('\n')
buf = read_f(buf_size)
print str(lines) + ' lines'
bufcount('file.txt')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment