Skip to content

Instantly share code, notes, and snippets.

@thedeetch
Last active August 29, 2015 14:12
Show Gist options
  • Save thedeetch/64f44c320c46e8324bfb to your computer and use it in GitHub Desktop.
Save thedeetch/64f44c320c46e8324bfb to your computer and use it in GitHub Desktop.
Python large file read optimization
n = 0
f = open("my_huge_file.txt", "r")
for line in f:
n+=1
f.close()
print(n)
# 2m19.330s
n = [0]
f = open("my_huge_file.txt", "r")
for line in f:
n[0]+=1
f.close()
print(n)
# 1m36.63s
$n = 0;
open(FILE, "<my_huge_file.txt");
$n++ while (<FILE>);
close(FILE);
print $n;
# 1m36.755s
n = 0
f = open("my_huge_file.txt", "r")
line = f.readline()
while line:
n+=1
line = f.readline()
f.close()
print(n)
# 4m4.982s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment