Last active
December 17, 2015 20:33
-
-
Save kain88-de/5487e9d9c8f7d0e4995b to your computer and use it in GitHub Desktop.
offset fileformat speed benchmark
Nope numpy is slower. once you actually read the data
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I forked the notebook https://gist.github.com/orbeckst/2d2e8c12288b202a6a02 and added benchmarks for
cPickle.HIGHEST_PROTOCOL
numpy.savez
Bottomline: if pickle then always use the latest protocol because it reduces file size to 1/3 and speeds up loading by a factor of 160 (in this case). The numpy native format is even faster, maybe another factor of 50.