- 2020 Load Data Faster in Python With Compressed Pickles
➝ focus on file size, bz2 and cPickle, 27x smaller, a little bit faster - 2019 The Best Format to Save Pandas Data
➝ Feather (Apache Arrow) - 2014 Don't Pickle Your Data
➝ MessagePack > Thrift > JSON > cPickle > Pickle (benchmark rate and size) - 2010 Pickle vs JSON — Which is Faster? ➝ json > pickle (25 times faster in reading and 15 times faster in writing)
- GitHub
- 2013 What is faster - Loading a pickled dictionary object or Loading a JSON file - to a dictionary?
➝ yajl > ujson > simplejson > json > cpickle > pickle - 2014 How to Reduce the time taken to load a pickle file in python
➝ Some tests, but mistake: pickle instead of cpickle used in loading cpickle
➝ Speed depends on data type as mentioned here with link to thread comment ➝ you can specify cPickle.HIGHEST_PROTOCOL
- 2013 What is faster - Loading a pickled dictionary object or Loading a JSON file - to a dictionary?
- 2021 Polars: The fastest DataFrame library you’ve never heard of
- 2019 Data Science I/O - A baseline benchmark for 2019
Parquet > CSV (no other formats included)