Skip to content

Instantly share code, notes, and snippets.

@richardkmichael
Created February 6, 2019 00:51
Show Gist options
  • Save richardkmichael/2bc653a1473f5a312596034116b6c360 to your computer and use it in GitHub Desktop.
Save richardkmichael/2bc653a1473f5a312596034116b6c360 to your computer and use it in GitHub Desktop.
Using jsonpickle instead of pickle has benefits.
# jsonpickle is human readable for inspection (security, etc.) before unpickling.
# It is smaller, when native.
#
# 246099 cd33.jpkl
# 1432509 cd33.pkl
#
# 241K cd33.jpkl
# 1.4M cd33.pkl
# Though may be larger when compressed.
#
# 183K cd33.jpkl.xz
# 99K cd33.pkl.xz
import pandas
import jsonpickle
import jsonpickle.ext.pandas as jsonpickle_pandas
jsonpickle_pandas.register_handlers()
pfile = 'cd33.pkl'
# On Python 3, data pickled with Python 2.7 may not be unpickled as usual. However, it may unpickled with pandas itself.
# https://stackoverflow.com/a/46389140/420947
#
# with open(pfile, 'rb') as file:
# d = pickle.load(file)
#
# ==> ModuleNotFoundError: No module named 'pandas.indexes'
#
d = pandas.read_pickle(pfile)
json = jsonpickle.encode(d)
with open('cd33.jpkl', 'w') as file:
file.write(json)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment