Skip to content

Instantly share code, notes, and snippets.

@aabadie

aabadie/profile_memory.py

Last active May 18, 2016 09:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save aabadie/7cba3385406d1cec7d3dd4407ba3f164 to your computer and use it in GitHub Desktop.
Save aabadie/7cba3385406d1cec7d3dd4407ba3f164 to your computer and use it in GitHub Desktop.
Download ZIP
Base python script used to profile memory usage of joblib dump/load functions
Raw
profile_memory.py
import numpy as np
import joblib
obj = [np.ones((5000, 5000)), np.random.random((5000, 5000))]
joblib.dump(obj, '/tmp/test.pkl', compress=True)
joblib.load('/tmp/test.pkl')
@aabadie
Copy link
Author

aabadie commented May 17, 2016 •
edited

How to use this script

  1. Add @profile decorator to joblib dump and load functions in joblib/numpy_pickle.py
  2. Run the following command in a console:
$ mprof run profile_memory.py && mprof plot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment