Skip to content

Instantly share code, notes, and snippets.

@senderle
Created November 6, 2015 19:40
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save senderle/2d48a5493ad8aecc56c5 to your computer and use it in GitHub Desktop.
Save senderle/2d48a5493ad8aecc56c5 to your computer and use it in GitHub Desktop.
An extremely simple object-size checker that can handle anything returned by JSON.
from collections import Sequence, Mapping
from sys import getsizeof
def traverse(obj):
if isinstance(obj, basestring):
yield obj
elif isinstance(obj, Sequence):
yield obj
for o in obj:
for inner_o in traverse(o):
yield inner_o
elif isinstance(obj, Mapping):
yield obj
for k, v in obj.iteritems():
for inner_k in traverse(k):
yield inner_k
for inner_v in traverse(v):
yield inner_v
else:
yield obj
def get_total_size(x):
unique = {id(o): o for o in traverse(x)}
return sum(map(getsizeof, unique.values()))
if __name__ == '__main__':
print get_total_size({'a': 'b', 'c': ['d', 'e', 'f'], 'g': {'h': 'i'}})
@senderle
Copy link
Author

senderle commented Nov 6, 2015

Warning:

This checks for duplicate objects when it calculates the memory usage, but it does not check for cycles during traversal and will explode if they are present.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment