Skip to content

Instantly share code, notes, and snippets.

@iAmGroute
Created March 24, 2024 11:23
Show Gist options
  • Save iAmGroute/e659523dcd622dba15573992f9912534 to your computer and use it in GitHub Desktop.
Save iAmGroute/e659523dcd622dba15573992f9912534 to your computer and use it in GitHub Desktop.
How to leak memory in Python
# A simple test showing possible memory leaks in Python
# import gc
import time
import resource
counter = 0
def get_new_id():
global counter
counter += 1
return counter - 1
class MyClass():
def __init__(self):
self.my_id = get_new_id()
self.friend = None
self.blob = b'1' * 1024 * 1024 # 1 MiB data
print('__init__', self.my_id)
def set_friend(self, friend):
self.friend = friend
def __del__(self):
if hasattr(self, 'my_id'):
print('__del__', self.my_id)
def my_fun():
print('my_fun: creating objects')
x = MyClass()
y = MyClass()
z = MyClass()
print('my_fun: setting friends')
x.set_friend(y)
y.set_friend(z)
z.set_friend(x)
print('my_fun: return')
def main():
for _ in range(100):
print('Memory KiB:', resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
my_fun()
# gc.collect()
print('Memory KiB:', resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
time.sleep(2)
if __name__ == '__main__':
main()
# You will notice that the memory keeps increasing on each iteration,
# and the past iteration objects' destructors are not being called,
# despite the fact that the objects are inaccessible, suggesting a memory leak.
# The culprit is the reference cycle together with the fact that the __del__ method is defined.
# CPython's garbage collector cannot decide which to (safely) call first!
# If you comment out the __del__ definition or break the cycle, the leak will be gone,
# as the GC will collect past objects on each iteration.
#
# Note that the topic is more nuanced than this, and with Python >= 3.4
# the GC will *eventually* destroy the objects, keeping memory usage bounded,
# while with older versions the process will eventually be killed by running OOM.
# more info here:
# https://docs.python.org/2.7/library/gc.html#gc.garbage
# https://docs.python.org/3.5/library/gc.html#gc.garbage
# https://peps.python.org/pep-0442/#disposal-of-cyclic-isolates
# Fun things to try:
# - Run with python2.7
# - Comment out memory printing and sleep(), increase the iterations
# and run with different interpreter versions, while watching memory usage.
# - Comment out "z.set_friend(x)"
# - Uncomment the gc lines
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment