Tracking *maximum* memory usage by a Python function

by

It is possible to do this with memory_profiler. The function memory_usage returns a list of values, these represent the memory usage over time (by default over chunks of .1 second). If you need the maximum, just take the max of that list. Little example:

from memory_profiler import memory_usage
from time import sleep

def f():
    # a function that with growing
    # memory consumption
    a = [0] * 1000
    sleep(.1)
    b = a * 100
    sleep(.1)
    c = b * 100
    return a

mem_usage = memory_usage(f)
print('Memory usage (in chunks of .1 seconds): %s' % mem_usage)
print('Maximum memory usage: %s' % max(mem_usage))

In my case (memory_profiler 0.25) if prints the following output:

Memory usage (in chunks of .1 seconds): [45.65625, 45.734375, 46.41015625, 53.734375]
Maximum memory usage: 53.734375

More Related Contents:

  1. How do I profile memory usage in Python?
  2. Which Python memory profiler is recommended? [closed]
  3. Why does Python code run faster in a function?
  4. Is it possible to dereference variable id’s?
  5. when does Python allocate new memory for identical strings?
  6. What is the difference between contiguous and non-contiguous arrays?
  7. How do I release memory used by a pandas dataframe?
  8. Python out of memory on large CSV file (numpy)
  9. In-memory size of a Python structure
  10. Access memory address in python
  11. What does “del” do exactly?
  12. Keep persistent variables in memory between runs of Python script
  13. How to limit the heap size?
  14. Upper memory limit?
  15. How to increase Jupyter notebook Memory limit?
  16. How to measure time taken between lines of code in python?
  17. How can you get the call tree with Python profilers?
  18. Python: Reducing memory usage of dictionary
  19. Searching for a string in a large text file – profiling various methods in python
  20. High Memory Usage Using Python Multiprocessing
  21. Is freeing handled differently for small/large numpy arrays?
  22. Python/Numpy MemoryError
  23. How can I check the memory usage of objects in iPython?
  24. Python multiprocessing application is getting stuck in docker container
  25. How to clear variables in ipython?
  26. In Python, what is `sys.maxsize`?
  27. Is there any working memory profiler for Python3 [closed]
  28. Limit Python VM memory
  29. How can I profile a multithread program in Python?
  30. Why doesn’t Python’s mmap work with large files?

Leave a Comment