Does Python have a stack/heap and how is memory managed?

by

How are variables and memory managed in Python.

Automagically! No, really, you just create an object and the Python Virtual Machine handles the memory needed and where it shall be placed in the memory layout.

Does it have a stack and a heap and what algorithm is used to manage
memory?

When we are talking about CPython it uses a private heap for storing objects. From the CPython C API documentation:

Memory management in Python involves a private heap containing all
Python objects and data structures. The management of this private
heap is ensured internally by the Python memory manager. The Python
memory manager has different components which deal with various
dynamic storage management aspects, like sharing, segmentation,
preallocation or caching.

Memory reclamation is mostly handled by reference counting. That is, the Python VM keeps an internal journal of how many references refer to an object, and automatically garbage collects it when there are no more references referring to it. In addition, there is a mechanism to break circular references (which reference counting can’t handle) by detecting unreachable “islands” of objects, somewhat in reverse of traditional GC algorithms that try to find all the reachable objects.

NOTE: Please keep in mind that this information is CPython specific. Other python implementations, such as pypy, iron python, jython and others may differ from one another and from CPython when it comes to their implementation specifics. To understand that better, it may help to understand that there is a difference between Python the semantics (the language) and the underlying implementation

Given this knowledge are there any recommendations on memory management for large number/data crunching?

Now I can not speak about this, but I am sure that NumPy (the most popular python library for number crunching) has mechanisms that handle memory consumption gracefully.

If you would like to know more about Python’s Internals take a look at these resources:

More Related Contents:

  1. How can I explicitly free memory in Python?
  2. How do I determine the size of an object in Python?
  3. when does Python allocate new memory for identical strings?
  4. What is the recommended way of allocating memory for a typed memory view?
  5. Is freeing handled differently for small/large numpy arrays?
  6. Is it possible to dereference variable id’s?
  7. Python 32-bit memory limits on 64bit windows
  8. Python out of memory on large CSV file (numpy)
  9. Python subprocess.Popen “OSError: [Errno 12] Cannot allocate memory”
  10. In-memory size of a Python structure
  11. Access memory address in python
  12. What does “del” do exactly?
  13. Keep persistent variables in memory between runs of Python script
  14. How to limit the heap size?
  15. Upper memory limit?
  16. How to increase Jupyter notebook Memory limit?
  17. How do i use subprocesses to force python to release memory?
  18. What do I need K.clear_session() and del model for (Keras with Tensorflow-gpu)?
  19. Extracting a zipfile to memory?
  20. How to clear memory completely of all matplotlib plots
  21. Why do ints require three times as much memory in Python?
  22. How do I check if PyTorch is using the GPU?
  23. How can I check the memory usage of objects in iPython?
  24. Data size in memory vs. on disk
  25. How to get largest possible precision? (Python – Decimal)
  26. Confused about Python’s id() [duplicate]
  27. Python multiprocessing memory usage
  28. CPython memory allocation
  29. memory error in python
  30. Why Python `Memory Error` with list `append()` lots of RAM left

Leave a Comment