Multiprocessing Share Unserializable Objects Between Processes

by

Most of the time it’s not really desirable to pass the reference of an existing object to another process. Instead you create your class you want to share between processes:

class MySharedClass:
    # stuff...

Then you make a proxy manager like this:

import multiprocessing.managers as m
class MyManager(m.BaseManager):
    pass # Pass is really enough. Nothing needs to be done here.

Then you register your class on that Manager, like this:

MyManager.register("MySharedClass", MySharedClass)

Then once the manager is instanciated and started, with manager.start() you can create shared instances of your class with manager.MySharedClass. This should work for all needs. The returned proxy works exactly like the original objects, except for some exceptions described in the documentation.

More Related Contents:

  1. multiprocessing: Understanding logic behind `chunksize`
  2. multiprocessing vs multithreading vs asyncio in Python 3
  3. Concurrent.futures vs Multiprocessing in Python 3
  4. Python selenium multiprocessing
  5. How to share numpy random state of a parent process with child processes?
  6. multiprocessing vs multithreading vs asyncio
  7. What’s the difference between ThreadPool vs Pool in the multiprocessing module?
  8. Is having a concurrent.futures.ThreadPoolExecutor call dangerous in a FastAPI endpoint?
  9. Python OpenCV multiprocessing cv2.VideoCapture mp4
  10. Using 100% of all cores with the multiprocessing module
  11. Multiprocessing AsyncResult.get() hangs in Python 3.7.2 but not in 3.6
  12. How to limit number of CPU’s used by a python script w/o terminal or multiprocessing library?
  13. Print elements of list horizontally [closed]
  14. How do i get my code to return the the answer without the extra '' and ()
  15. Removing Capitalized Strings from Input File in Python3.4?
  16. Python eval() not multiplying properly
  17. Python – concatenate NoneType and string in a new list
  18. Import error: No module name urllib2
  19. Fixed digits after decimal with f-strings
  20. Print multiple arguments in Python
  21. Python, TypeError: unhashable type: ‘list’
  22. zip variable empty after first use
  23. Python 3: How to specify stdin encoding
  24. How to create virtual env with python3
  25. User input boolean in python
  26. Python : The second for loop is not running
  27. Setting a fixed FPS in Pygame, Python 3
  28. How to fix this DeprecationWarning
  29. Print a float number in normal form, not exponential form / scientific notation [duplicate]
  30. Sharing a lock between gunicorn workers

Leave a Comment