Sharing a result queue among several processes

by

Try using multiprocessing.Manager to manage your queue and to also make it accessible to different workers.

import multiprocessing
def worker(name, que):
    que.put("%d is done" % name)

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=3)
    m = multiprocessing.Manager()
    q = m.Queue()
    workers = pool.apply_async(worker, (33, q))

More Related Contents:

  1. multiprocessing: Understanding logic behind `chunksize`
  2. Python multiprocessing.Pool: AttributeError
  3. multiprocessing pool example does not work and freeze the kernel
  4. How to use multiprocessing pool.map with multiple arguments
  5. Python multiprocessing PicklingError: Can’t pickle
  6. Shared-memory objects in multiprocessing
  7. What are the differences between the threading and multiprocessing modules?
  8. Shared variable in python’s multiprocessing
  9. multiprocessing.dummy in Python is not utilising 100% cpu
  10. multiprocessing.Pool: What’s the difference between map_async and imap?
  11. Script using multiprocessing module does not terminate
  12. Same output in different workers in multiprocessing
  13. Filling a queue and managing multiprocessing in python
  14. Can I use a multiprocessing Queue in a function called by Pool.imap?
  15. Using the multiprocessing module for cluster computing
  16. How do you pass a Queue reference to a function managed by pool.map_async()?
  17. Leveraging “Copy-on-Write” to Copy Data to Multiprocessing.Pool() Worker Processes
  18. Modify object in python multiprocessing
  19. Processes stuck in loop with PyInstaller-executable
  20. How to get the return value of a function passed to multiprocessing.Process?
  21. how to to terminate process using python’s multiprocessing
  22. Python Multiprocessing error: AttributeError: module ‘__main__’ has no attribute ‘__spec__’
  23. Multiprocessing AsyncResult.get() hangs in Python 3.7.2 but not in 3.6
  24. Creating and updating nested dictionaries and lists inside multiprocessing.Manager object
  25. Python multiprocessing application is getting stuck in docker container
  26. Understanding Multiprocessing: Shared Memory Management, Locks and Queues in Python
  27. Understanding Multiprocessing: Shared Memory Management, Locks and Queues in Python
  28. Multiprocessing: use only the physical cores?
  29. How can I abort a task in a multiprocessing.Pool after a timeout?
  30. How to limit number of CPU’s used by a python script w/o terminal or multiprocessing library?

Leave a Comment