Same output in different workers in multiprocessing

by

I think you’ll need to re-seed the random number generator using numpy.random.seed in your do_calculation function.

My guess is that the random number generator (RNG) gets seeded when you import the module. Then, when you use multiprocessing, you fork the current process with the RNG already seeded — Thus, all your processes are sharing the same seed value for the RNG and so they’ll generate the same sequences of numbers.

e.g.:

def do_calculation(data):
    np.random.seed()
    rand=np.random.randint(10)
    print data, rand
    return data * 2

More Related Contents:

  1. Shared-memory objects in multiprocessing
  2. What are the differences between the threading and multiprocessing modules?
  3. Sharing a result queue among several processes
  4. multiprocessing: Understanding logic behind `chunksize`
  5. multiprocessing.dummy in Python is not utilising 100% cpu
  6. Using the multiprocessing module for cluster computing
  7. Poor scaling of multiprocessing Pool.map() on a list of large objects: How to achieve better parallel scaling in python?
  8. Python multiprocessing.Pool: AttributeError
  9. Modify object in python multiprocessing
  10. multiprocessing pool example does not work and freeze the kernel
  11. Tensorflow and Multiprocessing: Passing Sessions
  12. How to run functions in parallel?
  13. Dead simple example of using Multiprocessing Queue, Pool and Locking
  14. multiprocessing.Pool: What’s the difference between map_async and imap?
  15. Log output of multiprocessing.Process
  16. pandas multiprocessing apply
  17. Multiprocessing : use tqdm to display a progress bar
  18. Appending to the same list from different processes using multiprocessing
  19. Exception thrown in multiprocessing Pool not detected
  20. Why can I pass an instance method to multiprocessing.Process, but not a multiprocessing.Pool?
  21. How do you pass a Queue reference to a function managed by pool.map_async()?
  22. Chrome crashes after several hours while multiprocessing using Selenium through Python
  23. What kind of problems (if any) would there be combining asyncio with multiprocessing?
  24. What is the difference between multiprocessing and subprocess?
  25. Python Multiprocessing Lib Error (AttributeError: __exit__)
  26. Is there a simple process-based parallel map for python?
  27. Show the progress of a Python multiprocessing pool imap_unordered call?
  28. Using multiprocessing.Process with a maximum number of simultaneous processes
  29. Understanding Multiprocessing: Shared Memory Management, Locks and Queues in Python
  30. aiohttp: rate limiting parallel requests

Leave a Comment