Multiprocessing example giving AttributeError

by

This problem seems to be a design feature of multiprocessing.Pool. See https://bugs.python.org/issue25053. For some reason Pool does not always work with objects not defined in an imported module. So you have to write your function into a different file and import the module.

File: defs.py

def f(x):
    return x*x

File: run.py

from multiprocessing import Pool
import defs

 if __name__ == '__main__':
    with Pool(5) as p:
        print(p.map(defs.f, [1, 2, 3]))

If you use print or a different built-in function, the example should work. If this is not a bug (according to the link), the given example is chosen badly.

More Related Contents:

  1. Can’t pickle when using multiprocessing Pool.map()
  2. Background thread with QThread in PyQt
  3. Run certain code every n seconds [duplicate]
  4. What are the differences between the threading and multiprocessing modules?
  5. How to use multiprocessing queue in Python?
  6. How to terminate a thread when main program ends?
  7. How to make built-in containers (sets, dicts, lists) thread safe?
  8. Using win32com with multithreading
  9. Threads and tkinter
  10. Multithreaded web server in python
  11. Program freezing during the execution of a function in Tkinter
  12. How to run multiple jobs in one Sparkcontext from separate threads in PySpark?
  13. Kill a running subprocess call
  14. What is “thread local storage” in Python, and why do I need it?
  15. Python threading. How do I lock a thread?
  16. Python multithreaded print statements delayed until all threads complete execution
  17. why isn’t numpy.mean multithreaded?
  18. Python Threading inside a class
  19. Meaning of daemon property on Python Threads
  20. Threaded Tkinter script crashes when creating the second Toplevel widget
  21. Are Generators Threadsafe?
  22. Is Keras thread safe?
  23. Python Tkinter Label redrawing every 10 seconds
  24. How to obtain a Thread id in Python?
  25. Dump stacktraces of all active Threads
  26. How to use QThread correctly in pyqt with moveToThread()?
  27. Is it thread-safe when using tf.Session in inference service?
  28. ‘sys.excepthook’ and threading
  29. Are locks unnecessary in multi-threaded Python code because of the GIL?
  30. Python threading interrupt sleep

Leave a Comment