Exception thrown in multiprocessing Pool not detected

by

Maybe I’m missing something, but isn’t that what the get method of the Result object returns? See Process Pools.

class multiprocessing.pool.AsyncResult

The class of the result returned by Pool.apply_async() and Pool.map_async().get([timeout])
Return the result when it arrives. If timeout is not None and the result does not arrive within
timeout seconds then multiprocessing.TimeoutError is raised. If the remote
call raised an exception then that exception will be reraised by get().

So, slightly modifying your example, one can do

from multiprocessing import Pool

def go():
    print(1)
    raise Exception("foobar")
    print(2)

p = Pool()
x = p.apply_async(go)
x.get()
p.close()
p.join()

Which gives as result

1
Traceback (most recent call last):
  File "rob.py", line 10, in <module>
    x.get()
  File "/usr/lib/python2.6/multiprocessing/pool.py", line 422, in get
    raise self._value
Exception: foobar

This is not completely satisfactory, since it does not print the traceback, but is better than nothing.

UPDATE: This bug has been fixed in Python 3.4, courtesy of Richard Oudkerk. See the issue get method of multiprocessing.pool.Async should return full traceback.

More Related Contents:

  1. Python: Getting a traceback from a multiprocessing.Process
  2. Multiprocessing vs Threading Python [duplicate]
  3. multiprocessing: How do I share a dict among multiple processes?
  4. Is it a good practice to use try-except-else in Python?
  5. Catch a thread’s exception in the caller thread?
  6. selenium.common.exceptions.InvalidSelectorException with “span:contains(‘string’)”
  7. multiprocessing: sharing a large read-only object between processes?
  8. Python sharing a lock between processes
  9. How to use multiprocessing queue in Python?
  10. where to put freeze_support() in a Python script?
  11. Multiprocessing a for loop?
  12. Python exception chaining [duplicate]
  13. Importing installed package from script with the same name raises “AttributeError: module has no attribute” or “ImportError: cannot import name”
  14. In Python, how does one catch warnings as if they were exceptions?
  15. Concurrent.futures vs Multiprocessing in Python 3
  16. Parallelizing a Numpy vector operation
  17. Python type hinting with exceptions
  18. multiprocessing.pool.MaybeEncodingError: ‘TypeError(“cannot serialize ‘_io.BufferedReader’ object”,)’
  19. High Memory Usage Using Python Multiprocessing
  20. “chunksize” parameter in multiprocessing.Pool.map
  21. Connection problems with SQLAlchemy and multiple processes
  22. Can multiprocessing Process class be run from IDLE
  23. Python: How to ignore an exception and proceed? [duplicate]
  24. Python multiprocess profiling
  25. What’s the difference between ThreadPool vs Pool in the multiprocessing module?
  26. Is there a difference between “raise exception()” and “raise exception” without parenthesis?
  27. Using 100% of all cores with the multiprocessing module
  28. Mix Python Twisted with multiprocessing?
  29. Using multiprocessing.Process with a maximum number of simultaneous processes
  30. Understanding Multiprocessing: Shared Memory Management, Locks and Queues in Python

Leave a Comment