Python Multiprocessing: Handling Child Errors in Parent

by

Why not to let the Process to take care of its own exceptions, like this:

from __future__ import print_function
import multiprocessing as mp
import traceback

class Process(mp.Process):
    def __init__(self, *args, **kwargs):
        mp.Process.__init__(self, *args, **kwargs)
        self._pconn, self._cconn = mp.Pipe()
        self._exception = None

    def run(self):
        try:
            mp.Process.run(self)
            self._cconn.send(None)
        except Exception as e:
            tb = traceback.format_exc()
            self._cconn.send((e, tb))
            # raise e  # You can still rise this exception if you need to

    @property
    def exception(self):
        if self._pconn.poll():
            self._exception = self._pconn.recv()
        return self._exception

Now you have, both error and traceback at your hands:

def target():
    raise ValueError('Something went wrong...')

p = Process(target = target)
p.start()
p.join()

if p.exception:
    error, traceback = p.exception
    print(traceback)

Regards,
Marek

More Related Contents:

  1. Multiprocessing vs Threading Python [duplicate]
  2. RuntimeError on windows trying python multiprocessing
  3. multiprocessing: How do I share a dict among multiple processes?
  4. multiprocessing: sharing a large read-only object between processes?
  5. Python sharing a lock between processes
  6. How to use multiprocessing queue in Python?
  7. where to put freeze_support() in a Python script?
  8. Compulsory usage of if __name__==”__main__” in windows while using multiprocessing [duplicate]
  9. Multiprocessing – Pipe vs Queue
  10. Concurrent.futures vs Multiprocessing in Python 3
  11. Parallelizing a Numpy vector operation
  12. multiprocessing.pool.MaybeEncodingError: ‘TypeError(“cannot serialize ‘_io.BufferedReader’ object”,)’
  13. High Memory Usage Using Python Multiprocessing
  14. “chunksize” parameter in multiprocessing.Pool.map
  15. Connection problems with SQLAlchemy and multiple processes
  16. OSError 38 [Errno 38] with multiprocessing
  17. Python raising FileNotFoundError for file name returned by os.listdir
  18. Can multiprocessing Process class be run from IDLE
  19. IndexError: too many indices for array
  20. Python multiprocess profiling
  21. What’s the difference between ThreadPool vs Pool in the multiprocessing module?
  22. Python OpenCV multiprocessing cv2.VideoCapture mp4
  23. Using 100% of all cores with the multiprocessing module
  24. Mix Python Twisted with multiprocessing?
  25. Can I get a return value from multiprocessing.Process?
  26. How to utilize all cores with python multiprocessing
  27. Tensorflow and Multiprocessing: Passing Sessions
  28. How can I share a class between processes?
  29. Multiprocessing useless with urllib2?
  30. Python multiprocessing linux windows difference

Leave a Comment