How to run functions in parallel?

by

You could use threading or multiprocessing.

Due to peculiarities of CPython, threading is unlikely to achieve true parallelism. For this reason, multiprocessing is generally a better bet.

Here is a complete example:

from multiprocessing import Process

def func1():
  print 'func1: starting'
  for i in xrange(10000000): pass
  print 'func1: finishing'

def func2():
  print 'func2: starting'
  for i in xrange(10000000): pass
  print 'func2: finishing'

if __name__ == '__main__':
  p1 = Process(target=func1)
  p1.start()
  p2 = Process(target=func2)
  p2.start()
  p1.join()
  p2.join()

The mechanics of starting/joining child processes can easily be encapsulated into a function along the lines of your runBothFunc:

def runInParallel(*fns):
  proc = []
  for fn in fns:
    p = Process(target=fn)
    p.start()
    proc.append(p)
  for p in proc:
    p.join()

runInParallel(func1, func2)

More Related Contents:

  1. Multiprocessing vs Threading Python [duplicate]
  2. Can’t pickle when using multiprocessing Pool.map()
  3. multiprocessing.Pool: When to use apply, apply_async or map?
  4. What are the differences between the threading and multiprocessing modules?
  5. How to use multiprocessing queue in Python?
  6. Processing single file from multiple processes
  7. multiprocessing vs multithreading vs asyncio in Python 3
  8. What kind of problems (if any) would there be combining asyncio with multiprocessing?
  9. multiprocessing vs multithreading vs asyncio
  10. Using multiprocessing.Process with a maximum number of simultaneous processes
  11. How to use asyncio with threading in Python is Method threading.Timer
  12. python multithreading wait till all threads finished
  13. RuntimeError: main thread is not in main loop
  14. Running infinite loops using threads in python
  15. How do you stop numpy from multithreading? [duplicate]
  16. How to combine python asyncio with threads?
  17. Multiprocessing – Pipe vs Queue
  18. Concurrent.futures vs Multiprocessing in Python 3
  19. Can you perform multi-threaded tasks within Django?
  20. Why is a Python I/O bound task not blocked by the GIL?
  21. multiprocessing.pool.MaybeEncodingError: ‘TypeError(“cannot serialize ‘_io.BufferedReader’ object”,)’
  22. High Memory Usage Using Python Multiprocessing
  23. OSError 38 [Errno 38] with multiprocessing
  24. How to use threading in Python?
  25. Can multiprocessing Process class be run from IDLE
  26. Python creating a shared variable between threads
  27. Using 100% of all cores with the multiprocessing module
  28. Threading in python using queue
  29. Execute Python function in Main thread from call in Dummy thread
  30. Why does Paramiko hang if you use it while loading a module?

Leave a Comment