Python code performance decreases with threading

by

This is sadly how things are in CPython, mainly due to the Global Interpreter Lock (GIL). Python code that’s CPU-bound simply doesn’t scale across threads (I/O-bound code, on the other hand, might scale to some extent).

There is a highly informative presentation by David Beazley where he discusses some of the issues surrounding the GIL. The video can be found here (thanks @Ikke!)

My recommendation would be to use the multiprocessing module instead of multiple threads.

More Related Contents:

  1. Python OpenCV streaming from camera – multithreading, timestamps
  2. How to capture multiple camera streams with OpenCV?
  3. Python threads all executing on a single core
  4. why isn’t numpy.mean multithreaded?
  5. Tkinter GUI, I/O & Threading: When to use queues, when events?
  6. Fastest save and load options for a numpy array
  7. How can I use threading in Python?
  8. Replace values in a pandas series via dictionary efficiently
  9. How to limit execution time of a function call?
  10. Python threading multiple bash subprocesses?
  11. What exactly is file.flush() doing?
  12. Make 2 functions run at the same time
  13. Unable to load files using pickle and multiple modules
  14. Thread Safety in Python’s dictionary
  15. Numpy and line intersections
  16. Python Requests vs PyCurl Performance
  17. Python Subprocess.Popen from a thread
  18. Thread vs. Threading
  19. numpy float: 10x slower than builtin in arithmetic operations?
  20. Reading a file with a specified delimiter for newline
  21. multiprocessing vs multithreading vs asyncio
  22. Fastest way to write huge data in file
  23. Using a global dictionary with threads in Python
  24. No speedup when summing uint16 vs uint64 arrays with NumPy?
  25. Python thread name doesn’t show up on ps or htop
  26. Threads are printing at the same time messing up the text output
  27. Access a QLCDNumber object from a external function
  28. Which is the most efficient way to iterate through a list in python?
  29. Python Global Interpreter Lock (GIL) workaround on multi-core systems using taskset on Linux?
  30. Length of longest word in a list

Leave a Comment