Does python support multiprocessor/multicore programming?

by

There is no such thing as “multiprocessor” or “multicore” programming. The distinction between “multiprocessor” and “multicore” computers is probably not relevant to you as an application programmer; it has to do with subtleties of how the cores share access to memory.

In order to take advantage of a multicore (or multiprocessor) computer, you need a program written in such a way that it can be run in parallel, and a runtime that will allow the program to actually be executed in parallel on multiple cores (and operating system, although any operating system you can run on your PC will do this). This is really parallel programming, although there are different approaches to parallel programming. The ones that are relevant to Python are multiprocessing and multithreading.

In languages like C, C++, Java, and C#, you can write parallel programs by executing multiple threads. The global interpreter lock in the CPython and PyPy runtimes preclude this option; but only for those runtimes. (In my personal opinion, multithreading is dangerous and tricky and it is generally a good thing that Python encourages you not to consider it as a way to get a performance advantage.)

If you want to write a parallel program which can run on multiple cores in Python, you have a few different options:

  • Write a multithreaded program using the threading module and run it in the IronPython or Jython runtime.
  • Use the processing module, (now included in Python 2.6 as the multiprocessing module), to run your code in multiple processes at once.
  • Use the subprocess module to run multiple python interpreters and communicate between them.
  • Use Twisted and Ampoule. This has the advantage of not just running your code across different processes, but (if you don’t share access to things like files) potentially across different computers as well.

No matter which of these options you choose, you will need to understand how to split the work that your program is doing up into chunks that make sense to separate. Since I’m not sure what kind of programs you are thinking of writing, it would be difficult to provide a useful example.

More Related Contents:

  1. multi core processing in for loop using numpy
  2. Python Global Interpreter Lock (GIL) workaround on multi-core systems using taskset on Linux?
  3. Does anyone know how to get info from a csv?
  4. Print the result obtained from an external window but without opening the window (the selected checkboxes are automatically saved in the database)
  5. Variable table name in sqlite
  6. testing whether a Numpy array contains a given row
  7. main() function doesn’t run when running script
  8. How do I convert a datetime to date?
  9. Convert string date to timestamp in Python
  10. Is it ever useful to use Python’s input over raw_input?
  11. Python in Xcode 4+?
  12. How to clamp an integer to some range?
  13. Split a string to even sized chunks
  14. How to escape curly-brackets in f-strings? [duplicate]
  15. Python webdriver to handle pop up browser windows which is not an alert
  16. How to kill a python child process created with subprocess.check_output() when the parent dies?
  17. With the Python Imaging Library (PIL), how does one compose an image with an alpha channel over another image?
  18. Is there a need to close files that have no reference to them?
  19. In Python, how do I indicate I’m overriding a method?
  20. python copy files by wildcards
  21. How to execute arithmetic operations between Model fields in django
  22. Python equivalent to ‘hold on’ in Matlab
  23. Plot multiple columns of pandas DataFrame on the bar chart
  24. Get column name where value is something in pandas dataframe
  25. Python pi calculation?
  26. Same name for classmethod and instancemethod
  27. UnicodeDecodeError when reading CSV file in Pandas
  28. First and last row cut in half of heatmap plot
  29. Run manage.py from AWS EB Linux instance
  30. How to perform filtering with a Django JSONField?

Leave a Comment