Multithreading with Python and C api

by

A correct order of steps to perform what you are trying to do is:

  • In the main thread:

    1. Initialize Python using Py_Initialize*.
    2. Initialize Python threading support using PyEval_InitThreads().
    3. Start the C++ thread.

At this point, the main thread still holds the GIL.

  • In a C++ thread:
    1. Acquire the GIL using PyGILState_Ensure().
    2. Create a new Python thread object and start it.
    3. Release the GIL using PyGILState_Release().
    4. Sleep, do something useful or exit the thread.

Because the main thread holds the GIL, this thread will be waiting to acquire the GIL. If the main thread calls the Python API it may release the GIL from time to time allowing the Python thread to execute for a little while.

  • Back in the main thread:
    1. Release the GIL, enabling threads to run using PyEval_SaveThread()
    2. Before attempting to use other Python calls, reacquire the GIL using PyEval_RestoreThread()

I suspect that you are missing the last step – releasing the GIL in the main thread, allowing the Python thread to execute.

I have a small but complete example that does exactly that at this link.

More Related Contents:

  1. Python (and Python C API): __new__ versus __init__
  2. Create an object using Python’s C API
  3. ImportError: dynamic module does not define init function (initfizzbuzz)
  4. Embedding python in multithreaded C application
  5. How to run functions in parallel?
  6. Freezing/Hanging tkinter GUI in waiting for the thread to complete
  7. Threading in a PyQt application: Use Qt threads or Python threads?
  8. What is the use of join() in Python threading?
  9. Speed comparison with Project Euler: C vs Python vs Erlang vs Haskell
  10. How to start and stop a thread
  11. A very simple multithreading parallel URL fetching (without queue)
  12. It is more efficient to use if-return-return or if-else-return?
  13. threading ignores KeyboardInterrupt exception
  14. Multi-threaded use of SQLAlchemy
  15. Python time.sleep() vs event.wait()
  16. How to exit the entire application from a Python thread?
  17. Accessing OpenCV CUDA Functions from Python (No PyCUDA)
  18. How to get a faster speed when using multi-threading in python
  19. How to run multiple functions at the same time?
  20. I have Python on my Ubuntu system, but gcc can’t find Python.h
  21. What is the use of join() in threading?
  22. Python: How to NOT wait for a thread to finish to carry on? [duplicate]
  23. Where can I find source or algorithm of Python’s hash() function?
  24. Threads vs. Async
  25. How do I pass an async function to a thread target in Python?
  26. Decrypting Chromium cookies
  27. How to reliably generate Ethernet frame errors in software?
  28. Extending python – to swig, not to swig or Cython
  29. How do I run a long-running job in the background in Python
  30. Call python code from c via cython

Leave a Comment