PyEval_InitThreads in Python 3: How/when to call it? (the saga continues ad nauseam)

by

Your understanding is correct: invoking PyEval_InitThreads does, among other things, acquire the GIL. In a correctly written Python/C application, this is not an issue because the GIL will be unlocked in time, either automatically or manually.

If the main thread goes on to run Python code, there is nothing special to do, because Python interpreter will automatically relinquish the GIL after a number of instructions have been executed (allowing another thread to acquire it, which will relinquish it again, and so on). Additionally, whenever Python is about to invoke a blocking system call, e.g. to read from the network or write to a file, it will release the GIL around the call.

The original version of this answer pretty much ended here. But there is one more thing to take into account: the embedding scenario.

When embedding Python, the main thread often initializes Python and goes on to execute other, non-Python-related tasks. In that scenario there is nothing that will automatically release the GIL, so this must be done by the thread itself. That is in no way specific to the call that calls PyEval_InitThreads, it is expected of all Python/C code invoked with the GIL acquired.

For example, the main() might contain code like this:

Py_Initialize();
PyEval_InitThreads();

Py_BEGIN_ALLOW_THREADS
... call the non-Python part of the application here ...
Py_END_ALLOW_THREADS

Py_Finalize();

If your code creates threads manually, they need to acquire the GIL before doing anything Python-related, even as simple as Py_INCREF. To do so, use the following:

// Acquire the GIL
PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

... call Python code here ...

// Release the GIL. No Python API allowed beyond this point.
PyGILState_Release(gstate);

More Related Contents:

  1. Remove and Replace Printed items [duplicate]
  2. How to know/change current directory in Python shell?
  3. Accepting only letters as input [closed]
  4. How to check which list an element is contained in
  5. How to remove English words from a file containing Dari words?
  6. How can I make a time delay? [duplicate]
  7. How to merge multiple dicts with same key or different key?
  8. How does asyncio actually work?
  9. Calling a python script with input within a python script using subprocess
  10. Why is a list comprehension so much faster than appending to a list?
  11. Why is parenthesis in print voluntary in Python 2.7?
  12. TypeError: a bytes-like object is required, not ‘str’ in python and CSV
  13. AttributeError: ‘module’ object has no attribute ‘urlopen’
  14. How to deal with multi-level column names downloaded with yfinance
  15. certificate verify failed: unable to get local issuer certificate
  16. TypeError: module.__init__() takes at most 2 arguments (3 given)
  17. Trying to delay a specific function for spawning enemy after a certain amount of time
  18. Python 3, Are there any known security holes in ast.literal_eval(node_or_string)?
  19. What is the fastest and most efficient way to append rows to a DataFrame?
  20. How to share numpy random state of a parent process with child processes?
  21. What is a “callable” in Python?
  22. How can I print multiple things (fixed text and/or variable values) on the same line, all at once?
  23. Why am I getting “invalid syntax” from an f-string? [duplicate]
  24. How can I use newline ‘\n’ in an f-string to format output?
  25. Python decorator? – can someone please explain this? [duplicate]
  26. Check if rows in one dataframe exist in another dataframe
  27. A general tree implementation? [closed]
  28. How to change the voice in pyttsx3?
  29. How to reuse an expression in a comprehension expression?
  30. How to move the background image with keys in pygame?

Leave a Comment