Embedding python in multithreaded C application

I had exactly the same problem and it is now solved by using PyEval_SaveThread() immediately after PyEval_InitThreads(), as you suggest above. However, my actual problem was that I used PyEval_InitThreads() after PyInitialise() which then caused PyGILState_Ensure() to block when called from different, subsequent native threads. In summary, this is what I do now:

1. There is global variable:

   static int gil_init = 0; 
   

2. From a main thread load the native C extension and start the Python interpreter:

   Py_Initialize() 
   

3. From multiple other threads my app concurrently makes a lot of calls into the Python/C API:

   if (!gil_init) {
       gil_init = 1;
       PyEval_InitThreads();
       PyEval_SaveThread();
   }
   state = PyGILState_Ensure();
   // Call Python/C API functions...    
   PyGILState_Release(state);
   

4. From the main thread stop the Python interpreter

   Py_Finalize()
   

All other solutions I've tried either caused random Python sigfaults or deadlock/blocking using PyGILState_Ensure().

The Python documentation really should be more clear on this and at least provide an example for both the embedding and extension use cases.

Eventually I figured it out.
After

PyEval_InitThreads();

You need to call

PyEval_SaveThread();

While properly release the GIL for the main thread.

Note that the if (!gil_init) { code in @forman's answer runs only once, so it can be just as well done in the main thread, which allows us to drop the flag (gil_init would properly have to be atomic or otherwise synchronized).

PyEval_InitThreads() is meaningful only in CPython 3.6 and older, and has been deprecated in CPython 3.9, so it has to be guarded with a macro.

Given all this, what I am currently using is the following:

In the main thread, run all of

Py_Initialize();
PyEval_InitThreads(); // only on Python 3.6 or older!
/* tstate = */ PyEval_SaveThread(); // maybe save the return value if you need it later

Now, whenever you need to call into Python, do

state = PyGILState_Ensure();
// Call Python/C API functions...    
PyGILState_Release(state);

Finally, from the main thread, stop the Python interpreter

PyGILState_Ensure(); // PyEval_RestoreThread(tstate); seems to work just as well
Py_Finalize()