Using function pointers to methods of classes without the gil
You can use with gil:
around the blocks that need the GIL, and then with nogil:
around the important inner blocks that will take most of your run time. To give a trivial example
from cython.parallel import prange
cdef class Simulation:
cdef double some_detail
def __cinit__(self,double some_detail):
self.some_detail = some_detail
def do_long_calculation(self, double v):
with nogil:
pass # replace pass with something long and time-consuming
return v*self.some_detail
def run_simulations(int number_of_simulations):
cdef int n
for n in prange(number_of_simulations,nogil=True):
with gil: # immediately get the gil back to do the "pythony bits"
sim = Simulation(5.3*n)
sim.do_long_calculation(1.2) # and release again in here"
Provided the nogil
section in do_long_calculation
runs from longer than the section where you set up and pass the simulations (which can run in parallel with do_long_calculation
, but not with itself) this is reasonably efficient.
An additional small comment about turning a bound method into a function pointer: you really struggle to do this in Cython. The best workround I have is to use ctypes (or possibly also cffi) which can turn any Python callable into a function pointer. The way they do this appears to involve some runtime code generation which you probably don't want to replicate. You can combine this method with Cython, but it probably adds a bit of overhead to the function call (so make sure do_long_calculation
is actually long!)
The following works (credit to http://osdir.com/ml/python-cython-devel/2009-10/msg00202.html)
import ctypes
# define the function type for ctypes
ftype = ctypes.CFUNCTYPE(ctypes.c_double,ctypes.c_double)
S = Simulation(3.0)
f = ftype(S.do_long_calculation) # create the ctypes function pointer
cdef someFunctionPointer cy_f_ptr = (<someFunctionPointer*><size_t>ctypes.addressof(f))[0] # this is pretty awful!