Is it true that $E_{\theta} \left ( \frac{\partial}{\partial \theta} \log p_{\theta} (X) \right ) = 0$?
I already found the answer in another statistics book! We have: \begin{align*} E _{\theta} \left ( \frac{\partial}{\partial \theta } \log p_{\theta} (X) \right ) & = E _{\theta} \left ( \frac{1}{p_{\theta} (X) } \frac{\partial }{\partial \theta} p_{\theta} (X) \right ) \\ & = \int p_{\theta} (x) \frac{1}{p_{\theta} (x) } \frac{\partial }{\partial \theta} p_{\theta} (x) dx \\ & = \int \frac{\partial }{\partial \theta} p_{\theta} (x) dx \\ & = \frac{d}{d\theta} \int p_{\theta} (x) dx \\ & = \frac{d}{d\theta} 1 \\ & = 0. \end{align*} If someone wants to delete this question, then that is ok.