Stein's lemma condition

(Apologies if I break some conventions, this is my first time posting!)

I am working on proving Stein's characterization of the Normal distribution: for Z $\sim N(0,1)$ and some differentiable function $f$ with $E[|f'(Z)|] < \infty$, $$E[Zf(Z)] = E[f'(Z)]$$ Writing the LHS expression in integral form and integrating by parts, I eventually obtain: $$E[Zf(Z)] = \frac{1}{\sqrt{2\pi}} \left[ -f(z) \cdot \exp \left\{ \frac{-z^2}{2} \right\} \right] \Bigg|_{-\infty}^{\infty} + E[f'(Z)]$$ Now I need to show that the first expression on the right hand size is zero. Intuitively, this seems clear because of the exponential term, but I am having trouble explicitly applying the condition on $f'$ to prove this rigorously. Any ideas?

Solution 1:

I know I'm a few years late to the party, but I'm not sure about Dougal's and soren's solutions.

In Dougal's solution, the replacement of $\phi(x)$ by its maximum $\phi(0)$ in the denominator results in a smaller quantity rather than a larger one.

In soren's solution, I don't understand why $f'(Z)$ having finite expectation would imply that $f$ is Lipschitz. What if $f'(z) = z$ for instance? Then $f'(Z)$ has finite expectation but $f(z) = z^2/2$ isn't Lipschitz.

Stefan's solution seems fine, but I agree with gogurt that a more "direct" attack on the product term of the integration by parts might be informative.

Almost every source I could find takes the Casella and Berger approach, saying something like "it can be shown that the product term is zero." Or some of them give Stefan's proof. Finally, I found some course notes from a class taught by Sourav Chatterjee with a terse proof that the product term is zero. See Lemma 2.

Edit: The idea from Chatterjee's notes seems to be that you can show that $\mathbb{E}|f(Z)|$ is finite, which means that its integrand $f(z) \phi(z)$ must approach zero as $z \rightarrow \pm \infty$ if those limits exist (according to this).

He first shows that $\mathbb{E} |Z f(Z)|$ is finite: \begin{align*} \int_{-\infty}^\infty |zf(z)| \phi(z) dz &\leq \int_{-\infty}^\infty |z| \left[|f(0)| + |f(z) - f(0)|\right] \phi(z) dz\\ &\leq \int_0^\infty z \left[\int_0^z |f'(t)| dt\right] \phi(z) dz + \int_{-\infty}^0 (-z) \left[\int_z^0 |f'(t)| dt\right] \phi(z) dz + |f(0)| \sqrt{2/\pi}\\ &= \int_0^\infty |f'(t)| \underbrace{\int_t^\infty z \phi(z) dz}_{\phi(t)} dt + \int_{-\infty}^0 |f'(t)|\underbrace{ \int_{-\infty}^t (-z) \phi(z) dz}_{\phi(t)} dt + |f(0)| \sqrt{2/\pi}\\ &= \int_{\infty}^\infty |f'(t)| \phi(t) dt + |f(0)| \sqrt{2/\pi}\\ &= \mathbb{E} |f'(Z)| + |f(0)| \sqrt{2/\pi} \end{align*} Finally, take expectations of both sides of the pointwise inequality \begin{align*} |f(Z)| \leq \sup_{|t| \leq 1} |f(t)| + |Zf(Z)| \end{align*} The continuity of $f$ ensures that its supremum on $[-1, 1]$ is finite.

Solution 2:

Let $Z\sim \mathcal{N}(0,1)$ and $f$ a differentiable function with $E[|f'(Z)|]<\infty$. Then $$ \begin{align} E[Zf(Z)]&=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}zf(z)\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}zf(z)\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz-f(0)E[Z]\\ &=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}z\left[f(z)-f(0)\right]\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz\\ &=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}z\left[\int_0^zf'(u)\,\mathrm du\right]\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz. \end{align} $$ On the other hand $$ \begin{align} E[f'(Z)]=&\int_{-\infty}^\infty f'(z)\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz\\ =&\frac{1}{\sqrt{2\pi}}\int_{-\infty}^0f'(z)\left[\int_{-\infty}^z-u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz\\ +&\frac{1}{\sqrt{2\pi}}\int_0^\infty f'(z)\left[\int_z^\infty u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz. \end{align} $$ So let us treat these two integrals seperately and use Fubini's theorem (justified by the assumption): $$ \int_0^\infty f'(z)\left[\int_z^\infty u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz=\int_0^\infty\int_z^\infty f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm du\,\mathrm dz\\ =\int_0^\infty \int_0^u f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du $$ and similarly $$ \begin{align} &\int_{-\infty}^0f'(z)\left[\int_{-\infty}^z-u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz=\int_{-\infty}^0\int_{u}^0f'(z)(-u)\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du\\ &=\int_{-\infty}^0\int_{0}^u f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du \end{align} $$ and thus $$ E[f'(Z)]=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}u\left[\int_{0}^u f'(z)\,\mathrm dz\right]\exp\left(-\frac{u^2}{2}\right)\,\mathrm du=E[Zf(Z)] $$

Solution 3:

This might be too late in the game but here is an answer to Gogurt's original question, that shows that the assumptions on $f$ imply that $f(z)\phi(z)$ vanishes at infinity.

Student45 establishes that $f$ and $zf(z)$ are integrable with respect to the normal distribution using Fubini's theorem. This implies that $f'(z)\phi(z)+ f(z)z\phi(z)=(\phi(z)f(z))'$ is Lebesgue integrable in $\mathbb{R}$. In particular, $$ f(z)\phi(z) = f(0)\phi(0) + \int^z_0 (\phi\cdot f)'(t)dt $$ for all $z$. This is due to the Fundamental Theorem of Calculus that says that if $f$ is differentiable in an interval $[a,b]$ and if $f'$ is Lebesgue integrable in $[a,b]$, then $$f(x)-f(a)=\int^x_af'(t)\,dt,\quad a\leq x\leq b.$$ Therefore, the limits $\lim_{z\rightarrow\pm\infty}f(z)\phi(z)=A_{\pm}$ exist. As $f(z)\phi(z)$ is Lebesgue integrable in $\mathbb{R}$, then one must have that $A_{\pm}=0$ as Student45 mentioned in his comment above.