Find $E[S]$ where $S$ is the standard deviation of a random sample from a $N(\mu,\sigma^2)$ population.

Recall that for a $N(\mu,\sigma^2)$ population $W=\frac{n-1}{\sigma^2}S^2\sim \chi^2(n-1)$.

[a] Find $E[S]$ where $S$ is the standard deviation of a random sample from a $N(\mu,\sigma^2)$ population.

[b] Use your answer to construct an unbiased estimator of $\sigma$.

[c] Use your unbiased estimator to estimate $\sigma$ based on a random sample of size 7 with a sample standard deviation of 5.

Okay, I am having a hard time with part [a]. Knowing that $\frac{n-1}{\sigma^2}S^2\sim \chi^2(n-1)$, it was easy for me to show that $E[S^2]=\sigma^2$. But finding $E[S]$ is a problem. I would like to simply find $E[\sqrt{S^2}]$, but I don't think that's the road I need to go down. I used Jensen's Inequality to show that $E[S]>\sigma$, but that is not enough. Any help would be much appreciated. Hopefully if I can solve [a], it shouldn't be too hard to complete parts [b] and [c].

Solution 1:

If $\frac{n-1}{\sigma^2} S^2 \sim \chi^2(n-1)$ is chi-squared distributed with $(n-1)$ degrees of freedom, then we can write

$$\frac{n-1}{\sigma^2} S^2 = \sum_{j=1}^{n-1} X_j^2$$

where $X_j \sim N(0,1)$ are iid random variables. From

$$S = \sqrt{\frac{\sigma^2}{n-1} \sum_{j=1}^{n-1} X_j^2}.$$

we conclude

$$\begin{align*} \mathbb{E}(S) &= \sqrt{\frac{\sigma^2}{n-1} } \int \sqrt{x_1^2+\ldots+x_{n-1}^2} \, d\mathbb{P}_{X_1,\ldots,X_{n-1}}(x_1,\ldots,x_{n-1}) \\ &= \sqrt{\frac{\sigma^2}{n-1} } \int \!\! \ldots \!\! \int \sqrt{x_1^2+\ldots+x_{n-1}^2} \prod_{j=1}^{n-1} \frac{1}{\sqrt{2\pi}} \exp \left(- \frac{x_j^2}{2} \right) \, dx_1 \ldots dx_{n-1} \\ &= \sqrt{\frac{\sigma^2}{n-1} } \frac{1}{\sqrt{2\pi}^{n-1}} \int \ldots \int \sqrt{x_1^2+\ldots+x_{n-1}^2} \exp \left(- \frac{x_1^2+\ldots+x_{n-1}^2}{2} \right) \, dx_1 \ldots dx_{n-1}. \end{align*}$$

Introducing generalized $(n-1)$-dimensional polar coordinates, we find

$$\begin{align*} &\quad \int \ldots \int \sqrt{x_1^2+\ldots+x_{n-1}^2} \exp \left(- \frac{x_1^2+\ldots+x_{n-1}^2}{2} \right) \, dx_1 \ldots dx_{n-1} \\ &= \sigma_{n-2} \int_0^{\infty} r \exp \left(-\frac{r^2}{2} \right) r^{n-2} \, dr \\ &= \sigma_{n-2} \sqrt{2}^{n-2} \int_0^{\infty} s^{\frac{n-2}{2}} e^{-s} \, ds \\ &= \sigma_{n-2} 2^{\frac{n-2}{2}} \Gamma \left( \frac{n}{2} \right) \end{align*}$$

where $$\sigma_{n-2} = 2\pi^{\frac{n-1}{2}} \frac{1}{\Gamma \left( \frac{n-1}{2} \right)}$$ denotes the surface of the $(n-1)$-dimensional sphere. Combining both equalities yields

$$\mathbb{E}(S) = \sqrt{\frac{2\sigma^2}{n-1} } \frac{\Gamma \left( \frac{n}{2} \right)}{\Gamma \left( \frac{n-1}{2} \right)}.$$

Remark: A close look on the above calculation reveals that the density of the distribution of $\sqrt{\frac{n-1}{\sigma^2}} S$ equals

$$p(x) = \frac{2^{1-(n-1)/2}}{\Gamma \left( \frac{n-1}{2} \right)} x^{n-2} e^{-x^2/2}.$$

Using this density, we can easily calculate higher moments $\mathbb{E}(S^k)$.