Expectation and variance of the geometric distribution

Solution 1:

Memorylessness means that either $X=0$, which happens with probability $p$, or that, with probability $1-p$, $X=1+X'$ where $X'$ has the same distribution as $X$. That is,

$$ X=U\cdot(1+X'),\qquad U\sim\mathrm{Ber}(1-p),\qquad U\ \text{independent of}\ X'. $$

This yields every moment of $X$, for example, $E[U]=1-p$ hence $$ E[X]=E[U]\cdot(1+E[X])\implies E[X]=\frac{E[U]}{1-E[U]}=\frac{1-p}p. $$ Likewise, $$ E[X^2]=E[U^2]\cdot E[(1+X)^2]=E[U]\cdot(1+2E[X]+E[X^2]), $$ which implies $$ E[X^2]=\frac{E[U]}{1-E[U]}\cdot\frac{1+E[U]}{1-E[U]}, $$ and $$ \mathrm{var}(X)=\frac{E[U]}{(1-E[U])^2}=\frac{1-p}{p^2}. $$ More generally, for every $x$ in $(0,1]$, $$ E[s^X]=E[s^{U(1+X)}]=p+(1-p)E[s^{1+X}]\implies E[s^X]=\frac{p}{1-(1-p)s}, $$ hence, differentiating $n$ times, $$ E[X(X-1)\cdots(X-n+1)s^X]=n!\,\frac{p(1-p)^n}{(1-(1-p)s)^{n+1}}, $$ and in particular, for $s=1$, $$ E[X(X-1)\cdots(X-n+1)]=n!\,\frac{(1-p)^n}{p^n}, $$ from which every moment of $X$ can be deduced.

Solution 2:

$\newcommand{\var}{\operatorname{var}}$ $\newcommand{\E}{\mathbb E}$

I will consider the geometric distribution supported on the set $\{0,1,2,3,\ldots\}$. This is the distribution of the number $X$ of failures before the first success in a sequence of independent Bernoulli trials. Call the probability of success on each trial $p$.

Then $$ X = \begin{cases} 0 & \text{with probability }p \\ 1 & \text{with probability }p(1-p) \\ 2 & \text{with probability }p(1-p)^2 \\ 3 & \text{with probability }p(1-p)^3 \\ \vdots & {}\qquad \vdots \end{cases} $$

Memorylessness of this distribution means that $\Pr(X\ge w+x\mid X\ge w)=\Pr(X\ge x)$, i.e. the probability distribution of the number of remaining trials, given the number of failures so far, does not depend on the number of failures so far.

Let $\displaystyle A=\begin{cases} 1, & \text{if }X\ge 1 \\[6pt] 0, & \text{if }X=0. \end{cases}$

Then $$ \E(X) = E(E(X\mid A)) = E\left.\begin{cases} 0 & \text{if }A=0 \\ 1+\E(X) & \text{if }A=1 \end{cases}\right\} = p\cdot0+(1-p)(1+\E(X)). $$ Thus we have $$ \E(X) = 1-p+(1-p)\E(X). $$ Therefore $$ \E(X) = \frac{1-p}{p}. $$

Now the variance: $$ \var(X) = \var(\E(X\mid A)) + \E(\var(X\mid A)) $$ $$ = \var\left.\begin{cases} 0 & \text{if }A=0 \\ 1 + \E(X) & \text{if }A=1 \end{cases}\right\} + \E\left.\begin{cases} 0 & \text{if }A=0 \\ \var(X) & \text{if }A=1 \end{cases}\right\} $$ $$ = \var\left.\begin{cases} 0 & \text{if }A=0 \\ 1/p & \text{if }A=1 \end{cases}\right\} + p\cdot0 + (1-p)\var(X) $$ $$ = \frac{1-p}{p} + p\cdot0 + (1-p)\var(X) = (1-p)\left(\frac1p+\var(X)\right). $$ So we get $$ \var(X) = (1-p)\left(\frac1p+\var(X)\right). $$ Therefore $$ \var(X) = \frac{1-p}{p^2}. $$