Probability that a head eventually turns up (From Grimmett and Stirzaker)
This problem is from the exercise section of Grimmett and Stirzaker. I have also listed my proof.
Question.
A fair coin is tossed repeatedly.
(a) Show that a head turns up sooner or later with probability one.
(b) Show similarly that any given finite sequence of heads and tails occurs eventually with probability one.
(c) Explain the connection with Murphy's law.
Proof.
Let $A_{1},A_{2},A_{3},...,A_{n}$ be the events that a head turns up on the $(1,2,3,...,n)$-th toss after a sequence of $n-1$ tails. This can be visualized in the form of binomial tree. The sample space $\Omega=\{H,TH,TTH,TTTH,...\}$ is a countably infinite set.
The probability measure $P$ is countably additive over disjoint sets.
$P(\bigcup\limits_{i=1}^{\infty}{A_{i}})=\sum_{i=1}^{\infty}P(A_{i})$
In this example, all the members of $\mathscr{F}$ are disjoint.
Hence, $P(\bigcup\limits_{i=1}^{\infty}{A_{i}})=P(A_{1})+P(A_{2})+...=(1/2)+(1/2^2)+(1/2^3)+...\infty=1$.
Thus, a head will turn up with a probability 1.
I would like to know, if this proof is correct and sound. Are there alternative approaches to it? Also, I was wondering, how I can extend it to a sequence of heads and tails of length $m$. What is Murphy's law? How do I define the $\sigma$-field $\mathscr{F}$?
Solution 1:
Your proof is correct, and extending it is probably possible, although I don't immediately see a way. An alternate way, which is easier to extend to the second case is the following.
The probability that no heads occur in $N$ tosses is $2^{-N}$. As $N \to \infty$, this goes to $0$. Thus, the probability that at least one head occurs in an infinite experiment is $1$.
This is easier to extend this as follows:
Pick any sequence of $m$ digits. The probability that it doesn't occur in $N$ tosses is upper bounded by the probability that it doesn't occur as sequence $(x_{jm - m}, \dots, x_{jm})$ for $1 \le j \le \lfloor N/m \rfloor$. The latter probability is $\left(\frac{2^m - 1}{2^m}\right)^{\lfloor N/m \rfloor} \to 0$ as $N \to \infty$.
Murphy's law is a tongue in cheek folk-saying that states that anything that can go wrong, will go wrong. Through the above, this is true for, well, coins, assuming some finite sequence makes things wrong for you, and if you're willing to hang around for a while.