Why is observing 100 heads for a fair coin flips surprising?

Solution 1:

You are right. It is not that surprising. Each specific pattern has probability $\frac{1}{2^{100}}$. The probability to obtain $100$ heads is the same as obtaining any other particular outcome.

But usually we do not ask for a specific pattern but ask for the probability to obtain e.g. $k$ heads and this makes the difference.

• There is just $\color{blue}{\binom{100}{100}=1}$ pattern to obtain $100$ heads out of $2^{100} \sim 1.3\cdot 10^{30}$.

• But we have $\color{blue}{\binom{100}{50}\sim 1.0\cdot 10^{29}}$ patterns which contain $50$ heads and $50$ tails.

Solution 2:

If the sequences that you consider "irregular" constitute the vast majority of all possible sequences, then the probability that the sequence you get is irregular is very high, and it's surprising if you get one of the relatively few "regular" sequences.

One could make the concept of "irregular" precise using something like Kolmogorov complexity. Most bit strings of a given length have no description shorter than the string itself; the ones that admit short descriptions are relatively few.

Solution 3:

Assume a fair coin. The idea of "surprising" means it's against our "expectations". The distinction is what is our "expectation"?

• If it were a specific exact sequence of heads and tails, then the all heads sequence is just as likely as any other specific sequence, $2^{-100}$. This is a very rare thing to "expect".

• More likely our expectation is specified as something like "roughly an even number of heads and tails", which is not one specific sequence of coin tosses but the event of seeing one of many, many sequences. We can calculate with the binomial theorem or approximating a Normal distribution that the probability of "seeing roughly and even number of heads and tails" is very high. For example, the probability of seeing 40-60 heads in our 100 tosses is $$2^{-100} \sum_{i=40}^{60} \binom{100}{i} \approx (1.22 \times 10^{30}) / (1.26 \times 10^{30}) \approx 96.4\% $$

Solution 4:

You're asking for a specific sequence of coin tosses of length 100. Sure, any such sequence is as rare as all others, but there are $2^{100}$ of them (all heads, all tails, HTHTHTHTHTH...HTHT, HHHTTTHHHTTT...TTH, etc). So picking one specific sequence means getting it right FOR EVERY SINGLE THROW. This is almost impossibly improbable ($P(\omega) = \frac{1}{2^{100}}$). If you were able to do that in a casino, starting from a "double-or-nothing" bet of 1$ you'd own all the money, debt contracts, resources, and companies in the world long before the end of the coin tosses.