If I flip a coin 1000 times in a row and it lands on heads all 1000 times, what is the probability that it's an unfair coin?
Solution 1:
First of all, you must understand that there is no such thing as a perfectly fair coin, because there is nothing in the real world that conforms perfectly to some theoretical model. So a useful definition of "fair coin" is one, that for practical purposes behaves like fair. In other words, no human flipping it for even a very long time, would be able to tell the difference. That means, one can assume, that the probability of heads or tails on that coin, is $1/2$.
Whether your particular coin is fair (according to above definition) or not, cannot be assigned a "probability". Instead, statistical methods must be used.
Here, you make a so called "null-hypothesis": "the coin is fair". You then proceed to calculate the probability of the event you observed (to be precise: the event, or something at least as "strange"), assuming the null-hypothesis were true. In your case, the probability of your event, 1000 heads, or something at least as strange, is $2\times1/2^{1000}$ (that is because you also count 1000 tails).
Now, with statistics, you can never say anything for sure. You need to define, what you consider your "confidence level". It's like saying in court "beyond a reasonable doubt". Let's say you are willing to assume confidence level of 0.999 . That means, if something that had supposedly less than 0.001 chance of happening, actually happened, then you are going to say, "I am confident enough that my assumptions must be wrong".
In your case, if you assume the confidence level of 0.999, and you have 1000 heads in 1000 throws, then you can say, "the assumption of the null hypothesis must be wrong, and the coin must be unfair". Same with 50 heads in 50 throws, or 20 heads in 20 throws. But not with 7, not at this confidence level. With 7 heads (or tails), the probability is $2 \times 1/2 ^ {7}$ , which is more than 0.001.
But if you assume confidence level at 95% (which is commonly done in less strict disciplines of science), then even 7 heads means "unfair".
Notice that you can never actually "prove" the null hypothesis. You can only reject it, based on what you observe is happening, and your "standard of confidence". This is in fact what most scientists do - they reject hypotheses based on evidence and the accepted standards of confidence.
If your events do not disprove your hypothesis, that does not necessarily mean, it must be true! It just means, it withstood the scrutiny so far. You can also say "the results are consistent with the hypothesis being true" (scientists frequently use this phrase). If a hypothesis is standing for a long time without anybody being able to produce results that disprove it, it becomes generally accepted. However, sometimes even after hundreds of years, some new results might come up which disprove it. Such was the case of General Relativity "disproving" Newton's classical theory.
Solution 2:
If you take a coin you have modified so that it always lands in heads and you get $1000$ heads then the probability of it being unfair is $100\%$.
If you take a coin you have crafted yourself and carefully made sure that it is a fair coin and then you get $1000$ heads then the probability of it being unfair is $0\%$.
Next, you fill a box with coins of both types, then take a random coin.
- $NF$ : fair coins in the box.
- $NU$ : unfair coins in the box
$P(U)$ : probability of having taken an unfair coin $$P(U) = \frac{NU}{NF + NU}$$
$P(F)$ : probability of having taken a fair coin $$ P(F) = \frac{NF}{NF + NU} = 1 - P(U) $$
- $P(H \mid{U})$ : Probability of having 1000 heads conditioned to having take an unfair coin $$P(H\mid{U}) = 1 $$
- $P(H\mid{F})$ : Probability of having 1000 heads conditioned to having taken a fair coin $$P(H\mid{F}) = \left( \tfrac{1}{2} \right)^{1000}$$
- $P(H)$ : Probability of having 1000 heads
\begin{align} P(H) &= P(U \cap H) + P(F \cap H)\\ &= P(H \mid{U})P(U) + P(H \mid{F})P(F)\\ &= P(U) + P(H \mid{F})P(F) \end{align}
By applying Bayes theorem :
$P(U \mid{H})$ : probability of the coin being unfair conditioned to getting 1000 heads $$P(U\mid{H}) = \frac{P(H \mid{U})P(U)}{P(H)} = \frac{P(U)}{P(U) + P(H\mid{F})P(F)}$$
And that is your answer.
In example
If $P(U)=1/(6 \cdot 10^{27})$ ($1$ out of every $6 \cdot 10^{27}$ coins are unfair) and you get 1000 heads then the probability of the coin being unfair is \begin{align} \mathbf{99}.&999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999944\% \end{align}
Very small coins like the USA cent have a weight of $2.5g$. We can safely assume that there are no coins with a weight less than 1 gram.
Earth has a weight of less than $6 \cdot 10^{27}$ grams. Thus we know that there are less than $6 \cdot 10^{27}$ coins. We know that there is at least one unfair coin ( I have seen coins with two heads and zero tails) thus we know that $P(U) \ge 1/(6 \cdot 10^{27})$.
And thus we can conclude that if you get 1000 heads then the probability of the coin being unfair is at least \begin{align} \mathbf{99}.&999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999944\% \end{align}
This analysis is only valid if you take a random coin and only if coins are either $100\%$ fair or $100\%$ unfair. It is still a good indication that yes, with $1000$ heads you can be certain beyond any reasonable doubt that the coin is unfair.
Solution 3:
In order to assign a probability to this event you need to start with a prior probability which can then be updated based on the data. Probabilities can't really be derived from experience alone, you need to start with some sort of "inductive bias" in order to draw conclusions from evidence.
The heuristic approach of hypothesis testing gives a framework for making decisions in these situations, but it doesn't pretend to assign probabilities to those hypotheses.
Solution 4:
In my comment, posted this link to a more thorough treatment of the general question being asked. However, I will directly answer the question using the attained result on posterior density functions, as discussed in the linked article.
In particular, we note that if $r$ is the actual probability that the coin will land on "heads" rather than "tails", then if we got $h$ heads and $t$ tails, the distribution of $r$ is described by the probability density function $$ f(r\mid T=t; H=h) = \frac{(h+t+1)!}{h! \, t!} r^h(1-r)^t $$ Now, let's say that the coin is "fair" if $r$ (which should be exactly $1/2$) falls between $0.45$ and $0.55$. Then the probability that our coin is fair, given $h$ heads and $t$ tails, is given by $$ P = \int_{.45}^{.55}f(r\mid T=t; H=h)\,dr = \frac{(h+t+1)!}{h! \, t!} \int_{0.45}^{0.55} r^h(1-r)^t dr $$ Suppose we get $0$ tails and $h$ heads. Then this probability comes out to $$ P = \int_{.45}^{.55}f(r\mid T=0; H=h)\,dr = (h+1) \int_{0.45}^{0.55} r^h dr = \\ \left. r^{h+1} \right|_{0.45}^{0.55} = (0.55)^{h+1} - (0.45)^{h+1} $$ As you can see, the more consecutive heads we get (assuming this is all the information we have), the less likely it becomes that the coin is fair. For $20$ straight heads, we compute $$ P = (0.55)^{21} - (0.45)^{21} = 3.48 \times 10^{-6} $$ Which is to say that it is extremely unlikely that the coin is fair. The probability that the coin is unfair is, correspondingly, very high.