Why is flipping a head then a tail a different outcome than flipping a tail then a head?

Solution 1:

An experiment has "outcomes." These are the observable results of having performed our experiment. For example, when we reach our hand into a bucket and pull out a ball this outcome might be the color of the ball, or the number on the ball, etc...

A sample space of an experiment is the collection of all possible outcomes of that experiment.

An event is a subset of the sample space.

Depending on the experiment, we may have many different possible sample spaces that we can choose from. Some choices will be more beneficial than others. For example, if our experiment was the "result" of a game of monopoly, we could have each outcome describe every moment in time throughout the game... who purchased a property here, who had to pay taxes there, etc... Alternatively, our outcomes could have been much more brief and just taken into account only the winner of the game.

So long as the events you are interested in describing are in fact subsets of your sample space, that is one of the deciding factors of if a sample space is "valid" to use.

That all being said, one of the reasons why you might choose a particular sample space over another is that you want to use counting techniques to calculate probabilities. We have the nice property that if $A$ is an event and $S$ is our finite sample space that in the event that $S$ is "equiprobable", in other words that every outcome is equally likely to occur, that we have

$$Pr(A)=\dfrac{|A|}{|S|}$$

This is a very powerful tool for calculating probabilities and for this reason we often prefer to use sample spaces where each outcome happens with equal chance.

For this reason, when talking about the experiment of flipping two coins, we often prefer to use the interpretation that the coins are flipped in sequence and order mattering, giving us the sample space of $\{HH,~HT,~TH,~TT\}$

Of course, we may choose to use the sample space where we did not keep track of the order in which these events occurred but just kept track of the total number of times we saw heads versus tails. Just keep in mind that if we do this we will sacrifice the ability to use counting techniques to calculate probability. When outcomes in our sample space are not equally likely to occur, we may not calculate the probability using the above formula. As an example, when playing the lottery you either win or you lose, but certainly winning the lottery doesn't occur half of the time.

As a final aside, it is worth mentioning that it is because we like to work with equiprobable sample spaces so much that we will many times reword problems or reimagine problems in such a way as to let us work with an equiprobable sample space.

For example, for the example where we have a bucket with $10$ indistinguishable white balls and $2$ indistinguishable red balls and we reach in and pick one ball at random and ask what the probability is that the ball was red, we in truth have only the two possible outcomes. The ball was white, or the ball was red, and there is no additional information that we can get by looking at the ball.

We can imagine what would happen if each ball were numbered as well with a unique number, suddenly making the balls all distinguishable. This would allow us to change the sample space into something much more favorable, letting the outcome be the number on the ball as well as the color rather than just the color, making each outcome suddenly become equally likely to occur. Now we can see that the probability is $\dfrac{2}{12}$ for the ball to be red.