What are the sample spaces when talking about continuous random variables?

When talking about continuous random variables, with a particular probability distribution, what are the underlying sample spaces?

Additionally, why these sample spaces are omitted oftentimes, and one simply says r.v. $X$ follows a uniform distribution on the interval $[0,1]$? Isn't the sample space critically important?

Solution 1:

You can take it to be a subset of $\mathbb{R}$ or, more generally, $\mathbb{R}^n$. A random variable uniformly distributed in $[0, 1]$ can be thought of as a random variable on the sample space $[0, 1]$ with probability density function $1$.

The sample space is in fact not critically important. (You may find it convenient to pick one to do computations in, but it doesn't matter which one you pick. This is analogous to choosing coordinates to do computations in linear algebra.) This point is explained very clearly in Terence Tao's notes here:

At a purely formal level, one could call probability theory the study of measure spaces with total measure one, but that would be like calling number theory the study of strings of digits which terminate. At a practical level, the opposite is true [emphasis added]: just as number theorists study concepts (e.g. primality) that have the same meaning in every numeral system that models the natural numbers, we shall see that probability theorists study concepts (e.g. independence) that have the same meaning in every measure space that models a family of events or random variables. And indeed, just as the natural numbers can be defined abstractly without reference to any numeral system (e.g. by the Peano axioms), core concepts of probability theory, such as random variables, can also be defined abstractly, without explicit mention of a measure space; we will return to this point when we discuss free probability later in this course.

Terence Tao's explanation of free probability is here. (I found it very enlightening; the framework Tao describes can be used to describe quantum probability with very little modification, unlike the measure theory framework.)

Solution 2:

While I certainly cannot do better than Tao (and admit that I haven't read Qiaochu's links) I'd like to make the following point:

I see the reason for essentially restricting to $(\Omega,\Sigma, \mu) = [0,1]$ as a rather technical one. If one would like to have sequences of i.i.d. random variables, one needs to be able to form the product space $(\Omega,\mu)^{\mathbb{N}}$ - you need some form of Kolmogorov's consistency theorem. If you allow too nasty $\Omega$'s product spaces can be very degenerate. There are examples in Halmos's or Neveu's books where they show that a countable product can be carrying the trivial measure even if the factors themselves are far from trivial. The point is that $\Omega$ might simply be "too big" to be reasonable.

A very flexible and technically useful class of "reasonable" measurable spaces is the class of standard Borel spaces (for which Kolmogorov's consistency theorem holds, luckily). By definition these are the measurable spaces which are (measurably) isomorphic to the Borel $\sigma$-algebra of a complete and separable metric space. Here's the surprise (Hausdorff, von Neumann):

Every uncountable standard Borel space is isomorphic to $[0,1]$ with the Borel $\sigma$-algebra. Moreover, every non-atomic probability measure on a standard Borel space is equivalent to Lebesgue-measure on $[0,1]$.

So from this point of view there is essentially no restriction in assuming $\Omega$ to be $[0,1]$ to begin with. Of course, atoms are not really an issue. Since we deal with a probability space, there are at most countably many of them, so we'll just get a union of an interval and some countable set.

Solution 3:

Begin with a probability space $(\Omega,\mathcal{F},P)$ ($\mathcal{F}$ is called a $\sigma$-algebra on $\Omega$, $P$ is called a probability measure). The collection of all Borel sets of $\mathbb{R}$ is denoted by $\mathcal{B}(\mathbb{R})$. A mapping $X:\Omega \to \mathbb{R}$ is a (real-valued) random variable if it is $\mathcal{F}$-measurable, that is, $\{ \omega \in \Omega: X(\omega) \in B \}$ is in $\mathcal{F}$ for each $B \in \mathcal{B}(\mathbb{R})$. Write $P[\{ \omega \in \Omega: X(\omega) \in B \}]$ as $P[X \in B]$. As a mapping of $B$, this is a probability measure on $\mathcal{B}(\mathbb{R})$, which can be denoted as $P_X$ and called the distribution of $X$.

Now, to say that a random variable $X$ on $(\Omega,\mathcal{F},P)$ follows a uniform distribution on the interval $[0,1]$ simply means that $P_X$ is the measure on $\mathcal{B}(\mathbb{R})$ satisfying $P_X (\mathbb{R}-[0,1]) = 0$ and $P_X (I) = b-a$ for any interval $I \subset [0,1]$ with endpoints $a<b$. For the simplest example, take the probability space $(\Omega,\mathcal{F},P) = ([0,1],\mathcal{B}([0,1]),P)$, where $P$ is the restriction to $[0,1]$ of the measure just defined above, and define $X:\Omega \to \mathbb{R}$ by $X(\omega) = \omega$. Then, for any $B \in \mathcal{B}(\mathbb{R})$, $$ P_X (B) = P[X \in B] = P[\{ \omega \in [0,1]:X(\omega ) \in B\} ] = P[[0,1] \cap B], $$ from which it follows that $X$ is uniform on $[0,1]$. (Note that it is not essential that $\Omega$ be the set $[0,1]$.)