Sigma algebra - motivation in measure theory
Let me take a sidestep. Trying to understand $\sigma$ algebra through Vitali sets and Lebesgue measure, etc. is, in my opinion, the wrong approach. Let me offer a much, much simpler example.
A probability space is a measure space $(X, \sigma, \mu)$ where $\mu(X)=1$.
What's the simplest thing you can model with probability theory? Well, the flip of a fair coin. Lets write out all the measure theoretic details.
What are the possible outcomes? Well you can get a heads, $H$, or a tail, $T$. Measure theoretically, this is the set $X$. That is, $X=\{H,T\}$.
What are the possible events? Well first of all nothing can happen or something can happen. This is a given in ANY probability space (or measure space). Also, you can get a heads or you can get a tail. This is the $\sigma$ algebra. That is, $\sigma=\{\emptyset, X, \{H\}, \{T\}\}$.
What is the (probability) measure? Well what are the probailities? First, what is the probability that NOTHING happens? Well, $0$. That is, $\mu(\emptyset)=0$. What is the probability that SOMETHING happens? Well, $1$. That is, $\mu(X)=1$ (this is what makes something a probability space). What is the probability of a heads or a tail? Well $1 \over 2$. That is, $\mu(\{H\})=\mu(\{T\})=\frac12$.
This is just filling out all the measure theoretic details in a very simple situation.
Okay. In this situation EVERY subset of $X$ is measurable. So still, who gives a damn about $\sigma$ algebras? Why can't you just say "everything is measurable" (which is a perfectly fine $\sigma$ algebra for every set, including $\Bbb{R}$!!) and totally forget this business about $\sigma$ algebras? Well lets very slightly modify the previous example.
Lets say you find a coin, and you're not sure if it's fair or not. Lets talk about about the flip of a possibly unfair coin and fill in all the measure theoretic details.
Again, what are the possible outcomes? Again $X=\{H,T\}$.
Now here is where things are interesting. What is the $\sigma$ algebra for this coin flip (the events)? This is where things differ. The $\sigma$ algebra is just $\{\emptyset, X\}$. Remember, a $\sigma$ algebra is the DOMAIN of the measure $\mu$. Remember that we don't know if the coin is fair or not. So we don't know what the measure (probability) of a heads is! That is, $\{H\}$ is not a measurable set!
What is the measure? $\mu(\emptyset)=0$ and $\mu(X)=1$. That is, something will happen for sure and nothing won't happen. What a remarkably uninformational measure.
Here is a situation where the $\sigma$ algebras ARE different because they describe two different situations. A good way to think about $\sigma$ algebras is "information", especially with probability spaces.
Lets connect this back to Vitali sets and Lebesgue measure.
AFAIK, I can have a sigma-algebra that contains Vitali set. Everything can be a sigma-algebra, as long as it satisfies its 3 simple axioms. So if there is something you could add to clarify the quoted explanation, I'd be very grateful.
You are quite right. You CAN have a $\sigma$ algebra of $\Bbb{R}$ that contains every Vitali set. Just like you can have a $\sigma$ algebra of $\{H,T\}$ that contains $\{H\}$. But the point is that you might not be able to assign this a measure in a satisfactory way! Meaning, that if you have a list of properties that a you want Lebesgue measure wants to satisfy, a Vitali set necessarily can't satisfy them. So you "don't know" what measure to assign to it. It is not measurable with respect to a specified measure. You can create all sorts of measures where Vitali sets are measurable (for example one where the measure of everything is $0$).
Let me motivate the axioms of a $\sigma$ algebra in terms of probability.
The first axiom is that $\emptyset, X \in \sigma$. Well you ALWAYS know the probability of nothing happening ($0$) or something happening ($1$).
The second axiom is closed under complements. Let me offer a stupid example. Again, consider a coin flip, with $X=\{H, T\}$. Pretend I tell you that the $\sigma$ algebra for this flip is $\{\emptyset, X, \{H\}\}$. That is, I know the probability of NOTHING happening, of SOMETHING happening, and of a heads but I DON'T know the probability of a tails. You would rightly call me a moron. Because if you know the probability of a heads, you automatically know the probability of a tails! If you know the probability of something happening, you know the probability of it NOT happening (the complement)!
The last axiom is closed under countable unions. Let me give you another stupid example. Consider the roll of a die, or $X=\{1,2,3,4,5,6\}$. What if I were to tell you the $\sigma$ algebra for this is $\{\emptyset, X, \{1\}, \{2\}\}$. That is, I know the probability of rolling a $1$ or rolling a $2$, but I don't know the probability of rolling a $1$ or a $2$. Again, you would justifiably call me an idiot (I hope the reason is clear). What happens when the sets are not disjoint, and what happens with uncountable unions is a little messier but I hope you can try to think of some examples.
I hope this cleared things up.
What are the 'privileged subsets of X'? Borel sets for instance? Is it that because all Borel sets form the smallest sigma algebra (smallest meaning it has the least elements among all sigma algebras containing all Borel sets), and Borel sets have the nice property that they are made of open intervals, which means we can assign size to those sets that is equal to the length of that interval. That's obviously one of many possible measures we can use.
The 'privileged subsets of X' are the measurable sets, where measurable means that they are in the domain of the chosen measure. Of course the Borel measure is an option, as is Lebesgue measure for the reals; the term is context-dependent.
AFAIK, I can have a sigma-algebra that contains Vitali set. Everything can be a sigma-algebra, as long as it satisfies its 3 simple axioms.
This is true. More precisely, for any collection of subsets of $X$ there is a unique minimal $\sigma$-algebra containing all the sets in the collection.
But that's not what we really want. A $\sigma$-algebra is nice, but we also need a measure on that $\sigma$-algebra, and this is where Vitali sets cause issues. If $\mu$ satisfies
- $\mu$ is a (countably additive) measure,
- $\mu$ is translation invariant,
- $\mu([0,1])<\infty$, and
- $\mu$ measures a Vitali set,
then it must necessarily be trivial (measure everything as $\mu(A)=0$). So although there are such "large" $\sigma$-algebras, they are not equipped with any nice, nontrivial measures, so we don't get as much use out of them.