Integral of probability density over a Borel set

I have a random variable $\xi : \Omega\to\mathbb{R}$ which distribution function has a density, so by definition I have that the probability measure of each $(-\infty,x]$ can be calculated by:

$$F_\xi(x)=P_\xi (-\infty,x]=\int_{-\infty}^x f_\xi(y)\mathrm{d}y \quad (1)$$

where the integral above is in the Lebesgue sense, with respect to the Lebesgue measure in $ \mathbb{R} $.

My book (Shiryayev, Probability, pg. 195) says that a wider formula holds, that is:

$$ P_\xi(B)=\int_B f_\xi \mathrm{d}x, \quad \forall B\in\mathcal{B}(\mathbb{R}) $$

How can I use (1) in order to obtain this last formula? In other words, how can I extend (1) to every Borel set?

Solution 1:

If suffices to show that the equality holds for sets of the form $\{(a,b]:a<b\}$. Indeed, $$ \mathsf{P}_{\xi}((a,b])=\mathsf{P}_{\xi}((-\infty,b])-\mathsf{P}_{\xi}((-\infty,a])=\int_a^b f_{\xi}(y)\,dy. $$ Now, let $\mathcal{C}:=\{B\in\mathcal{B}(\mathbb{R}):\mathsf{P}_{\xi}(B)=\int_B f_{\xi}(y)\,dy\}$. It is a monotone class that contains the above-mentioned sets. Thus, $\mathcal{C}=\sigma(\{(a,b]:a<b\})=\mathcal{B}(\mathbb{R})$ (see Theorem 1 in Section 2 of the textbook).

Solution 2:

This is by uniqueness of extension of probability measures, for which Dynkin's $\pi$-$\lambda$ theorem is one of the go-to hammers. It is such a useful result that I would highly suggest you look up the proof (it's one of those proofs which are simple after you see it, but you end up wondering how anyone even thought of it... atleast fro me), and commit the result to memory. A standard corollary of Dynkin's theorem is the following uniqueness result:

Theorem.

Suppose $(S,\mathscr{A})$ is a measurable space, and $\mu,\nu$ are probability measures defined on $\mathscr{A}$, and suppose $\mathcal{P}$ is a $\pi$-system which generates the $\sigma$-algebra $\mathscr{A}$, i.e $\mathscr{A}=\sigma(\mathcal{P})$. If $\mu,\nu$ agree on $\mathcal{P}$, then $\mu,\nu$ agree on the full $\sigma$-algebra $\mathscr{A}$.

To see how to apply this theorem to your specific case, consider $S=\Bbb{R}$, with $\mathscr{A}$ being the Borel $\sigma$-algebra of $\Bbb{R}$. Define $\mu=\Bbb{P}_{\xi}$ to be the distribution of your random variable, and define $\nu(B)=\int_{B}f_{\xi}\,dx$. So, $\mu,\nu$ are both probability measures defined on $\mathscr{A}$ (to see that $\nu$ is indeed a proabbility measure, you need to know $f_{\xi}\geq 0$ a.e, and you need to invoke the monotone convergence theorem).

Now, consider $\mathcal{P}=\{(-\infty,x]\,:\, x\in\Bbb{R}\}$, the collection of all right-closed rays. This is a $\pi$-system because clearly the intersection of such intervals is again another such interval. Also, $\mathcal{P}$ generates the Borel $\sigma$-algebra (this is a standard exercise: show that it generates all open intervals, hence all open sets, therefore the full Borel $\sigma$-algebra). By hypothesis, $\mu,\nu$ agree on $\mathcal{P}$, so by the theorem, they agree on the full Borel $\sigma$-algebra $\mathscr{A}$.

For completeness, here are the definitions: let $S$ be a non-empty set.

• A $\pi$-system on $S$ is by definition a set $\mathcal{P}\subset \text{power set of $S$}$, which is closed under finite intersections (i.e $A,B\in \mathcal{P}\implies A\cap B\in \mathcal{P}$).
• A $\lambda$-system on $S$ is by definition a collection $\mathcal{L}\subset \text{power set of $S$}$, such that three conditions are satisfied: $\emptyset\in \mathcal{L}$, and $\mathcal{L}$ is closed under complements, and $\mathcal{L}$ is closed under countable disjoint unions.

The reason why $\pi$-systems and $\lambda$-systems are useful to work with is that usually, we can take $\pi$-systems to be very small, in the sense that it has very few sets, but still generates the entire $\sigma$-algebra (e.g half-infinite intervals in our example above). The reason why we like $\lambda$-systems is that due to the disjoint union condition, it is literally begging to be somehow related to measures (because measures are by definition countably additive).

Dynkin's theorem says the following: if $\mathcal{P}$ is a $\pi$-system on $S$ and $\mathcal{L}$ is a $\lambda$-system on $S$ which contains $\mathcal{P}$, then it contains the $\sigma$-algebra generated by $\mathcal{P}$. In symbols, $\mathcal{P}\subset \mathcal{L}\implies \sigma(\mathcal{P})\subset \mathcal{L}$.

Let us see how the above theorem about uniqueness follows from Dynkin's $\pi$-$\lambda$ theorem. Define $\mathcal{L}:= \{A\in\mathscr{A}\,:\, \mu(A)=\nu(A)\}$. Then,

• Clearly $\mu(\emptyset)=\nu(\emptyset)=0$, so $\emptyset\in \mathcal{L}$.
• If $A\in\mathcal{L}$, then $\mu(A^c)=1-\mu(A)=1-\nu(A)=\nu(A^c)$, so $A^c\in\mathcal{L}$.
• If $\{A_n\}_{n=1}^{\infty}\subset \mathcal{L}$ is a disjoint collection of sets, then $\mu\left(\bigcup_{n=1}^{\infty}A_n\right)=\sum_{n=1}^{\infty}\mu(A_n)=\sum_{n=1}^{\infty}\nu(A_n)=\nu\left(\bigcup_{n=1}^{\infty}A_n\right)$, so $\bigcup_{n=1}^{\infty}A_n\in\mathcal{L}$.

This proves $\mathcal{L}$ is a $\lambda$-system. By hypothesis, it contains the $\pi$-system $\mathcal{P}$. So, from Dynkin's $\pi$-$\lambda$ theorem we get $\mathscr{A}=\sigma(P)\subset \mathcal{L}\subset \mathscr{A}$. Hence, $\mathcal{L}=\mathscr{A}$, i.e the two measures agree on the entire $\sigma$-algebra.

So, the only thing missing from my answer is an actual proof of Dynkin's $\pi$-$\lambda$ theorem itself. But this can be found in quite a few textbooks, or you can just google it.

I should also mention that Dynkin's theorem is not the only approach. Another common measure-theory result is the monotone-class lemma, which can also prove many of the results Dynkin's theorem can. Also, see Folland's real analysis text, theorem 1.14 where there is a uniqueness-of-extension proof under slightly different hypotheses (but still applicable in your case).