Closure in the Space of Probability Measures with the Prohorov metric

Solution 1:

The issue is basically that on the space of probability measures, the topology induced by the Prohorov metric is not always the weak* topology. Basically, referring to convergence in the Prohorov metric as "weak convergence" of probability measures is somewhat unfortunate in this sense. If this claim is true, your example of $\Theta = \mathbb{R}$ does not yield a contradiction after all.

To put this into perspective, note the following. Let $C_b(\Theta)$ denote the set of bounded continuous functions on $\Theta$. Define $T_f:\mathcal{P}(\Theta)\to\mathbb{R}$ by $T_f(\mu) = \int f \; d\mu$. Two definitions:

1. A sequence of of probability measures $(\mu_n)$ converges to $\mu$ in the Prohorov metric if and only if $T_f(\mu_n)$ converges to $T_f(\mu)$ for all $f\in C_b(\Theta)$.
2. When $\mathcal{P}(\Theta)$ is a subset of the dual of $C_b(\Theta)$, a sequence of probability measures $(\mu_n)$ converges to $\mu$ in the weak* topology if and only if $T_f(\mu_n)$ converges to $T_f(\mu)$ for all $f\in C_b(\Theta)$.

So, when $\mathcal{P}(\Theta)$ is a subset of the dual of $C_b(\Theta)$, convergence in the Prohorov metric and weak* convergence are the same. However, this is not always the case.

If $(\Theta,d)$ is compact, it holds that $C(\Theta) = C_c(\Theta) = C_b(\Theta)$, i.e. the spaces of continuous functions, continuous functions with compact support and bounded continuous functions are the same, and furthermore, $C(\Theta)' = rca(\Theta)$, where $rca(\Theta)$ is the set of regular Borel measures on $(\Theta,d)$, and so $\mathcal{P}(\Theta)\subseteq rca(\Theta) = C_b(\Theta)$. Thus, in this case, convergence in the Prohorov metric and weak* convergence is the same. However, when $(\Theta,d)$ is not compact, the duality relationships do not always hold. I don't really have a counterexample at hand, but I'm confident that you can find one somewhere.

The above is basically one of the good reasons for having a theory of "weak convergence" for probability measures which is not just a reference to weak* convergence. For probability theory, our main interest is measures which are not supported on compact sets, and therefore we need a convergence theory which holds for such measures. The Prohorov metric and the associated convergence concept based on $C_b(\Theta)$ turns out to be a good solution.