Conditional Independence vs Independence of Conditional Expectations

Are the two concepts distinct (is one stronger than the other) or equivalent?

In other words given two random variables $X$ and $Y$ defined on a sigma algebra $\mathcal{F}$ with sub sigma algebra $\mathcal{G} \subset \mathcal{F}$, is $E[X|\mathcal{G}] \perp E[Y|\mathcal{G}]$ (independence of the conditional expectations) equivalent to $X$ and $Y$ being conditionally independent given $\mathcal{G}$ (for an example, let $\mathcal{G} = \sigma(Z)$ for some random variable $Z$)?

What if $X$ and $Y$ were independent? Would that guarantee conditional independence (no!, I have counterexamples $X$ first coin flip, $Y$ second coin flip, $Z = X+Y$) or independence of conditional expectations?

Solution 1:

Consider the following very simple example.

Let $\pi$ $\sim U[0,1]$ or something like that -- a random number between $0$ and $1$. And let $$X=\begin{cases}1&\text{ with probability }\pi\\0&\text{ with probability }1-\pi&\end{cases}$$ and let $$Y=\begin{cases}1&\text{ with probability }1-\pi\\0&\text{ with probability }\pi\end{cases}$$ and make sure that if $\pi$ is given then $X$ and $Y$ realize independently.

That is $$P(X=1\mid \pi\ )=P(Y=0\mid\pi\ )=\pi,\ P(Y=1\mid \pi\ )=P(X=0\mid \pi\ )=1-\pi$$

and $X$ and $Y$ are conditionally independent with respect to $\pi$:

$$P(X=1\cap Y=1\mid \pi\ )=P(X=0\cap Y=0\mid \pi\ )=\pi(1-\pi)$$

$$P(X=0\cap Y=1\mid \pi\ )=(1-\pi)^2$$ $$P(X=1\cap Y=0\mid \pi\ )=\pi^2.$$

Now, compute the conditional expectations:

$$E[\ X\mid \pi\ ]=\pi, \ \text{ and } E[\ Y\mid \pi\ ]=1-\pi.$$

$\pi$ and $1-\pi$ are not independent.

EDIT

Considering independent $X$ and $Y$.

Let $\Omega=\{1,2,3,4,5,6,7,8,9\}$, $\mathscr F=2^{\Omega}$, $P(I)=\frac19$ and let's visualize $\Omega$ lie this:

[![enter image description here][1]][1]

Furthermore let $$X(\omega)=\begin{cases}1&\text{ if }\omega \in\{1,2,3\}\\ 2&\text{ if }\omega \in\{4,5,6\}\\ 3&\text{ if }\omega \in\{7,8,9\} \end{cases}$$ and let $$Y(\omega)=\begin{cases}1&\text{ if }\omega \in\{1,4,7\}\\ 2&\text{ if }\omega \in\{2,5,8\}\\ 3&\text{ if }\omega \in\{3,6,9\}. \end{cases}$$ Check that $X$ and $Y$ are independent.

Now, define $\mathscr G$ as $$\{\Omega,\emptyset, G_1=\{1,2,4,5,7\},G_2=\{3,6,8,9\}\}.$$

In order to get the conditional expectations $E[X\mid \mathscr G\ ]$ , we need to calculate the following conditional probabilities:

$$P(X=i\mid G_1\ )=\frac{P(X=i \cap G_1 )}{P(G_1)}=\begin{cases}\frac35&\text{ for }&i=1\\\frac25&\text{ for }&i=2\\0&\text{ for }& i=3,\end{cases}$$ $$P(X=i\mid G_2\ )=\frac{P(X=i \cap G_2 )}{P(G_2)}=\begin{cases}0&\text{ for }&i=1\\\frac14&\text{ for }&i=2\\\frac34&\text{ if }&i=3.\end{cases}$$ So

$$E[\ X\mid G_1\ ]= 1\times P(X=1\mid G_1\ )+2\times P(X=2\mid G_1)+3\times P(X=3\mid G_1\ )=$$ $$=\frac35+2\times \frac25=\frac75$$ and $$E[\ X\mid G_2\ ]= 1\times P(X=1\mid G_2\ )+2\times P(X=2\mid G_2)+3\times P(X=3\mid G_2\ )=$$ $$=2\times \frac14+3\times \frac34=\frac{11}4$$ then

$$E[X\mid \mathscr G\ ](\omega)=\begin{cases}\frac75&\text{ if }&\omega\in G_1\\\frac{11}4&\text{ if }&\omega\in G_2\end{cases}.$$

In order to get the conditional expectations $E[Y\mid \mathscr G\ ]$ , we need to calculate the following conditional probabilities:

$$P(Y=i\mid G_1\ )=\frac{P(Y=i \cap G_1 )}{P(G_1)}=\begin{cases}\frac25&\text{ for }&i=1\\\frac25&\text{ for }&i=2\\\frac15&\text{ for }& i=3,\end{cases}$$ $$P(Y=i\mid G_2\ )=\frac{P(Y=i \cap G_1 )}{P(G_2)}=\begin{cases}\frac14&\text{ for }&i=1\\\frac14&\text{ for }&i=2\\\frac12&\text{ if }&i=3.\end{cases}$$ So

$$E[\ Y\mid G_1\ ]= 1\times P(Y=1\mid G_1\ )+2\times P(Y=2\mid G_1)+3\times P(Y=3\mid G_1\ )=$$ $$=\frac25+2\times \frac25+3\times \frac15=\frac95$$ and $$E[\ Y\mid G_2\ ]= 1\times P(Y=1\mid G_2\ )+2\times P(Y=2\mid G_2)+3\times P(Y=3\mid G_2\ )=$$ $$=1\times \frac14+2\times \frac14+3\times \frac12=\frac{9}4$$ then

$$E[Y\mid \mathscr G\ ](\omega)=\begin{cases}\frac95&\text{ if }&\omega\in G_1\\\frac{9}4&\text{ if }&\omega\in G_2\end{cases}.$$

As far as the independence of $E[X\mid\mathscr G\ ]$ and $E[Y\mid\mathscr G\ ]$. Consider the following example

$$P\left(E[\ X\mid \mathscr G\ ]=\frac75\right)=P(G_1)=\frac59,\ \ P\left(E[\ Y\mid \mathscr G\ ]=\frac95\right)=P(G_1)=\frac59$$ and $$P\left(E[\ X\mid \mathscr G\ ]=\frac75 \cap E[\ Y\mid \mathscr G\ ]\right)=P(G_1)=\frac59$$ and not $\frac{25}{81}$.