If $X \perp Y|Z$, does this mean that $X \perp Y|Z, W$? How about the other way around?

Suppose $X, Y, Z, W$ are random variables. Let $\perp$ denote independence. $f$ denotes the probability density function. For example, $f(X|Z)$ is the conditional pdf of $X$ given $Z$.

Does $X \perp Y|Z \Rightarrow X \perp Y|Z, W$?

If $X \perp Y|Z$, then $f(X, Y|Z) = f(X|Z)f(Y|Z)$. Now I want to see if $X \perp Y|Z, W$. However, I don't think it's true that $f(X, Y|Z, W) = f(X|Z,W)f(Y|Z, W)$

Therefore, $X \perp Y|Z \nRightarrow X \perp Y|Z, W$.

Does $X \perp Y|Z, W \Rightarrow X \perp Y|Z$?

If $X \perp Y|Z, W$, then $f(X, Y|Z, W) = f(X|Z, W)f(Y|Z, W)$.

\begin{align*} f(X, Y|Z) &= \int_W f(X, Y|Z, W)f(W|Z) dW\\ &= \int_W f(X|Z, W) f(Y|Z, W)f(W|Z) dW\\ &\neq f(X|Z)f(Y|Z) \end{align*}

Therefore, $X \perp Y|Z, W \nRightarrow X \perp Y|Z$.

Is the above correct?

Solution 1:

A very simple (and IMHO more fundamental) way to see the answers:

Since in your examples, everything is conditioned on $Z$, we might as well consider the case of everything NOT conditioned on $Z$, because conditioning on $Z=z$ just restricts the sample space.

So your question is equivalent to asking: $X \perp Y \implies X \perp Y \mid W$? And the converse. Both of which are clearly false.

There is no real need to bring in Gaussians or Rademachers or anything like them.

Solution 2:

Both are correct.

Counterexample for the first implication: $Z=1$ (constant random variable), $Y$ Rademacher variable, $W\sim\mathcal{N}(0,1)$ (namely, standard normal) independent of $Y$ and $X=WY$. Then $X\perp Y|Z$ but $X\not\perp Y|Z,W$.

Counterexample for the second implication: $Z=1$, $(U,V,W)$ are i.i.d. standard normal, $X=W+U$ and $Y=W+V$.