Covariance of two random vectors
IF I have $X,Y_1,...,Y_n$ iid then how do I calculate:
cov $\left [\begin{pmatrix}X\\.\\.\\.\\X \end{pmatrix}, \begin{pmatrix}Y_1\\.\\.\\.\\Y_n \end{pmatrix}\right]$?
Solution 1:
This is known as the cross-covariance between vectors, and is defined by $$ \text{cov}[\boldsymbol{X},\boldsymbol{Y}] = \text{E}[(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T}] $$
where $$ \boldsymbol{\mu_X} = \text{E}[\boldsymbol{X}]\\ \boldsymbol{\mu_Y} = \text{E}[\boldsymbol{Y}] $$
In your case, because all the components of $\boldsymbol{X}$ are the same, things simplify greatly.
$$ \boldsymbol{X} = X \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right], \;\; \boldsymbol{\mu_X} = \mu_X \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right] $$ Where $\mu_X=\text{E}[X]$. Then $$ \boldsymbol{X}-\boldsymbol{\mu_X} = (X-\mu_X) \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right] $$ Now $$ (\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T} = (X-\mu_X) \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right]\left[ \begin{array}{cccc}Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n\end{array} \right] $$ where $\mu_m=\text{E}[Y_m]$ for $m\in[1,2,\cdots,n]$. Expanding out that matrix product we have $$ (\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T} = (X-\mu_X)\left[ \begin{array}{cccc} Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n\\ Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n\\ \vdots&\vdots&\ddots&\vdots\\ Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n \end{array} \right] $$
Taking that scalar inside the matrix, we see it multiplies each entry in the matrix. Then taking the expectation of the result finally gives $$ \text{E}[(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T}] = \left[ \begin{array}{cccc} \text{E}[(X-\mu_X)(Y_1-\mu_1)]&\text{E}[(X-\mu_X)(Y_2-\mu_2)]&\cdots&\text{E}[(X-\mu_X)(Y_n-\mu_n)]\\ \text{E}[(X-\mu_X)(Y_1-\mu_1)]&\text{E}[(X-\mu_X)(Y_2-\mu_2)]&\cdots&\text{E}[(X-\mu_X)(Y_n-\mu_n)]\\ \vdots&\vdots&\ddots&\vdots\\ \text{E}[(X-\mu_X)(Y_1-\mu_1)]&\text{E}[(X-\mu_X)(Y_2-\mu_2)]&\cdots&\text{E}[(X-\mu_X)(Y_n-\mu_n)] \end{array} \right] $$ $$ = \left[ \begin{array}{cccc} \text{cov}(X,Y_1)&\text{cov}(X,Y_2)&\cdots&\text{cov}(X,Y_n)\\ \text{cov}(X,Y_1)&\text{cov}(X,Y_2)&\cdots&\text{cov}(X,Y_n)\\ \vdots&\vdots&\ddots&\vdots\\ \text{cov}(X,Y_1)&\text{cov}(X,Y_2)&\cdots&\text{cov}(X,Y_n) \end{array} \right] $$
Now we are at the answer: you specified all the variables to be identically distributed and independent. Independent variables have covariance $0$. SO, you get the all zeros matrix for your answer $$ \text{cov}(\boldsymbol{X},\boldsymbol{Y})=\text{E}[(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T}] = \left[ \begin{array}{cccc} 0&0&\cdots&0\\ 0&0&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&0\\ \end{array} \right] $$