Kronecker Product Interpretation

The algebraic expression for a Kronecker product is simple enough. Is there some way to understand what this product is?

The expression for matrix-vector multiplication is easy enough to understand. But realizing that the multiplication yields a linear combination of the columns of the matrix is a useful insight.

Is there some analogous insight for the Kronecker product?

Solution 1:

A nice motivating example is provided in Kronecker Products & Matrix Calculus with Applications by A. Graham.

We consider two linear transformations \begin{align*} \boldsymbol{x}=\boldsymbol{A}\boldsymbol{z}\qquad\text{and}\qquad \boldsymbol{y}=\boldsymbol{B}\boldsymbol{w} \end{align*} and the simple case \begin{align*} \begin{pmatrix} x_1\\x_2 \end{pmatrix} = \begin{pmatrix} a_{11}&a_{12}\\ a_{21}&a_{22}\\ \end{pmatrix} \begin{pmatrix} z_1\\z_2 \end{pmatrix} \qquad\text{and}\qquad \begin{pmatrix} y_1\\y_2 \end{pmatrix} = \begin{pmatrix} b_{11}&b_{12}\\ b_{21}&b_{22}\\ \end{pmatrix} \begin{pmatrix} w_1\\w_2 \end{pmatrix} \end{align*}

The Kronecker product:

Now we consider the two transformations simultaneously by defining two vectors \begin{align*} \color{blue}{\boldsymbol{\mu}=\boldsymbol{x}\otimes\boldsymbol{y}} = \begin{pmatrix} x_1y_1\\x_1y_2\\x_2y_1\\x_2y_2 \end{pmatrix} \qquad\text{and}\qquad \color{blue}{\boldsymbol{\nu}=\boldsymbol{z}\otimes\boldsymbol{w}} = \begin{pmatrix} z_1w_1\\z_1w_2\\z_2w_1\\z_2w_2 \end{pmatrix} \end{align*}

We want to find the transformation between $\boldsymbol{\mu}$ and $\boldsymbol{\nu}$. We look at the relation between the components of the two vectors and show it exemplarily for $x_1y_1$.

We obtain \begin{align*} x_1y_1&=\left(a_{11}z_1+a_{12}z_2\right)\left(b_{11}w_1+b_{12}w_2\right)\\ &=a_{11}b_{11}\left(z_1w_1\right)+a_{11}b_{12}\left(z_1w_2\right)+a_{12}b_{11}\left(z_2w_1\right)\\ &\qquad+a_{12}b_{12}\left(z_2w_2\right) \end{align*}

It follows \begin{align*} \color{blue}{\boldsymbol{\mu}}&= \begin{pmatrix} a_{11}b_{11}&a_{11}b_{12}&a_{12}b_{11}&a_{12}b_{12}\\ a_{11}b_{21}&a_{11}b_{22}&a_{12}b_{21}&a_{12}b_{22}\\ a_{21}b_{11}&a_{21}b_{12}&a_{22}b_{11}&a_{22}b_{12}\\ a_{21}b_{12}&a_{21}b_{22}&a_{22}b_{21}&a_{22}b_{22}\\ \end{pmatrix}\boldsymbol{\nu}\\ &\color{blue}{=\left(\boldsymbol{A}\otimes\boldsymbol{B}\right)\boldsymbol{\nu}} \end{align*} We find \begin{align*} \color{blue}{\boldsymbol{Az}\otimes\boldsymbol{Bw} =\left(\boldsymbol{A}\otimes \boldsymbol{B}\right) \left(\boldsymbol{z}\otimes \boldsymbol{w}\right)} \end{align*}