Closed form solution of $\displaystyle\arg\min_{\alpha \in \Bbb R} \|X - \alpha Y\|_{\text F}^2$

Given $m \times n$ matrices $X$ and $Y$, I am interested in the following least-squares problem.

$$\hat \alpha := \arg\min_{\alpha \in \Bbb R} \|X - \alpha Y\|_{\text F}^2$$

Is there any way to express $\hat \alpha$ in terms of $X$ and $Y$?

Solution 1:

In your problem, the matrix structure of $X$ and $Y$ doesn't matter and we can instead interpret them as vectors $x,y \in \mathbb{R}^{mn}$. The optimization problem is

$$\arg \min_{\alpha \in \mathbb{R}} \|x - \alpha y\|.$$

If $y \neq 0$, then the solution of the optimization problem is obviously such $\alpha$ that $\alpha y$ is the orthogonal projection of $x$ onto $y$, i.e.,

$$\alpha y = \left\langle x, \frac{y}{\|y\|} \right\rangle \frac{y}{\|y\|},$$ which gives us

$$\alpha = \frac{\langle x, y \rangle}{\|y\|^2}.$$

Solution 2:

Let $\phi(\alpha) = \| \mathbf{X} - \alpha \mathbf{Y}\|_F^2 $. The differential writes $$ df = 2 (\mathbf{X} - \alpha \mathbf{Y}) : (- d\alpha \mathbf{Y}) = 2 \left[ (\alpha \mathbf{Y}-\mathbf{X}):\mathbf{Y}\right] d\alpha $$ The derivative is $$ \frac{\partial \phi}{\partial \alpha} = 2(\alpha \mathbf{Y}-\mathbf{X}):\mathbf{Y} $$ Setting the derivative to zero yields the relation $ \hat \alpha (\mathbf{Y}:\mathbf{Y}) = \mathbf{X}:\mathbf{Y} $ from which you can deduce $$ \hat \alpha = \frac{\mathbf{X}:\mathbf{Y}}{\| \mathbf{Y}\|_F^2} $$ This is analogous to the solution from CrabMan but in matrix form.