Is the pseudoinverse matrix the solution to the least squares problem?

linear-algebra matrices inverse least-squares pseudoinverse

I'm trying to verify that, given a matrix M, the pseudo-inverse $$M^{+}=(M^TM)^{-1}M^T$$ is the solution for the least squares.. but something went wrong and I can't undestand why...

$$e=\frac{1}{2}||y-Mx||=\frac{1}{2}(y-Mx)^T(y-Mx)\\ =\frac{1}{2}(y^Ty-y^TMx-x^TM^Ty+x^TM^TMx)=\\ =\frac{1}{2}(y^Ty-2y^TMx+x^TM^TMx) $$

so $$\frac{de}{dx}=\frac{1}{2}(-2y^TM+x^TM^TM)=0\\ x^TM^TM=2y^TM\\M^TMx=2M^Ty\\x=2(M^TM)^{-1}M^Ty$$

why can't I cancel the factor '2'?


Solution 1:

For a symmetric matrix $A$, the derivative of $x^\top A x$ with respect to $x$ is $2Ax$. You can prove this directly.

Using this fact you get $M^\top y = M^\top M x$ which yields the answer.

Related

Convergence of $\sum \frac{(2n)!}{n!n!}\frac{1}{4^n}$
Is there any way to prove this assumption?
For a prime ideal $\def\p{\mathfrak{p}}\p\subset A$ and a ring map $A\to B$, what is $B_\p/\p B_\p$? Can $\mathfrak{p}$ be a prime ideal of $B$?
Probability of exceeding a value with a given expectation and variance?
Gelfand formula for the spectral radious.
Arithmetic structure including both unique factorization and Dedekind domains
Line integral of a force field
Definition of the cotensor in sSet
Overcomplete matrix eigendecomposition
Prove that $\bigcup_{x\in (1,\infty)}[\frac{1}{x},x)=(0, \infty)$
To find all integers $n > 1$ for which $(n-1)!$ is a zero-divisor in $Z_n$.
Is the derivative of a differentiable Lipschitz function also Lipschitz?