Query about the Moore Penrose pseudoinverse method
I have recently discovered the Moore-Penrose psuedoinverse method, and I am currently testing the waters with it. I noticed if I have a system, say
$$a_1x_1=0$$ $$a_2x_1+a_3x_2=0$$ $$\vdots$$ $$a_5x_1+a_6x_2=0$$
we can write it in matrix form as $$AX=\hat{0}$$ where $$A=\begin{bmatrix}a_1 & 0\\ a_2 & a_3 \\ \vdots & \vdots\\ a_5& a_6 \end{bmatrix},\qquad X= [x_1\;\; x_2],\quad \hat{0}=\begin{bmatrix}0\\ \vdots\\ 0\end{bmatrix}$$
after doing all the necessary calculations it does indeed find an (left) inverse. My question is upon observation if $a_1\neq 0 $ then clearly $x_1=0$ is a solution and this would fix $x_2=0$, but the solutions obtained via the Moore-Penrose pseudoinverse method doesn't give these trivial solutions. Would the left inverses obtained via this method indeed be the correct solutions to this system?
There is confusion in either my understanding or the question. Start with a full column rank matrix, $$ \mathbf{A} \in \mathbb{R}^{m\times 2}_{2} $$ with $m>2$. The span of the row space is the entire plane: $$ \color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)} = \mathbb{R}^{2}. $$ The colors distinguish $\color{blue}{range}$ spaces from $\color{red}{null}$ space.
Next, you construct the Moore-Penrose pseudoinverse, $\mathbf{A}^{\dagger}$. Recall this pseudoinverse is a projector onto the range space $\color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)}$.
Problems:
- It seems you are looking for a $\color{red}{null}$ space solution with a $\color{blue}{range}$ space tool.
- There is no $\color{red}{null}$ space $\color{red}{\mathcal{N} \left( \mathbf{A} \right)}$ for this problem.
Perhaps this toy problem will help. $$ \begin{align} \mathbf{A} x & = b \\ % \left[ \begin{array}{cc} 1 & 0 \\ 0 & 0 \\ \end{array} \right] % \left[ \begin{array}{c} x_{1} \\ x_{2} \\ \end{array} \right] % &= % \left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right] % \end{align} $$ The Moore-Penrose pseudoinverse is $$ \mathbf{A}^{\dagger} = % \left[ \begin{array}{cc} 1 & 0 \\ 0 & 0 \\ \end{array} \right]. $$
Case 1: No existence
$$\left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right] =\left[ \begin{array}{c} 0 \\ b_{2} \\ \end{array} \right]$$
The data vector is in the null space: $b\in\color{red}{\mathcal{N}\left( \mathbf{A}^{*}\right)}$. There is no solution.
Case 2: Existence and uniqueness
$$\left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right] =\left[ \begin{array}{c} b_{1} \\ 0 \\ \end{array} \right]$$
The exclusion is $b_{1}\ne0$. There is no $\color{red}{null}$ space: $\color{red}{\mathcal{N}\left( \mathbf{A}^{*}\right)} = \mathbf{0}$. The data vector is in the $\color{blue}{range}$ space: $b\in\color{blue}{\mathcal{R}\left( \mathbf{A}\right)}$. The solution exists and is unique. The direct solution is also the least squares solution $$ x = x_{LS} = \left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right] $$
Case 3: Existence, no uniqueness
$$\left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right] =\left[ \begin{array}{c} b_{1} \\ b_{2} \\ \end{array} \right]$$
he exclusion is $b_{1}\ne0$, $b_{2}\ne0$. Now there is a $\color{red}{null}$ space: $\color{red}{\mathcal{N}\left( \mathbf{A}^{*}\right)}$. The data vector inhabits both $\color{blue}{range}$ and $\color{red}{null}$ space: $$ b = \color{blue}{b_{\mathcal{R}}} + \color{red}{b_{\mathcal{N}}} $$ A solution exists and it is not unique. There is no direct solution. The least square solution includes an arbitrary vector $y\in\mathbb{R}^{2}$: $$ x_{LS} = \color{blue}{\mathbf{A}^{\dagger}b} + \color{red}{\left( \mathbf{I}_{2} - \mathbf{A}^{\dagger}\mathbf{A}\right)^{-1} y} = \color{blue}{ \left[ \begin{array}{c} b_{1} \\ 0 \\ \end{array} \right]} + \alpha \color{red}{ \left[ \begin{array}{c} 0 \\ 1 \\ \end{array} \right]} $$