How is the column space of a matrix A orthogonal to its nullspace?

How do you show that the column space of a matrix A is orthogonal to its nullspace?

Solution 1:

What you have written is only correct if you are referring to the left nullspace (it is more standard to use the term "nullspace" to refer to the right nullspace).

The row space (not the column space) is orthogonal to the right null space.

Showing that row space is orthogonal to the right null space follows directly from the definition of right null space.

Let the matrix $A \in \mathbb{R}^{m \times n}$. The right null space is defined as $$\mathcal{N}(A) = \{z \in \mathbb{R}^{n \times 1} : Az = 0 \}$$

Let $ A = \left[ {\begin{array}{c} a_1^T \\ a_2^T \\ \ldots \\ \ldots \\ a_m^T \end{array} } \right]$. The row space of $A$ is defined as $$\mathcal{R}(A) = \{y \in \mathbb{R}^{n \times 1}: y = \sum_{i=1}^m a_i x_i \text{ , where }x_i \in \mathbb{R} \text{ and }a_i \in \mathbb{R}^{n \times 1} \}$$ Now from the definition of right null space we have $a_i^T z = 0$.

So if we take a $y \in \mathcal{R}(A)$, then $y = \displaystyle \sum_{k=1}^m a_i x_i \text{ , where }x_i \in \mathbb{R}$. Hence, $$y^Tz = (\sum_{k=1}^m a_i x_i)^T z = (\sum_{k=1}^m x_i a_i^T) z = \sum_{k=1}^m x_i (a_i^T z) = 0$$

This proves that row space is orthogonal to the right null space. A similar analysis proves that column space of $A$ is orthogonal to the left null space of $A$.

Note: The left null space is defined as $\{z \in \mathbb{R}^{m \times 1}: z^TA = 0\}$

Solution 2:

You can't show this: it is false. Take

$$\begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix};$$

both its column space and null space are the subspace $\{(x, 0) \mid x \in \mathbb{R}\}$ (the $x$-axis, so to speak). I think you need self-adjointness (symmetry in the real case) to get this.

EDIT: Sivaram anticipated me.

Solution 3:

The OP's question would have been well-phrased had he specified that the matrix $A$ is symmetric i.e. $A=A^\top$, in which case ${\rm colspan}(A)={\rm rowspan}(A^\top)={\rm rowspan}(A)$. Now, consider the definition of ${\rm null}(A)$ as the space of all vectors $\mathbf{v}$ such that $A\mathbf{v}=\mathbf{0}$. Letting $\mathbf{a}_1,\ldots,\mathbf{a}_n$ be the rows (and columns) of $A$, matrix multiplication tells us that $\mathbf{a}_i\cdot\mathbf{v}=0$ for each $i=1,\ldots,\dim(A)$. Thus any vector $\mathbf{v}\in{\rm null}(A)$ is orthogonal to ${\rm colspan}(A)$. It follows that ${\rm null}(A)\perp{\rm colspan}(A)$.$\square$