Intuitive explanation of the Fundamental Theorem of Linear Algebra

Solution 1:

Imagine a projection, for example, from the whole $\mathbb{R}^3$ to the $x$-$y$ plane. It compress each line that parallels to $z$-axis to a point on the plane. So there is a one-to-one relationship between the lines and the points. Notice that the lines are the translations of the $z$-axis -- that is also what quotient means. And the $z$-axis, is just the kernel of the projection, so we can see that $\operatorname{im} A\simeq V/\ker A$.

As for the dimension, the dimension of $\ker A$ measures how much we compress, while the dimension of $\operatorname{im} A$ measures how much we leave -- the amount we compress, plus the amount we leave, equals the whole thing, intuitively.

Solution 2:

The theorem is fundamental because it rests upon two elementary ideas for solving for vector $x$ in a linear homogeneous algebraic equation $Ax=0$.

The fundamental theorem of linear algebra (FTLA) has two parts, each originating from simple ideas in college algebra, especially the topic of linear algebraic equations in the case of infinitely many solutions.

A quick example of a $3 \times 3$ homogeneous linear algebraic system in scalar form:

$$ \begin{cases} &x + 2y + 3z = 0, \\ &0=0, \\ &0=0. \\ \end{cases} $$

The system has infinitely many solutions, expressed in terms of the lead variable $x$ and free variables $y,z$. The general scalar solution is

$$ \begin{align} x &= - 2t - 3s, \\ y &= t, \\ z &= s. \\ \end{align}$$

where $t,s$ are invented symbols for the free variables.

IDEA 1: In the equation $Ax=0$, the number, $n$, of components in vector $x$ equals the lead variable count plus the free variable count.

IDEA 2: Any solution $x$ of $Ax=0$ is perpendicular to every row of $A$.

Idea 1 is called the rank-nullity theorem, part 1 of the FTLA. The rank is the lead variable count and the nullity is the free variable count.

Idea 2 is part 2 of the FTLA, presented in textbooks as a deeper result using the notation of kernel, image and orthogonal complement. The FTLA part 2 is opaque. But Idea 2 is not itself opaque: it is visible from the definition of matrix times vector (the meaning of $Ax$). In short, you can derive the result in Idea 2 by looking at the equation $Ax=0$, almost no background required.

When ready to make the jump to the abstract results, consider reading Gilbert Strang's explanations. Here are links,

• The Fundamental Theorem of Linear Algebra
• The Four Fundamental Subspaces: 4 Lines

Solution 3:

Suppose $V$ and $W$ are finite dimensional inner product spaces over $F$ (where $F$ is $\mathbb R$ or $\mathbb C$). Let $T:V \to W$ be a linear transformation.

There are four subspaces that are naturally associated with $T$: $N(T), R(T), N(T^*)$, and $R(T^*)$. (Here $T^*$ is the adjoint of $T$. So $\langle T(x), y \rangle = \langle x, T^*(y) \rangle$ for all $x \in V, y \in W$.)

What Strang calls the fundamental theorem of Linear Algebra is the fact that \begin{equation} V = N(T) \perp R(T^*) \end{equation} and \begin{equation} W = N(T^*) \perp R(T). \end{equation}

This theorem is easy to prove: \begin{align} & x \in N(T) \\ \iff & T(x) = 0 \\ \iff & \langle T(x),y \rangle = 0 \forall y \in W \\ \iff & \langle x, T^*(y) \rangle = 0 \forall y \in W \\ \iff & x \in R(T^*)^{\perp}. \end{align} This shows that $N(T)$ is the orthogonal complement of $R(T^*)$. Similarly, $N(T^*)$ is the orthogonal complement of $R(T)$.

It's natural to seek bases for these four subspaces, and perhaps the "most natural" or nicest bases are the ones that arise in the SVD of $T$.

Edit: It's interesting that, as @WillieWong pointed out, a similar theorem can be formulated in a more general setting, without using inner product spaces.

Let $V$ and $W$ be finite dimensional vector spaces over a field $F$ and let $T:V \to W$. There are four subspaces naturally associated with $T$: $N(T), R(T), N(T^*)$, and $R(T^*)$. (Now $T^*$ is the dual transformation $T^*:W^* \to V^*$.)

We may not have an inner product to work with, but we can still write \begin{equation} \langle T^*(w^*),v \rangle = \langle w^*,T(v) \rangle \end{equation} if we use the notation \begin{equation} \langle x^*,y \rangle := x^*(y) \end{equation} for $x^* \in V^*, y \in V$.

Moreover, $N(T)$ may not have an orthogonal complement, but it does have an annihilator, which is the analogous thing in a more general setting. And we have that $R(T^*)$ is the annihilator of $N(T)$, and also that $N(T^*)$ is the annihilator of $R(T)$. This is a more general form of Strang's "fundamental theorem".

Note that for any subspace $U$ of $V$, we have \begin{equation} \text{dim} \, U + \text{dim} \,U^{\perp} = \text{dim} \, V \end{equation} where $U^{\perp}$ is the annihilator of $U$. In particular, \begin{equation} \text{dim} \, N(T) + \text{dim} \, R(T^*) = \text{dim} \, V. \end{equation} But we also know that \begin{equation} \text{dim} \, N(T) + \text{dim} \, R(T) = \text{dim} \, V. \end{equation} This shows that \begin{equation} \text{dim} \, R(T) = \text{dim} \, R(T^*). \end{equation}

These easy but fundamental theorems and proofs appear in chapter 2 of Lax's book Linear Algebra and its Applications. (See theorems 5, 5', and 6.)