What is the intuitive meaning of the basis of a vector space and the span?

The formal definition of basis is:

A basis of a vector space $V$ is defined as a subset $v_1, v_2, . . . , v_n$ of vectors in that are linearly independent and span vector space $V$.

The definition of spanning is:

A set of vectors spans a space if their linear combinations fill the space.

But what is the intuitive meaning of this, and the idea of a vector span? All I know how to do is the process of solving by putting a matrix into reduced row-echelon form.

Separately, I"m not sure if I should put this in a new question, but could someone relate this to an intuitive explanation for the row space and column space? So a column space is all the linear combinations of each column of matrix $A$. So what? What does this imply? And a row space is, is it a linear combination of all the rows of $A$, because the book just says its the column space of $A^T$, which I hope means the same thing. So, sure, that's what the definitions of the row space and column space are, but how do all these concepts relate? I'm getting especially confused getting to the fundamental theorems of linear algebra part where we talk about row space, column space, and nullspaces all together.

Solution 1:

Take, for example $V = \mathbb R ^2$, the $x$-$y$ plane. Write the vectors as coordinates, like $(3,4)$.

Such a coordinate could be written as a sum of its $x$ component and $y$ component: $$(3,4) = (3,0) + (0,4)$$ and it could be decomposed even further and written in terms of a "unit" x vector and a "unit" y vector: $$(3,4) = 3\cdot(1,0) + 4\cdot(0,1).$$ The pair $\{(1,0),(0,1)\}$ of vectors span $\mathbb R^2$ because ANY vector can be decomposed this way: $$(a,b) = a(1,0) + b(0,1)$$ or equivalently, the expressions of the form $a(1,0) + b(0,1)$ fill the space $\mathbb R^2$.

It turns out the $(1,0)$ and $(0,1)$ are not the only vectors for which this is true. For example if we take $(1,1)$ and $(0,1)$ we can still write any vector:

$$(3,4) = 3\cdot(1,1) + 1\cdot(0,1)$$ and more generally $$(a,b) = a \cdot (1,1) + (b-a)\cdot(0,1).$$

This fact is intimately linked to the matrix $$\left(\begin{array}{cc} 1 & 1 \\ 0 & 1 \end{array} \right)$$ whose row space and column space are both two dimensional.

I would keep going, but your question is general enough that I could write down an entire linear algebra course. Hopefully this gets you started.

Solution 2:

There is a natural tension between the ideas of linear independence and span. Given a particular $n$ dimensional space, we would like to represent any vector $\mathbf x$ in that space as a linear combination of a set of vectors $V = \{\mathbf v_1,\mathbf v_2, \ldots,\mathbf v_m\}$. So the question is, what type of requirements does this set of vectors need to satisfy for this to be possible? By definition, the set of vectors should span $\mathbb R^n$ meaning that the entire space is reachable with some linear combination of the vectors, so clearly $m\geq n$. Unfortunately the does not yield a definitive test that we can construct $\mathbf x$ from $V$. We can conceive of situations where $V$ contains more than $n$ vectors yet they do not span $\mathbb R^n$ (you should understand that $m\geq n$ is a necessary condition but not a sufficient condition). So if we consider the case where $m > n$ and they span $\mathbb R^n$, then we have too many vectors. How then do we tell which ones are unnecessary? The answer is linear independence. Any vector in $v$ which is a linear combination of the others is unnecessary and can be removed from the set. If we systematically remove all of the linearly dependent vectors from the set $V$, we then have a linearly independent set of vectors which span $\mathbb R^n$. That is a basis. A basis is both linearly independent (it doesn't have too many vectors) and it spans the space (it has enough vectors). Thus the basis strikes a balance between span and linear independence.

Regarding column and row space, you should understand that a multiplication of a matrix times a vector can be interpreted in two different ways. Consider the following $$\mathbf{y=Ax}$$ where $\mathbf{A}$ is $m\times n$ and $\mathbf{x}$ is $n\times 1$. You may interpret this multiplication as

1. The linear combination of the columns of $\mathbf{A}$, or
2. The set of dot products of the rows of $\mathbf{A}$ with the vector $\mathbf{x}$.

Both viewpoints are valid. In view (1), if we think about the columns of $\mathbf{A}$ (column space) forming a basis, we can think about multiplication as a composition of the basis vectors to form a new vector $\mathbf y$. In view (2), if we think about the rows of $\mathbf{A}$ (row space) forming a basis, we can think about the multiplication $\mathbf{Ax}$ as the representation of $\mathbf x$ in that basis (i.e. we are projecting the vector $\mathbf x$ onto each row vector).

Now consider the case where the rows of $\mathbf A$ do not span $\mathbb R^n$. Then there are some values of $\mathbf x$ which are orthogonal to the entire row space. This is called the null space. The fundamental theorem of linear algebra says that the dimension of the row space plus the dimension of the null space is $n$. Can you see why that is? It is because $\mathbb R^n$ is spanned by the row space and the null space, however every vector in the row space is orthogonal to every vector in the null space.

Solution 3:

Here is an explanation of spanning in terms of linear equation solving: A system of $n$ linear equations with $m$ variables may be written in the form $$a_1 x_1 + ... + a_m x_m = b$$ where $a_1,...,a_m$ and $b$ are columns of length $n$ (When the equation is given in matrix form $Ax=b$, then $a_1,...,a_m$ are just the columns of the matrix $A$). And taking a second look at this form, $b$ is written as a linear combination of the $a_j$.

This means that $m$ vectors $a_1,...,a_m$ of $\mathbb{R}^n$ span $\mathbb{R}^n$ if and only if the system of equations $Ax=b$ (where $A$ is the matrix formed by $a_1,...,a_m$ as columns) has a solution for all right-hand sides $b\in \mathbb{R}^n$.