Demystifying the Determinant
One of my favorite explanations is this one from Ask A Mathematician. Basically starting off of the idea of finding the volume of the parallelogram that any transformation takes the unit cube, it creates from scratch the definition of determinant. I found this extremely amazing because usually determinants are introduced in the "here's how you calculate it" way and then later shown to be the volume of the transformed unit cube, but this resource starts from the very intuitive notion of "volume of the transformed unit cube" and builds the formula from there, based on some very important but still intuitive properties. Brilliant!
This "volume of transformed unit cube" idea is why it's important in inconsistent systems and invertible transformations/matrices; if the determinant is $0$ (i.e it takes the unit cube and squishes it into a lower dimension), then that means that the transformation is not invertible (not injective) and hence may not have solutions or has infinitely many solutions.
For the cross product, this resource may be illuminating: Motivation for construction of cross-product (Quaternions?). The links therein of course also provide more information, but I personally really like the one I explicitly linked. I'm not entirely sure why determinants are involved, but understanding the formula for the cross product may help in that quest for understanding. If someone has ideas, please comment or answer! I would love to know too.
And finally for eigenvalues, we want to find vectors and eigenvalues such that $A\vec v = \lambda \vec v$, which is equivalent to finding vectors in the null space of $(A-\lambda I)$. Like I said in part $(b)$, determinants are really good at telling us if the unit cube has been squished to a lower dimension (i.e. a non-zero vector has been sent by the transformation to $0$), so it's good at telling us what values of $\lambda$ give us matrices that have a non-zero null space, and hence telling us the eigenvectors/eigenspaces corresponding to a particular $\lambda$.
I might be adding more details later, but if you have questions on what I've written so far, feel free to ask.
P.S. if you want a more formal introduction to determinants and eigenvalues, one of my favorite linear algebra books has been Linear Algebra Done Right, by Sheldon Axler. See chapter 10 for determinants and chapter 5 for eigenvalues. It really sheds light on alternate ways of thinking about these things (outside of the common approach of "here's a crap ton of matrices, do some computations with them")
P.P.S I also wonder a lot about these "origin" questions in mathematics, and if you're interested in more, I'll share some awesome stuff that I've found over eons of prowling MSE:
- Physical interpretation of Laplace transforms
- Could *I* have come up with the definition of Compactness (and Connectedness)?
The answers in all the things I've linked above are just so helpful for any student of mathematics I just couldn't resist sharing!
If one defines the determinant of a square matrix $M$ to be the signed volume of the result of applying $M$ to the unit cube (i.e. the region spanned by the unit vectors), then one can observe:
- $\det(MN) = \det(M)·\det(N)$ for any square matrices $M,N$ of the same size.
- $\det(D)$ is the product of the diagonal entries of $D$ for any diagonal matrix $D$, because $D$ is the transformation matrix of a scaling.
- $\det(P) = 0$ for any matrix $P$ that has only $0$ in some row, because $P$ is the transformation matrix of a projection that collapses everything into some hyperplane with some coordinate zero.
- $\det(R) = -1$ for any matrix $R$ that is obtained from $I_k$ by swapping two rows, because $R$ is the transformation matrix of a reflection that swaps two coordinate axes.
- $\det(E) = 1$ for any matrix $E$ that has only $1$ on its diagonal and only one non-diagonal entry, because $D$ is the transformation matrix of an axis-parallel shear.
Now it is not hard to show that every matrix $M$ can be expressed (via Gaussian elimination) as a product $A_1 A_2 \cdots A_k B$ where each $A_i$ is of the form (4) or (5) above, and $B$ is of the form (2) or (3) above. Thus $\det(M)$ can be computed in the process of Gaussian elimination, and indeed is most efficiently done that way.
Moreover, if $B$ is of form (2), then $M$ is invertible, since all matrices of form (2) or (4) or (5) are invertible. But if $B$ is of form (3), then $M$ is not invertible since $B$ will map some input vectors to the same output vector (this can be easily observed from the row-echelon form of $B$). Thus $M$ is invertible iff $\det(M) ≠ 0$. Which should really be expected, since intuitively $M$ is invertible iff it does not flatten the unit cube, and once flattened no linear transformation can unflatten it. If $M$ is not invertible, then the equation $Mx = y$ may have no solution or infinitely many solutions, depending on whether the range of $M$ contains $y$ or not, respectively.
Furthermore, it is easy to check that the Leibniz definition of determinant satisfies all the above properties, and hence must be the same. The Leibniz definition can hence be motivated by the desire to have a rigorous definition that coincides with the 'intuitive geometric definition'. This is captured by the algebraic property of the determinant being an "alternating multilinear map with respect to the columns" as described by the wikipedia article. Here "multilinear" corresponds to (2)+(3)+(5), and the "alternating" qualifier corresponds loosely to (4).
The cross product is peculiar to $3$ dimensions, and celtschk has already explained how one can arrive at its relation to the determinant of $3×3$ matrices.
As for eigenvectors of a $k×k$ matrix $M$, we want to find some nonzero vector $v$ and scalar $λ$ such that $Mv = λv$, equivalently $(M-λI_k)v = 0$, which requires $(M-λI_k)$ to be not invertible, which as explained above is equivalent to $\det(M-λI_k) = 0$.
The beautiful interpretation of the determinant as a signed volume gets so much attention that sometimes I worry new students won't realize that the determinant can easily be discovered just by solving $Ax = b$ by hand, using high school algebra, in the case where $A$ is a $2 \times 2$ or $3 \times 3$ matrix. For example, if $A$ is $2 \times 2$, we want to solve \begin{align} a_{11} x_1 + a_{12} x_2 &= b_1 \\ a_{21} x_1 + a_{22} x_2 &= b_2. \end{align} Multiplying both sides of the the first equation by $a_{21}$, and then multiplying both sides of the second equation by $a_{11}$ and subtracting, we find that $$ (a_{11} a_{22} - a_{21} a_{12}) x_2 = a_{11} b_2 - a_{21} b_1. $$ If the number $a_{11} a_{22} - a_{21} a_{12}$ is not zero, then we can solve for $x_2$. (And we find a similar formula for $x_1$.)
So immediately, we have discovered the determinant of a $2 \times 2$ matrix, and we see that if this special number is not zero then $Ax = b$ has a unique solution for any choice of $b$. The case where $A$ is $3 \times 3$ is not much harder. Someone who is sufficiently obsessed, as many mathematicians are, would not hesitate to work out the $4 \times 4$ case by hand if the pattern is not already clear. So, there's nothing difficult about discovering the determinant. (By the way, we have also discovered Cramer's rule in the process.)
If $\lambda$ is an eigenvalue of $A$, this means that there exists some nonzero vector $x$ such that $Ax = \lambda x$. Equivalently, $$ \tag{1} (A - \lambda I) x = 0, $$ where $I$ is the identity matrix.
But, as we have discovered above, if $\det (A - \lambda I) \neq 0$, then (1) has a unique solution, which is $x = 0$. So the fact that there is a connection between eigenvalues and the determinant is immediate. You might conjecture that if $\det (A - \lambda I) = 0$ then (1) has a nonzero solution, and it turns out to be possible to show that this is correct. So, $\lambda$ is an eigenvalue of $A$ if and only if $\det (A - \lambda I) = 0$.