Why non-trivial solution only if determinant is zero
The equation $(A−λI)x=0$ has a nontrivial solution (a solution where $x≠0$) if and only if $det(A−λI)=0$.
Why is that?
How can this be proven?
If $\det (A-\lambda I) \neq 0$, then it has an inverse and so the equation has solution $x=(A-\lambda I)^{-1} 0 = 0 $ as its only solution. So in order for any other solution to exist (a non-trivial one, that is) $A-\lambda I$ can't have an inverse. Therefore its determinant is $0$.
Reverse: If $det (A-\lambda I) = 0$ then it has less than full rank. So when you row reduce, you get at least one row of zeros. So the solution has at least one free variable. You can pick the value of the free variable as you please, specifically not $0$, and get a non-trivial solution.