$\left \| \cdot \right \|$ is an induced norm. If $\left \| A \right \|<1$, how to show that $I-A$ is nonsingular and ...?

Solution 1:

It can be made quite simple. If $\|A\| < 1$ and you assume by contradiction that $(I-A)$ is singular, then there exists $x \neq 0$ such that $Ax=x$ and without loss of generality, you can scale $x$ so that $\|x\|=1$. But then $\|Ax\| = 1$ and therefore $\sup_{\|x\|=1} \|Ax\| \geq 1$, which is a contradiction.

For the second part, by definition $\|(I-A)^{-1}\| = \sup_{\|y\|=1} \|(I-A)^{-1}y\| = \sup_{\|y\|=1} \|x_y\|$ where $x_y := (I-A)^{-1}y$, i.e., $y = (I-A) x_y$. Now $$ 1 = \|y\| = \|(I-A) x_y\| = \|x_y - A x_y\| \geq \|x_y\| - \|A x_y\| \geq \|x_y\| - \|A\| \|x_y\| = (1 - \|A\|) \|x_y\|. $$ Finally, $\|x_y\| \leq 1/(1 - \|A\|)$ and so is the $\sup$.

Solution 2:

$I - A$ is nonsingular means that for $x \not= 0$, $(I - A)x \not= 0$. But expanding that, you get just $x - Ax \not= 0$, i.e. $x \not= Ax$.

So $I - A$ is nonsingular precisely when $A$ does not have any fixed points. Now, $\| Ax \| \le \| A \| \| x \|$ (prove this if you don't already know it), so if $\| A \| < 1$ then $\| Ax \| < \| x \|$, so $A$ can't possibly have any fixed points.

Basically, if you have $\| A \| < 1$ then $A$ makes every vector shorter, so $-Ax$ can't make it all the way back to the origin from $x$.

For the second part, I would use (again, if you don't know these you should prove them) $\| AB \| \le \| A \| \| B \|$ and $\|A + B\| \le \|A\| + \|B\|$ (so in particular, set $B = I - A$ and you get $\| I \| - \|A \| \le \| I - A \|$)