Equivalent Definitions of the Operator Norm

Let $$\begin{align*} I &= \inf\{ c\;\colon\; \lVert Av\rVert\leq c\lVert v\rVert \text{ for all }v\in V\}\\ S_1&=\sup\{ \lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert\leq 1\}\\ S_2&=\sup\{\lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert = 1 \}\\ S_3&=\sup\left\{ \frac{\lVert Av\rVert}{\lVert v\rVert}\;\colon\; v\in V\text{ with }v\neq 0\right\}. \end{align*}$$ Notice that $S_2 \le S_1$ and as $\|Av\| /\|v\| = \| A(v / \|v\|)\|$ we have $S_3 \le S_2$. Now if $\|v\|\le 1$ we have $\|Av\| \le \|Av\| /\|v\|$. Then $S_1 \le S_3$ and $$ S_1=S_2=S_3.$$ Now note that $$ \|Av\| \le S_3 \|v\| \quad \forall v \in V.$$ Then $I \le S_3$ and by definition of $\sup$ we have $$ I \ge \|Av_n\| /\|v_n\| \ge S_3 - 1/n \quad \forall n.$$ Then $S_3 = I$.

Remember that if $s=\sup X$ and $x\leq t$ for all $x\in X$, then $s\leq t$. Also, if $s=\inf X$ and $t\in S$, then $s\leq t$. Now, let us write $$\begin{align*} a&= \inf\{ k\;\colon\; \lVert Av\rVert\leq k\lVert v\rVert \text{ for all }v\in V\},\\ \\ b&=\sup\{ \lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert\leq 1\},\\ \\ c&=\sup\{\lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert = 1 \},\\\\ d&=\sup\left\{ \frac{\lVert Av\rVert}{\lVert v\rVert}\;\colon\; v\in V\text{ with }v\neq 0\right\}. \end{align*}$$

Note that:

• $\|Av\|\leq a\|v\|$ for all $v\in V$. Taking $v\in V$ with $\|v\|\leq 1$ we obtain $\|Av\|\leq a$ and thus $b\leq a$;

• $\|Av\|\leq b$ for all $v\in V$ with $\|v\|\leq 1$. Take $v\in V$ with $\|v\|=1$ and define $v_n=(1-1/n)v$. Since $\|v_n\|=1-1/n\leq 1$, we conclude that $\|Av_n\|\leq b$ for all $n\in\mathbb{N}$. Taking the limit we obtain $\|Av\|\leq b$ and thus $c\leq b$;

• $\|Av\|\leq c$ for all $v\in V$ with $\|v\|= 1$. Taking $v\in V$ with $v\neq 0$, we obtain $\left\|\frac{v}{\|v\|}\right\|=1$. Hence $\frac{\|Av\|}{\|v\|}=\left\|A\left(\frac{v}{\|v\|}\right)\right\|\leq c$ and thus $d\leq c$;

• $\frac{\|Av\|}{\|v\|}\leq d$ for all $v\in V$ with $v\neq 0$. Hence $\|Av\|\leq d\|v\|$ for all $v\in V$ and thus $a\leq d$.

This shows that $a\leq d\leq c\leq b\leq a$ and thus $a=b=c=d$.

I'll give you part of one to give you an idea of the flavor, but you should really do them yourself.

Let $w\neq 0$. Then $\frac{1}{\lVert w\rVert}$ makes sense. Now notice that $$A\left(\frac{1}{\lVert w\rVert}w\right) = \frac{1}{\lVert w\rVert}A(w).$$ Therefore, $$\left\lVert A\left(\frac{w}{\lVert w\rVert}\right)\right\rVert = \frac{\lVert Aw\rVert}{\lVert w\rVert}.$$ But $\frac{w}{\lVert w\rVert}$ is a vector of norm $1$, so...

The four definitions are equivalent only if $V\neq\{0\}$ (which, admittedly, is usually the case).

Reason: When $V=\{0\}$, there is no $v\in V$ such that $\|v\|=1$. So $\sup\varnothing = -\infty$, but we clearly want $\|A\|=0$ if $A$ is the zero operator. This (small but still) mistake can be found in many textbooks.

In summary, only the first two expressions make sense.