Physical meaning of norm of a matrix

I know norm of a vector is a length of a vector from origin. So what is the motivation behind defining the norm of the matrix? What is the physical meaning of norm of a matrix?

Any help is appreciated.

Thanks.

Solution 1:

There are actually multiple ways to assign a norm to a matrix, in fact there are multiple ways to give a norm to a vector. With vectors in $\mathbb{R}^n$ the choice that is most "geometrically appealing" is the Euclidean one

$$\|(x_1,...,x_n)\| = \sqrt{x_1^2 + ... + x_n^2}$$

However, there are others, I'd advise looking up $\ell^p$-norms.

One "obvious" choice for a matrix norm is simply to do a Euclidean norm by summing the squares of the entries and square-rooting that. But the algebra of the situation actually suggests something a little more interesting. That is, let $A$ be a matrix and $x$ a vector of appropriate dimensions, and let $\|x\|$ denote the Euclidean vector norm. Then we give the matrix the "operator norm"

$$\|A\| = \max\limits_{x \in \mathbb{R}^n}\frac{\|Ax\|}{\|x\|}$$

Which represents the max that the matrix $A$ stretches the vector $x$ in some sense. We choose the max so that the norm is positive definite. If $A$ sends any non-zero vector to a nonzero vector (that is, $A$ is nonzero) then $\|A\| > 0$.

Solution 2:

Take a real matrix $A \in \mathbb{R}^{n \times m}$ for example. Now let this matrix represent an input-output relation like $y = Au$. The $p$-norm of $A$ is $\sup\limits_{||u||_p \neq 0}\frac{||y||_p}{||u||_p}$ where $||v||_p$ denotes the $p$-norm of vector $v$. In other words, it is a measure of amplification of the input.

Solution 3:

The picture is particularly simple if you consider a positive self-adjoint matrix. In this case the operator norm is the largest eigenvalue. From the point of view of quantum mechanics, this is the largest possible outcome for the corresponding variable in any given experiment.